From Fedora Project Wiki
 
(30 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Name resolution ==
== Overview ==


=== Resolving using <code>getaddrinfo()</code> in applications ===
Name resolution topics:
 
* [[/DNS]]
* [[/DNSSEC]]
 
Host and services are often called by names which need to be translated to IP addresses and port numbers.
 
Input arguments:
 
* node
* service
* filters (address family, socktype, protocol)
* flags
 
On output, we expect a list of address records that can be used for connection and other purposes, each consisting of the following fields:
 
* address family and data (network layer)
* protocol and port (transport layer)
* socktype (mode of operation)
 
Additional information can be provided:
 
* canonical name
 
== Use cases ==
 
* Generic translation between hostnames and IP addresses (e.g. for ACL purposes)
* Preparing for connect() or sendto() to contact a service
* Preparing for bind() to provide a service
 
== Name resolution methods ==
 
* [[Networking/NameResolution/Hosts|files (/etc/hosts)]]
* [[Networking/NameResolution/Hosts|myhostname]]
* [[Networking/NameResolution/DNS|DNS]]
* [[Networking/NameResolution/mDNS|mDNS]]
 
== Using getaddrinfo() ==
 
=== Connecting to services using <code>getaddrinfo()</code> ===


The <code>getaddrinfo()</code> function is a dualstack-friendly API to name
The <code>getaddrinfo()</code> function is a dualstack-friendly API to name
Line 19: Line 58:
     .ai_family = AF_UNSPEC,
     .ai_family = AF_UNSPEC,
     .ai_socktype = SOCK_STREAM,
     .ai_socktype = SOCK_STREAM,
     .ai_protocol = SOL_TCP,
     .ai_protocol = IPPROTO_TCP,
     .ai_flags = 0,
     .ai_flags = AI_ADDRCONFIG,
     .ai_canonname = NULL,
     .ai_canonname = NULL,
     .ai_addr = NULL,
     .ai_addr = NULL,
Line 100: Line 139:
service = "http"
service = "http"
family = socket.AF_UNSPEC
family = socket.AF_UNSPEC
socktype = socket.SOCK_DGRAM
socktype = socket.SOCK_STREAM
protocol = socket.SOL_TCP
protocol = socket.SOL_TCP
flags = 0
flags = 0
Line 132: Line 171:
* AI_NUMERICSERV: use numeric service, don't perform service resolution
* AI_NUMERICSERV: use numeric service, don't perform service resolution
* AI_CANONNAME: save canonical name to the first result
* AI_CANONNAME: save canonical name to the first result
* AI_ADDRCONFIG: this never really worked, as far as I know
* AI_ADDRCONFIG: to be used with connect(), this never really worked, as far as I know, but [[Networking/NameResolution/ADDRCONFIG|we want to fix it]]
* AI_V4MAPPED+AI_ALL: only with AF_INET6, return IPv4 addresses mapped into IPv6 space
* AI_V4MAPPED+AI_ALL: only with AF_INET6, return IPv4 addresses mapped into IPv6 space
* AI_V4MAPPED: I don't see any real use for this, only returns mapped IPv4 if there are no IPv6 addresses
* AI_V4MAPPED: I don't see any real use for this, only returns mapped IPv4 if there are no IPv6 addresses


==== Using <code>getaddrinfo()</code> for multiple transport channels ====
==== Connecting to multiple transport channels ====


Some applications need to open several TCP or UDP channels to the same host. The
Some applications need to open several TCP or UDP channels to the same host. The
Line 145: Line 184:


1) Run <code>getaddrinfo()</code> once per channel. This may for example cause
1) Run <code>getaddrinfo()</code> once per channel. This may for example cause
multiple DNS requests for the same information.
multiple DNS requests for the same information, which is suboptimal


2) Run <code>getaddrinfo()</code> only for the main (or first) channel. When
2) Run <code>getaddrinfo()</code> only for the main (or first) channel. When
Line 262: Line 301:
case the NULL host. But it's more of a hack.
case the NULL host. But it's more of a hack.


3) Support full name name resolution. Always create one socket per
3) Support full name resolution. Always create one socket per
resulting address. Set IPV6_V6ONLY on IPv6 sockets to suppres the
resulting address. Set IPV6_V6ONLY on IPv6 sockets to suppres the
transparent dualstack. This works for all cases.
transparent dualstack. This works for all cases.
Line 356: Line 395:
=== Flag AI_ADDRCONFIG considered harmful ===
=== Flag AI_ADDRCONFIG considered harmful ===


As far as I know, AI_ADDRCONFIG was added for the following reasons:
Current implementation of AI_ADDRCONFIG flag is a source problems and confusion. Detailed description of the problem was moved to a separate article: [[Networking/NameResolution/ADDRCONFIG|Flag AI_ADDRCONFIG considered harmful]].
 
* Some buggy DNS servers would be confused by AAAA requests
* Optimization of the number DNS queries
 
Currently, I'm aware of several documents that define AI_ADDRCONFIG:
 
* POSIX1-2008: useless but harmless
* RFC 3493 (informational): useless but (partially) breaks IPv4/IPv6 localhost
* RFC 2553 (obsolete informational): useless but hopefully harmless
* GLIBC getaddrinfo(3): like RFC 3493
 
Actual GLIBC <code>getaddrinfo()</code> behavior differs from the manual
page.
 
==== Problem statement ====
 
Currently, any of the definitions above prevents AI_ADDRCONFIG from filtering
out IPv6 addresses when a link-local IPv6 address is present.
These addresses are automatically added to interfaces that are otherwise
only configured for IPv4. Therefore, on a typical linux system, AI_ADDRCONFIG
cannot meet its goals and is effectively useless.
 
But it builds on a false assumption, that no IPv4 communication is feasible
without a non-loopback address. But why would we have a loopback address
if we can't use it for node-local communication? AI_ADDRCONFIG breaks
''localhost'', ''localhost4'', ''localhost6'', ''127.0.0.1'', ''::1'' and
more if there's no non-loopback address of the respective protocol.
 
This can happen if the computer is connected to an IPv4-only network or
and IPv6-only network, when it loses IPv4 or IPv6 connectivity and when
it's used offline.
 
==== Making AI_ADDRCONFIG useful ====
 
A possible solution for the first problem (that AI_ADDRCONFIG is useless)
is to treat link-local addresses the same as loopback (or node-local)
addresses. But this is even more harmful.
 
Fedora's GLIBC was patched to do exactly the above thing. The consequence
was that even link-local IPv6 stopped working when a global IPv6 address
was absent. And what would we have link-local addresses for if they didn't
work without global addresses? This patch has been already reverted.
 
==== Conclusion ====
 
The whole idea of filtering-out non-DNS addresses is flawed and breaks
so many things including IPv4 and IPv6 literals. There is no reason
to filter them out.
 
Proposed solutions:
 
1) Make <code>getaddrinfo()</code> ignore AI_ADDRCONFIG. It has not been working for years and nobody
cared enough to fix it, there is a substantial probability that it's not
needed.  Remove the code that implements it ([https://bugzilla.redhat.com/attachment.cgi?id=615840 patch]).
 
2) Patch all software to avoid using AI_ADDRCONFIG. Follow new development, and
prevent/reject modifications that add it.
 
3) Only process AI_ADDRCONFIG in the nsswitch DNS plugin. This requires
implementing <code>getaddrinfo()</code> in nsswitch which is required
for zeroconf networking anyway. Use solution (1) as a temporary fix.
 
The first solution is trivial. The second is rather political. And
the third solution has been so far the most acceptable. It is an
output of long discussions between me (Pavel Šimerda) and Tore Anderson,
who explained me the original purpose of AI_ADDRCONFIG.
 
More resources:


* IPv4: [http://bugzilla.redhat.com/show_bug.cgi?id=721350 <code>getaddrinfo("127.0.0.1", ...)</code> fail with some AI_ADDRCONFIG configurations]
=
* IPv6: [http://bugzilla.redhat.com/show_bug.cgi?id=808147 Fedora 808147 - <code>getaddrinfo("::1", ...)</code> fails with some configurations of AI_ADDRCONFIG]
* IPv6: <code>getaddrinfo("fe80::1234:56ff:fe78:90%eth0", ...)</code> also fails as above
* IPv6: [http://bugzilla.redhat.com/show_bug.cgi?id=843054 GLIBC's nsswitch doesn't support overriding <code>getaddrinfo</code> which is requred to resolve link-local IPv6 addresses]


==== Comments and discussion ====
=== Comments and discussion ===


Please send any remarks and questions to psimerda-at-redhat-dot-com or use [[Talk:Networking]]. Edit with care.
Please send any remarks and questions to psimerda-at-redhat-dot-com or use [[Talk:Networking]]. Edit with care.

Latest revision as of 09:59, 8 October 2015

Overview

Name resolution topics:

Host and services are often called by names which need to be translated to IP addresses and port numbers.

Input arguments:

  • node
  • service
  • filters (address family, socktype, protocol)
  • flags

On output, we expect a list of address records that can be used for connection and other purposes, each consisting of the following fields:

  • address family and data (network layer)
  • protocol and port (transport layer)
  • socktype (mode of operation)

Additional information can be provided:

  • canonical name

Use cases

  • Generic translation between hostnames and IP addresses (e.g. for ACL purposes)
  • Preparing for connect() or sendto() to contact a service
  • Preparing for bind() to provide a service

Name resolution methods

Using getaddrinfo()

Connecting to services using getaddrinfo()

The getaddrinfo() function is a dualstack-friendly API to name resolution. It is used by applications to translate host and service names to a linked list of struct addrinfo objects. It has its own manual page getaddrinfo(3) in the Linux Programmer's Manual.

Running getaddrinfo()

And example of getaddrinfo() call:

const char *node = "www.fedoraproject.org";
const char *service = "http";
struct addrinfo hints = {
    .ai_family = AF_UNSPEC,
    .ai_socktype = SOCK_STREAM,
    .ai_protocol = IPPROTO_TCP,
    .ai_flags = AI_ADDRCONFIG,
    .ai_canonname = NULL,
    .ai_addr = NULL,
    .ai_next = NULL
};
struct addrinfo *result;
int error;

error = getaddrinfo(node, service, &hints, &result);

The input of getaddrinfo() consists of node specification, service specification and further hints.

  • node: literal IPv4 or IPv6 address, or a hostname to be resolved
  • service: numeric port number or a symbolic service name
  • hints.ai_family: enable dualprotocol, IPv4-only or IPv6-only queries
  • hints.ai_socktype: select socket type
  • hints.ai_protocol: select transport protocol

Socktype and protocol are somewhat duplicate for TCP/IP stack with just TCP and UDP. getaddrinfo() can be futher tweaked with the hints.ai_flags. Other attributes are not supposed to be set in hints (ai_canonname, ai_addr and ai_next).

On success, the error variable is assigned to 0 and result is pointed to a linked list of one or more struct addrinfo objects.

Never assume that getaddrinfo() returns only one result or that the first result actually works!

Using getaddrinfo() results

It is necesary to try all results until one successfully connects. This works perfectly for TCP connections as they can fail gracefully at this stage.

struct addrinfo *item;
int sock;

for (item = result; item; item = item->ai_next) {
    sock = socket(item->ai_family, item->ai_socktype, item->ai_protocol);

    if (sock == -1)
        continue;

    if (connect(sock, item->ai_addr, item->ai_addrlen) != -1) {
        fprintf(stderr, "Connected successfully.");
        break;
    }

    close(sock);
}

For UDP, connect() succeeds without contacting the other side (if you are using connect() with udp at all). Therefore you might want to perform additional actions (such as sending a message and recieving a reply) before crying out „success!“.

Freeing getaddrinfo() results

When we're done with the results, we'll free the linked list.

freeaddrinfo(result);

Using getaddrinfo() in Python

Python's socket.getaddrinfo() API tries to be a little bit more sane than the C API.

#!/usr/bin/python3

import sys, socket

host = "www.fedoraproject.org"
service = "http"
family = socket.AF_UNSPEC
socktype = socket.SOCK_STREAM
protocol = socket.SOL_TCP
flags = 0

result = socket.getaddrinfo(host, service, family, socktype, protocol, flags)

sock = None
for family, socktype, protocol, canonname, sockaddr in result:
    try:
        sock = socket.socket(family, socktype, protocol)
    except socket.error:
        continue
    try:
        sock.connect(sockaddr)
        print("Successfully connected to: {}".format(sockaddr))
    except socket.error:
        sock.close()
        sock = None
        continue
    break

if sock is None:
    print("Failed to connect.", file=sys.stderr)
    sys.exit(1)

Tweaking getaddrinfo() flags

  • AI_NUMERICHOST: use literal address, don't perform host resolution
  • AI_PASSIVE: return socket addresses suitable for bind() instead of connect(), sendto() and sendmsg()
  • AI_NUMERICSERV: use numeric service, don't perform service resolution
  • AI_CANONNAME: save canonical name to the first result
  • AI_ADDRCONFIG: to be used with connect(), this never really worked, as far as I know, but we want to fix it
  • AI_V4MAPPED+AI_ALL: only with AF_INET6, return IPv4 addresses mapped into IPv6 space
  • AI_V4MAPPED: I don't see any real use for this, only returns mapped IPv4 if there are no IPv6 addresses

Connecting to multiple transport channels

Some applications need to open several TCP or UDP channels to the same host. The classic usage of getaddrinfo() returns a linked list of addrinfo objects for just one channel.

Solutions:

1) Run getaddrinfo() once per channel. This may for example cause multiple DNS requests for the same information, which is suboptimal

2) Run getaddrinfo() only for the main (or first) channel. When connection succeeds, reuse the addrinfo structure for the other channels. It is usually safe to assume that when one channel succeeds, the machine is available for the other channels, too.

An example of such an application is Spice. Thanks to David Jaša for information on this subject.

Binding to addresses using getaddrinfo()

According to the manual page, you should use AI_PASSIVE flag when you use getaddrinfo() to retrieve addresses to bind() to. Those are usually stored as user configuration with NULL being the default value.

getaddrinfo() returns a linked list of addrinfo structures. Developers should not generally assume that it only returns one address nor that the first address is the best and only one to bind() to.

The general idea is that one would loop through getaddrinfo() structure and bind() one socket to each of them. But this doesn't work in general. Read on.

Binding to the INADDR_ANY and/or in6addr_any addresses

This is the most common option that doesn't limit the service to a particular set of local addresses.

getaddrinfo(NULL, ...) with AI_PASSIVE returns two addresses, 0.0.0.0 and ::, in this order. If you use the general rule above, the resulting actions would look like:

sock1 = socket(AF_INET, SOCK_STREAM, SOL_TCP)
bind(sock1, INADDR_ANY, sizeof (INADDR_ANY))
...
sock2 = socket(AF_INET6, SOCK_STREAM, SOL_TCP)
bind(sock2, in6addr_any, sizeof (in6addr_any))
...

This won't work. The first bind() will successfully bind to 0.0.0.0, while the second tries to bind() in a dualstack manner, taking both :: and 0.0.0.0 (unless sysctl net.ipv6.bindv6only is enabled) and therefore it will fail as 0.0.0.0 is already taken.

The addresses are obviously returned in different order than usual (IPv4 first) and that should probably be fixed in glibc. But even if it's fixed, it only changes the order of the actions:

sock1 = socket(AF_INET6, SOCK_STREAM, SOL_TCP)
bind(sock1, in6addr_any, sizeof (in6addr_any))
...
sock2 = socket(AF_INET, SOCK_STREAM, SOL_TCP)
bind(sock2, INADDR_ANY, sizeof (INADDR_ANY))
...

The first socket succeeds and binds to both addresses. The second one fails. If the application ignores (or only warns about) the second failure, it would work without problem.

The correct™ way to handle this with getaddrinfo() would be to prevent the linux kernel from using the dualstack hack with the setsockopt() call (yes is an int constant that equeals 1).

sock1 = socket(AF_INET6, SOCK_STREAM, SOL_TCP)
setsockopt(sock1, IPPROTO_IPV6, IPV6_V6ONLY, &yes, sizeof(yes))
bind(sock1, in6addr_any, sizeof (in6addr_any))
...
sock2 = socket(AF_INET, SOCK_STREAM, SOL_TCP)
bind(sock2, INADDR_ANY, sizeof (INADDR_ANY))
...

After writing this, I found a resource from Bert JW Regeer on the same subject.

Binding to specific addresses by number

This works well because in this case only one address is ever returned (if socktype and protocol are specified). It is possible to limit getaddrinfo() to this case using the AI_NUMERICHOST flag. AI_PASSIVE flag is not used in this case.

Binding to a list of specific addresses by hostname

This is just getaddrinfo() resolving. The AI_PASSIVE flag is not used in this case. Always remember that getaddrinfo() returns a linked list of addresses. If you support binding by name, you should always create one socket for each address and bind() it.

Proposed solutions

1) Only support listening on all addresses. Always bind() to the IPv6 address in dualstack mode.

sock1 = socket(AF_INET6, SOCK_STREAM, SOL_TCP)
bind(sock1, in6addr_any, sizeof (in6addr_any))
...

2) Use only numeric addresses and NULL setting. Special case NULL setting and solve them separately with #1. Only then, if you use AI_NUMERICHOST and properly specify the socktype and/or protocol fields, you can assume that getaddrinfo() only returns one result.

If result order is fixed in glibc, you wouldn't actually have to special case the NULL host. But it's more of a hack.

3) Support full name resolution. Always create one socket per resulting address. Set IPV6_V6ONLY on IPv6 sockets to suppres the transparent dualstack. This works for all cases.

There are many other possibilities that seem to work at first glance but don't behave as expected in many situations.

Testing code (in Python for simplicity)

The following test shows how it works:

getaddrinfo-test-ai-passive.py:

#!/usr/bin/python3
from socket import *
hosts = [
    None,
    "localhost",
    "info.nix.cz",
    "www.google.com",
]
for host in hosts:
    print(host)
    for item in getaddrinfo(host, "http", AF_UNSPEC, SOCK_STREAM, SOL_TCP, AI_PASSIVE):
        print("  ", item)
# ./getaddrinfo-test-ai-passive.py
None
   (2, 1, 6, '', ('0.0.0.0', 80))
   (10, 1, 6, '', ('::', 80, 0, 0))
localhost
   (10, 1, 6, '', ('::1', 80, 0, 0))
   (2, 1, 6, '', ('127.0.0.1', 80))
info.nix.cz
   (10, 1, 6, '', ('2a02:38::1001', 80, 0, 0))
   (2, 1, 6, '', ('195.47.235.3', 80))
www.google.com
   (10, 1, 6, '', ('2a00:1450:4016:801::1012', 80, 0, 0))
   (2, 1, 6, '', ('173.194.35.144', 80))
   (2, 1, 6, '', ('173.194.35.145', 80))
   (2, 1, 6, '', ('173.194.35.147', 80))
   (2, 1, 6, '', ('173.194.35.148', 80))
   (2, 1, 6, '', ('173.194.35.146', 80))

You can clearly see the reversed order for NULL host and multiple results for hostnames. You can further improve the code with creation and binding of the sockets.

Using getaddrinfo() for accesslists

Some software uses addresses where you can put addresses, hostnames or more sophisticated filters based on them. Names can be resolved at configuration time or at check time.

Various hosts are connecting to the hostname using IPv4 and IPv6 addresses. Whenever the administrator uses a hostname in the accesslist, the hostname must point to all addresses that can be used for the connection.

On the other hand, when the administrator puts addresses to the accesslist, he must put all possible addresses there, otherwise an unpleasant surprise awaits him when the host connects using an address that is not in the ACL.

localhost is a special case that, on most current distributions resolves to ::1 and 127.0.0.1, in this order. This is usually only driven by the /etc/hosts configuration file, except the list is reordered by getaddrinfo().

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

For example, when a service has 127.0.0.1 in the accesslist, it is illegal to try to connect to it over ::1. But that's the first address getaddrinfo("localhost", ...) returns. This is a configuration problem but even many administrators skilled in IP networking won't realize that localhost is not identical to 127.0.0.1.

Proposed solutions

1) Make administrators fix their configurations. Tell them about the breakage of identity between "localhost" and "127.0.0.1".

2) Workaround: Change client behavior on access denied error from server and treat it as a network error. Always try other addresses from the list returned by getaddrinfo().

3) Workaround: Prefer 127.0.0.1 over ::1. This assumes that anyone putting ::1 in an access list knows what he's doing.

Flag AI_ADDRCONFIG considered harmful

Current implementation of AI_ADDRCONFIG flag is a source problems and confusion. Detailed description of the problem was moved to a separate article: Flag AI_ADDRCONFIG considered harmful.

=

Comments and discussion

Please send any remarks and questions to psimerda-at-redhat-dot-com or use Talk:Networking. Edit with care.