linux - Cannot bind to a specific IPv4 address when making outbound TCP connections, to hostnames that resolve to both IPv4 and IPv6 addresses

I just spent about 6 hours trying to figure this out, and I now believe CentOS/Linux is unable to bind to a specific virtual IPv4 address when connecting to a hostname that has an IPv6 address. This is a problem on servers that have multiple IP addresses.

I am using Centos 6 (Linux kernel 2.6.32-573.12.1.el6.x86_64)

To reproduce this big:

1. Find a Linux machine with at least a /29 IPv4 public address space, and an IPv6 public IP.



2. Alias at least one additional IPv4 to the main interface (eth0 or otherwise). For this example, I will say 30.0.0.1 as the machine's primary eth0 IPv4 address, and 30.0.0.2 is an alias bound to eth0:2 on a network of 30.0.0.0/29.


3. Find a hostname that has both IPv4 and IPv6 addresses. For example, www.microsoft.com.


4. telnet -b 30.0.0.2 www.microsoft.com 80 (This tests making an outbound connection using a specific ipv4 address)


5. The IPv4 request connects successfully after unsuccessfully trying the hostname's IPv6 addresses, but the TCP connection actually initiates from the machine's main IP (30.0.0.1), not the IP you want (30.0.0.2). Netstat says otherwise, but it is WRONG. If you connect to a hostname that you own and can view its logs, the connection comes from the main IP (30.0.0.1) not the IP you want (30.0.0.2).


6. Try another domain that does not have an IPv6 address, like: telnet -b 30.0.0.2 serverfault.com. It works. It makes the connection from the IP you want.

This is a problem, because certain programs, like mail (exim), need to use certain IPs when making outbound TCP requests, which is not necessarily the main machine's IP. Certain clients/programs on the machine rely on ACLs or reverse DNS to match up correctly when making outbound TCP connections.

So if anyone else is noticing the same weird problem where their program cannot bind to the correct interface when making outgoing connections, this is probably why.

This problem only effects IPv4 connections. IPv6 connections correctly bind to whatever outgoing IP you have on the machine.

This is not an issue with telnet. I tested this problem using my mail server (exim) too, and got similar results. It makes IPv4 connections from the wrong IP if the target hostname has an IPv6 address.

Maybe someone has a solution to this strange problem, but at this time, I think it may be a Linux networking bug.

Ps- If someone wonders why not just make an IPv6 connection if the hostname resolves to an IPv6 address...sometimes the IPv6 address is down or the connection cannot establish, then it reverts to its IPv4 address.

Answer

You can trust netstat to give you correct information about the IP addresses (at least as long as -n is used).

If the endpoints of a TCP connection disagree on which IP addresses are in use, it means there is a NAT somewhere between the two.

From the additional information provided in the comments we learned that in this particular case a superfluous iptables rule -A POSTROUTING -j MASQUERADE was the cause of the problem.