networking - DNS resolution failing over to secondary DNS - why?

itemprop="text">

We have large number of branch offices
connected via VPN, but without any kind of server infrastructure. The client machines in
each office get their network configuration from an ASA 5505, which is also used for the
VPN connection.

The Windows XP client machines
are configured to use one of our corporate DNS servers as the primary, with the DNS
server of the ISP as the secondary. The idea is that if the VPN connection fails for any
reason, staff in the office will still be able to access the internet, and access our
webmail and home access portal. In the majority of cases this works
fine.

However, for offices based in South
America we are seeing DNS resolution on the client machines regularly being done against
the ISP DNS server - this results in our corporate resources being effectively
unavailable to staff in the
offices.

The client machines are able
to ping the corporate DNS server ok. When doing an nslookup of a corporate hostname, I
get a reply.

I'm thinking one of the following
(or a combination) is
happening:

• our corporate
  DNS server is not always replying to requests in a timely fashion (although why this
  would only affect clients in one geographic region I don't
  know)


• DNS queries from Latin America are somehow delayed,
  causing the client to treat it as failed (although we have offices at the end of much
  slower VSAT connections which do not have this issue)


• a
  single failure is resulting in a DNS cache entry in Windows that somehow results in the
  lookups not happening on subsequent
  tries

Has
anyone else come across this issue? Any ideas for resolutions?

Answer

Windows href="http://technet.microsoft.com/en-us/library/cc775637%28WS.10%29.aspx" rel="nofollow
noreferrer">queries DNS in this
order:

• hosts
  file


• local DNS
  cache


• Preferred DNS
  servers


• Other DNS
  servers

MS
also has an article describing href="http://technet.microsoft.com/en-us/library/cc779517%28WS.10%29.aspx" rel="nofollow
noreferrer">how the DNS server list is
obtained:

The DNS Client service uses a server search list, ordered by preference. This
list includes all preferred and alternate DNS servers configured for each of the active
network connections on the system.

The list is
arranged based on the following criteria:

• Preferred DNS servers are given first
  priority.



• If no preferred DNS servers are
  available, then alternate DNS servers are used.



• Unresponsive servers are removed temporarily from these
  lists.

Windows has an
escalating timeout for DNS
requests:

Value Default
            value Attempt
1st limit 1 second Query the preferred DNS server on a preferred
            connection.
2nd limit 2 seconds Query the preferred DNS server on all
            connections.

3rd limit 2 seconds Query all DNS servers on all
            connections (1st attempt).
4th limit 4 seconds Query all DNS servers on all
            connections (2nd attempt).
5th limit 8 seconds Query all DNS servers on all
            connections (3rd attempt).
6th value (Must be
            0.)

I could not find a
clear answer on this exact point, but it sounds like if it doesn't get a response from
your primary DNS in 1 or 2 seconds (1st or 2nd attempt, respectively), then that server
will be removed from the DNS server lookup list for 15 minutes, and so it will use the
secondary DNS servers. Since those servers have up to an 8 second timeout, they are much
more likely to respond. (It's unclear to me if it continues to query the preferred DNS
server during the 3rd+ attempt if it's already
failed).

I also suspect that you do indeed have
a WAN latency issue for this geographical area, as it would explain why the timeouts are
working.

/>

One solution is to change the DNS query timeouts,
using the rel="nofollow noreferrer">DNSQueryTimeouts registry parameter. See also
href="http://drewthaler.blogspot.com/2005/09/changing-dns-query-timeout-in-windows.html"
rel="nofollow
noreferrer">http://drewthaler.blogspot.com/2005/09/changing-dns-query-timeout-in-windows.html

/>

Another solution is to put a local caching DNS
server on the network, and have the clients use that. You can use a DNS server that may
be built in to a router, or install something like href="http://thekelleys.org.uk/dnsmasq/doc.html" rel="nofollow
noreferrer">dnsmasq.