Last updated at Tue, 26 Feb 2019 16:50:31 GMT

There has been an obsession of late with encrypting all traffic that traverses the internet. This obsession has taken many forms, ranging from Let’s Support Phishing! Encrypt! to myriad standards and technologies trying to bolster the confidentiality of email exchanges. Whether you’re trying to keep prying digital eyes from seeing which cat videos you’re watching in your browser or which memes you’re forwarding via email, there’s one additional layer that needs securing to truly (mostly) prevent these third parties from knowing crucial details of where you are and what you’re communicating: domain name system (DNS) lookups that translate names like www.rapid7.com to 143.204.153.174.

When you enter https://www.rapid7.com/ into your browser’s location bar, a number of things occur. The browser knows it will need to make a connection to the IP address that hosts www.rapid7.com and make an SSL/TLS connection to it. So, it will query DNS for the address in order to establish a secure handshake/connection and then make the page request. The problem lies in that (pretty much) anyone on your local network, anyone that has access to your router/gateway, your internet service provider (ISP), and all the routers that DNS request goes through—all the way to the DNS server that responds to the request—knows you want to go to www.rapid7.com.

Why is this important? Well, first, replace www.rapid7.com with www.medical-related-website.com or www.whistleblower-reporting-site.com or www.activist-cause-you-support.com. Now, all the prying eyes on the path to the DNS server know (or at least suspect) you’re unwell, disgruntled, or involved in certain activities that may be sensitive enough that you don’t prefer not disclosing to random third parties. At a less scary level, ISPs—at least in the U.S.—siphon off this data in-stream (regardless of normal DNS server used) and from their DNS servers (if you use them for DNS) and sell your activity data to anyone (plus share it with the U.S. government on demand).

So, how can you conceal this last link in the confidentiality chain? Use encrypted DNS lookups.

There are multiple “competing” ways to encrypt DNS lookups. Some of the major ones are/have been:

You will need to do some work to use either of those three methods unless you run the latest Ubuntu desktop operating system or are using a fairly recent Google Android device.
This inertia is one big reason secure DNS queries haven’t taken off outside of niche security-conscious communities.

We’ll be focusing on DNS over TLS for the remainder of this blog for a number of reasons:

  • Unlike DNSCrypt, it has an IETF standards-track RFC (standards are good!).
  • It’s the new hotness, and major providers like Quad9, Cloudflare, Google, and CleanBrowsing all provide DNS over TLS servers.
  • The barriers to entry for using it are diminishing.
  • It has potentially faster query/response performance than DNS over HTTPS and is likely to win out in the end in the Battle of Secure DNS lookups (aka the most boring pay-per-view event ever).

DNS? HTTPS? TLS? WT … Heck?

Regular ol’ DNS works over the user datagram protocol (UDP), which is an efficient but potentially lossy way of sending/receiving data over a network. In reality, on modern networks, it’s pretty darned fast and reliable, especially when protocols that run over UDP are not verbose. Traditional DNS lookups are not verbose and use this method, since y’all want super-fast access to those cat pictures. While you can use your internet service provider’s DNS servers, DNS can be a major factor in slowing down web page loads, which is why there are alternate and optimized, free, and commercial options available.

Both DNS over HTTPS (DoH) and DNS over TLS (DoT) use the transmission control protocol (TCP) to transmit and receive queries. The big difference is how they do this. The “HTTPS” part of DNS over HTTPS is quite telling, as you’re making a bona-fide HTTP request. If you don’t believe that, then just visit https://dns.google.com/resolve?name=www.rapid7.com right now. You just made a DoH request. If that seems inefficient, you’re right! You’re making an API request to a web server, and even with long session caching, a client has to make a full HTTP GET request, the server has to process it and respond with an JSON HTTP response, and the client then has to process the JSON response and work with the resulting data.

DNS over TLS takes a different approach. The client establishes a secure TLS session with a server (Note: I’m deliberately glossing over destination integrity details for now for both DoH and DoT) that can remain open for as long as it is supported by the client and server, then regular, small UDP-like DNS protocol queries are sent and the same diminutive responses are received and processed with the same type of code used by existing DNS clients and servers today. The big difference is that TCP TLS session.

You mentioned something about “research?”

Ah, yes. Rapid7 Labs has created an experimental Project Sonar study with the paramount purpose being to monitor the number of IPv4 addresses hosting DNS over TLS servers (on TCP 853). Notice we didn’t say just “the number of DNS over TLS servers,” since services like Google (8.8.8.8/8.8.4.4), Quad9 (9.9.9.9), Cloudflare (1.1.1.1), and others have loads and loads of physical servers, all fronted by just the one public IP address.

However, there will be more DNS over TLS servers as time goes by—possibly (hopefully?) as many as there are on current UDP/TCP 53. Using one of the centralized “big ones” is just transferring the snooping from the level of your immediate network and ISP to, say, Google, Cloudflare, and IBM (plus some others for Quad9). How much you trust any of them is up to you, and there are and will be many DNS over TLS providers with verifiable privacy practices that you can take advantage of. Or, you could even set up your own service.

One other reason for tracking the presence of DNS over TLS is that attackers use DNS to great effect now when establishing a foothold in an organization and engaging in command-and-control operations and data exfiltration. In the same way the “encrypt everything” zealots have created blind spots for many security solutions, DNS over TLS presents a very interesting new way for attackers to stand up infrastructure to support their activities, especially if DoT is widely adopted and security teams are unable to prevent the use of or introspect the protocol via person-in-the-middle technologies (the “encrypt everything” proponents never really talk about this gaping hole in their privacy argument). As we gather data on DoT servers we find, we can create models to try to figure out which ones are adversary-related.

On to the results!

As noted, Jon Hart and Shan Skidar wired up an experimental lightweight study that looks for nodes on TCP 853 and then issues a VERSION.BIND request so we can get some metadata back about what the server is running. Not all servers respond to this request, which means we did and will miss some nodes in this study (more DoT studies are likely coming to examine the integrity of address and other responses).

So, where are these servers, and how many of them are there? Well, there are about 600 of them (that respond to VERSION.BIND) lightly peppered all over the globe:

Country n
Ireland 502
Germany 62
France 26
United Kingdom 17
Switzerland 16
Austria 8
Denmark 7
Canada 6
Australia 4
Czechia 4
India 4
Hong Kong SAR China 3
Brazil 2
China 2
Finland 2
Iceland 2
Israel 2
United Arab Emirates 1
Bulgaria 1
Spain 1

Ireland has some prominence due to the CleanBrowsing service.

These DoT servers are fairly concentrated network-wise as well:

Organization n
Daniel Cid 502
Internet Initiative Japan 28
Hetzner Online GmbH 22
Core-Backbone GmbH 20
Choopa, LLC 18
OVH SAS 12
Interactive 3D 10
LeaseWeb Netherlands B.V. 10
Faelix Limited 7
Google Cloud 7
Blix Group As 6
Digital Ocean 6

Since this is a service designed for confidentiality, knowing a bit about where the certificates come from can give us an idea of how each organization values them. That is, Let’s Encrypt offers certificates for free with some fairly spiffy renewal automation whereas others still charge because they at least somewhat believe there is still at least some integrity merit associated with certificates:

Issuer n %
Go Daddy 532 61.4%
Let's Encrypt 234 27.0%
Other 53 6.1%
GlobalSign 32 3.7%
DigiCert 8 0.9%
COMODO 3 0.3%
Thawte 3 0.3%
RapidSSL 2 0.2%

Go Daddy == CleanBrowsing and the use of Let’s Encrypt is unsurprising given the nature of that project and the overlap in that community and the secure DNS community.

There is quite a bit of commonality in the certificate Common Names:

Apex domain n
cleanbrowsing.org 531
perfect-privacy.com 106
2iij.net 32
nazwa.pl 12
quad9.net 8

This also applies to key sizes:

Remember that for DNS over TLS to truly help you with privacy all of the assurance and integrity configurations of the server software, hosts, and certificates must be well-designed. In other words, this is going to be an easy service to misconfigure.

Remember, too, that the TLS part of DNS over TLS is, by and large, just a shim over the existing DNS software at the provider. If we just look at the BIND versions, we can see that some services choose “stability” over currency:

So what?

The state and change rate of DNS over TLS on the internet should be something you take into consideration both personally and in your organization. On a personal level, DoT gives you a way to prevent ISP snooping and gain some confidence in the results of DNS lookups, provided you know and trust the server you’re using. You will need to keep an eye on what browsers and operating systems (especially Android, now) are doing to make sure they aren’t subverting your own efforts to use a trusted provider as well.

Organizations should be monitoring attempts to use DNS over TLS traffic. Naive attackers will lazily use the default port, but clever ones may try to use DoT on others ports, including traditional encrypted web ports like TCP 443. Since the traffic is encrypted by default, detecting DNS over TLS means spending budget dollars and personnel time on SSL breaking gear, or figuring out clever ways to detect the use of this protocol by examining features such as session times, packet sizes, and packet frequencies.

Rapid7 Labs will continue to enhance and expand our work studying DNS over TLS and will hopefully be adding the result sets to OpenData in the coming months. You can reach out to research@rapid7.com if you have questions about the DNS over TLS study or any of our other Project Sonar studies.