The Dyn Attack – One Year Later
One year ago, a threat actor launched a DDoS attack that disrupted service of some of the internet’s biggest names. The Mirai botnet had enslaved hundreds of thousands of IoT devices and was used to attack several entities, including the managed Domain Name System (DNS) provider Dyn.
The attack on Dyn was an event that many referred to as a wake-up call for internet security.
Except the industry, by and large, never really woke up.
Think of DNS servers as the contact list on your mobile phone. If you couldn’t access that list, how many phone numbers would you actually remember? By attacking the DNS infrastructure, internet users were no longer able to connect to many of the websites and services they use every day. The attack disrupted the connection made between domain names and the IP addresses that they belong to, essentially breaking a critical connection in how we access information and content on the internet.
The Dyn attack exposed a vulnerability that is not new, but few businesses had done anything to prevent. One year later, my new analysis demonstrates how a similar attack could occur.
One Year Later
One of the ways for website operators to protect themselves from this type of attack is to use more than one provider for DNS, providing a failover option for when a portion of the DNS system can’t do its job.
After the Dyn attack, you would expect that more companies would adopt the model of diverse DNS providers. One provider could be the primary, another could be secondary, and even more could be added for redundancy.
I was curious about how many companies had actually adopted this model. Using Alexa’s rankings of web traffic and page rankings, I pulled a list of the top 100 U.S. websites. I fed this list into a small script that I wrote to give me all of the authoritative name servers for each domain in the list. The query is effectively the following in Linux:
$ dig @[recursive DNS server] +short NS example.com
From here I could analyze the data on domains and their respective authoritative name servers. What I found was that 68 of the top 100 U.S. websites (or more than two thirds by Alexa’s rankings) still use only one DNS provider for their domain. In fact, several of the major companies directly affected by the Dyn attack still only use one provider.
However, perhaps this isn’t as simple as it seems, so I wanted to dig in further.
[You might also like: How Friday’s Massive DDoS Attack on the U.S. Happened]
First, I noticed that some of the websites that were affected by the original attack are still single homed behind a single (albeit new) provider, like Amazon’s managed DNS service:
$ dig @8.8.8.8 +short NS [redacted].com [redacted].awsdns-43.org. [redacted].awsdns-56.co.uk. [redacted].awsdns-18.net. [redacted].awsdns-10.com. $
One can’t help but wonder why a company that was affected by single-homed DNS would move to another, single DNS provider. Well, in the case above, Amazon has a tremendous amount of bandwidth globally, and because most of their traffic is outbound, they have the bandwidth to support inbound attacks. There are other benefits too, but it shouldn’t take long before you realize that a single provider is still a single provider and thus a single point of failure. API attacks and human error resulting in cascading automated failures are concerns in large scale, highly automated networks.
Also, as you can see in the example above, many DNS companies spread their services across different Top Level Domains (TLDs), which protects against problems or attacks against a particular TLD. Examples of TLDs include “.com” “.net” and “.org”.
And in the case of Amazon themselves? Well, they do use diverse providers and none of them are their own DNS service:
$ dig @8.8.8.8 +short NS amazon.com [redacted].ultradns.net. [redacted].dynect.net. [redacted].dynect.net. [redacted].dynect.net. [redacted].dynect.net. [redacted].ultradns.co.uk. $
Clearly those responsible for the domain Amazon.com see the value in redundant DNS providers.
Who else was affected?
Of course Dyn and their customers were affected. But there was other collateral damage in the ISP networks too. The anatomy of this DNS attack is as follows:
– A real device on the network makes a request for a random subdomain or Fully Qualified Domain Name (FQDN). For example, abc123xyz.example.com, asdf9876.example.com.
– These hostnames are unique and thus not cached on the local recursive DNS resolver, so it must pass its request upstream to the authoritative name server. This task can be resource intensive on the recursive server when done in large waves like this, so local DNS services are also impacted for local networks (including ISPs).
– The authoritative name server receives lookup requests for countless non-existent FQDNs from legitimate DNS resolvers around the world. Not only are they bombarded by high packet per second lookup requests, but they must process each one.
It’s important to note that individual ISPs reported impact on their own infrastructure caused by just the lookups of devices within their footprint. This highlights the need to also protect your recursive servers if you run them.
[You might also like: Everything You Need to Know About Brickerbot, Hajime, and IoT Botnets]
Could it happen again?
The short answer is: yes! Will it happen exactly the same way? Probably not.
When the source code for Mirai was released by threat actor Anna-senpai, the threat landscape changed even further. Suddenly, everyone had the capability to build and deploy their own Mirai botnet. Not only did they have the code, but they had instructions on how to do it. People even offer a service to set it up.
Anna-senpai had released the first open source IoT botnet.
Although the quantity of connected IoT devices continues to grow and the number of Mirai botnets continues to grow, those armies are now splintered. What was once a single botnet of 380,000 devices is now many botnets with much smaller bot counts. We also have fighting among the botnets, including Mirai, Hajime, and BrickerBot.
You might say that Anna-senpai had the right recipe at the right time.
What’s interesting about Mirai is that it’s incredibly versatile and customizable. Technically, the bots don’t have to be IoT devices. There was a Windows variant reported in the wild and if you’re like me, maybe you run the bot in your lab on top of Linux.
You can see how this can become a much different problem. Surely there are active Mirai bots that are actually powerful servers with big uplinks, rather than tiny, low powered devices sprinkled around a home network.
How will it happen?
The attack on Dyn was reportedly an attack on one of their customers. But what if attackers targeted the entire global DNS infrastructure? Could a large scale version of this attack happen again, targeting the top DNS providers? Yes, it could.
[You might also like: Let’s discuss facts: An insight into Mirai’s source-code]
How it will happen is open for debate. One of the strengths of Mirai is that attack traffic originates from real hosts. Another strength is that it’s customizable and attackers can add their own attack vectors to the code.
Gartner estimates that there were about 6.4 billion IoT devices on the internet in 2016 and that there will be around 20.4 billion in the year 2020. Some estimates that I’ve seen are as high as 50 billion for 2020. We’ll see when we get there, but it’s clear that we’re on a trajectory of adding a tremendous quantity of devices to the network. A portion of these devices will absolutely be capable of being exploited and used in attacks.
Even if they aren’t directly herded into botnets, these devices pose the risk of traditional attack vectors too, like reflection attacks. Simply put, many of the people developing these devices are not thinking about security. They’re thinking of how to build a nice User Interface (UI) and functionality onto a compute device with a small amount of resources (storage, CPU, memory, etc.).
API Attacks
Another way we could see this scale of attack is with threat actors targeting Application Programming Interfaces (APIs). APIs are an open interface for external connectivity to request a function inside of a third-party network. Maybe it’s an attack on the provider’s API, or maybe it’s an attack that leverages an exposed API to then launch an external attack.
What is the best solution?
In the past year, I’ve been surprised to meet security professionals who have never heard of the Mirai botnet, or the attack on Dyn and how it impacted the internet. I don’t think it’s necessarily bad to have only one DNS provider. I do think that website operators must continue to think about the impact of availability in every aspect of their online presence, including DNS.
We learned a few lessons last year. First, don’t forget about the threats and vulnerabilities that are in plain sight. Of course redundant DNS providers can benefit people. There’s a reason the protocol allows for multiple DNS servers in a zone. But many people haven’t given diversity in DNS providers much thought until recently.
Second, attacks can come from where you least expect them. How many of us thought one day we’d be talking about an IP camera botnet army that broke the internet? It all makes sense, but did we see it coming, exactly the way it unfolded? No.
As we see in the research above, some companies who were directly affected by the Dyn attack made the decision to stay with a single provider. They made other changes and I can understand why. They determined that was the best solution for their network. Other companies chose to add diversity in their providers to increase their resiliency.
[You might also like: The Expansion of IoT since Mirai.]
Nevertheless, only 32% of the top 100 U.S. websites have diverse DNS providers and that’s still a risk for the other 68%. Even if the DNS provider’s network is robust and includes DDoS protection, a single provider is just that, a single provider, and this carries some risk.
Looking Forward
As a connected ecosystem, traditional security simply isn’t keeping pace with today’s automated operational infrastructures. In civil engineering, we had years to develop codes and safety regulations. We had catastrophic failures, but we had time to rebuild and learn from design weaknesses. This is not the case in today’s connected world. It’s a race to do more, build more, and expose more. Our rate of connectivity, change and advancement is unprecedented. It’s also our greatest vulnerability as a whole.
Continue thinking about how you will protect your own online infrastructure and remember that a year ago, much of the internet was caught by surprise with this attack. You won’t always see it coming and you usually won’t know what the attack will look like until it happens. Choose a protection set that uses behavioral technologies to identify anomalous traffic and can take action to protect your infrastructure. Your business might depend on it.
