Today, we launched our annual report studying the internet traffic of our global client base to analyze bot behavior and identify trends in automated, bot-generated threats that businesses face. The report provides a detailed examination of the impact of bad bots across industries and sheds light on recommended preventive measures to safeguard business operations.
While good bots help accelerate productivity and business processes such as data collection and decision making, bad bots target websites, mobile apps and APIs to steal data and disrupt service. Good bots can be used to crawl websites for web indexing, content aggregation and market intelligence, and they can free up human resources to focus on other responsibilities. Conversely, bad bots disrupt applications, steal data, perform fraudulent activities and even spread fake news.
This year’s report reveals that not only has bot traffic grown, the portion of bots with malicious intent has also grown. Even worse, within the malicious bots, we see more and more sophisticated evasion behaviors of bots mimicking a real user or such that mutate during an attack.
These days we see a massive effort by cybercriminals to take advantage of the COVID-19 situation where they deploy bad bots to spread spam, false information, phishing links and even scrape e-commerce sites for personal protective equipment to later resell in the second-hand market.
Increase in Bad Bot Traffic
First, we see that bot traffic has grown at the expense of human traffic. Second, the portion of bad bots has increased 20% year-over-year. Just think how much could you save on public cloud consumption if you didn’t have to buy that bigger firewall or load balancer…
We also recognize seasonality in bot traffic. For example, during the winter holiday season, cyber criminals looking for profit — whether by payment fraud, account takeover, ticket and inventory scalping or cart abandonment — have plenty of opportunity and prey on e-commerce sites.
Most bots were programmed to take advantage of common internet browsers. With the shift to mobile apps, we see more and more bots that are tailored to attack mobile applications. For example, consider the gaming industry, which suffers from digital fraud, rewards scraping, and denial of service attacks, or mobile banking apps that face credential stuffing attacks. Plus, the e-commerce and retail industries make a substantial amount of revenue from their mobile apps, and can be really hurt by bots disrupting product availability or competitors scraping pricing information.
In parallel, as the world is moving from user-to-app communication to app-to-app or machine-to-machine communication (IoT devices are a great example), the primary channel of data transactions is APIs. Hackers are already there targeting APIs more and more with malicious bots.
As protection technologies make progress and businesses are better equipped to stop bot attacks, cyber criminals now leverage cutting-edge technologies to bypass conventional detection heuristics and deploy more sophisticated capabilities of bad bots. In 2019, cyber attackers favored distributed bots that mimic human behavior when carrying out automated attacks. For example, 38% of bad bots used to execute account takeover attacks are classified as such.
In recent years, we have seen bots gradually evolve from basic scripts to large-scale distributed bots with sophisticated, human-like interaction capabilities to evade detection mechanisms. The latest breed of bots has advanced human-like interaction characteristics, such as moving the mouse pointer in a random human-like pattern instead of in straight lines. These bots are also massively distributed across tens of thousands of IP addresses, or worse – device IDs (to overcome FP technologies).
An example of a typical distributed bot that rotates IPs and user agents, coming from a single ISP. The reason for using only one ISP is that a large number of legitimate users are also coming from this ISP and bots are hiding behind those genuine users, which make detection challenge and IP-based blocking ineffective.
As far as geographical distribution, when looking at the representation of bots (good and bad) as part of the total outbound country traffic, it is surprising to see smaller countries at the top. The top five “dirty traffic” countries are:
- Russian Federation
Bad bots are present across nearly all industries and verticals. Some industries collect data that is more compelling to cyber criminals and naturally attract more bad bots than others. The industries that cyber criminals are most likely to attack for a monetary reward are e-commerce, travel and financial services. Companies in these verticals need to be more cautious and implement stricter security measures.
Since every disruption affects revenue, most e-commerce companies invest heavily in protecting their applications. And for good reason, as we see an extremely high amount (58%) of distributed, mutating bots within the total bad bot activity for this vertical. Hackers use sophisticated bots to evade bot management technologies that rely on deterministic rules or even behavioral profiling that are not big enough to produce correlations between different violations.
Travel and hospitality organizations, such as airlines, transportation, hotel chains rely heavily on online purchases. Cyber criminals target their sites with attacks that mainly use human-like and distributed mutating bots to bypass security tools with IP-based or even device fingerprinting measures.
Predictions and Recommendations
1. The use of bad bots to disseminate misinformation will ramp up in 2020. For example, the use of bots to spread misinformation about the COVID-19 pandemic.
2. Automated attacks on APIs are growing. The rate of API adoption will continue to grow because they facilitate communication between web applications.
3. Mobile applications are increasingly being used by botmasters to launch attacks. These are harder to detect as mobile apps change IPs and access points constantly.
4. Account takeover attacks will increase in number and severity, rendering personal, corporate and government data sources more vulnerable to breaches than ever before.
We encourage you to assess the impact of bots on your business. Here are a few considerations to take into account:
- Identify the intent: Good and bad scraping look the same. A login with the right credentials looks legit. That isn’t enough though, and you have to use behavioral analysis to detect whether this is legitimate or malicious.
- Flexible deployment modes: Since bots are used by all business units, it is imperative that flexibility in deployments is provided. Website integration will look different than into the devops ecosystem, and others may prefer a SaaS or an SDK for a mobile app. This is important because, at the end of the day, the solution is a business enabler and not a show stopper.
- Different mitigation options and the ability to control bots: Blocking is obvious, but then the bots will try something new — so you will have to, too. Different levels of CAPTCHAs, feeding fake data, throttle, redirect or any customer response are great ways to deceive bots.
- Visibility and customization: There is a variety of needs between different businesses, even within the same industry or even different functions within the same company (for example marketing, security and development). Each needs to get a handle on the situation by granular analytics and customization options that relate to them.