Bot management is a feature of application security software that identifies whether a traffic request comes from a human or from a machine and then controls or blocks non-human and other suspicious requests.
Roughly half of the traffic on the internet comes from bots, which are bits of code created to perform automated tasks. There are good bots, such as those that index content for search engines, making it easier for people to find websites. But there are also malicious bots that exploit vulnerabilities, expose data, shut down entire websites, and steal intellectual property. As such, bot management is a crucial part of modern website security.
Benefits of bot management software
Bot managers enable organizations to reap the benefits of good bots while protecting themselves against malicious bots.
Some machine traffic, such as that from Google’s bots, is fine. Organizations obviously want Google bots to access every one of their pages and index them in search results. But they don’t want bad bots or unknown bots trying to access their sites, which are supposed to be for humans.
Bot management software classifies incoming requests into two buckets: humans or good bots, and unknown or malicious bots. It treats unknown bots the same as malicious bots, because organizations can’t afford to assume something unknown has good intentions. The software directs malicious and suspect bot traffic away from the site, protecting data and preventing detrimental effects on the end-user experience.
This video explains the different types of malicious bots.
Additionally, just because a request comes from a good bot or a human, that doesn’t mean it contains no attack payload. A web application firewall would look for that. For that reason, bot management must be part of a bigger web application security platform that vets every traffic request.
How bot management works: Detection and mitigation
There are a few different bot management features for detection and mitigation, and they have different levels of sophistication.
Basic capabilities include IP rate limiting, which prevents too many requests from the same address from overloading the server, and CAPTCHA challenges, which present a puzzle the user must solve before accessing the application.
This bot mitigation technique is very efficient, because it’s expensive for hackers to simulate a full internet browser. This is especially true when they launch a DDoS attack, because it’s even more expensive to run full internet browser simulations inside of a botnet. It’s not impossible, however, so there needs to be other bot management features at organizations’ disposal.
Second-generation bot management features are much more sophisticated. They analyze the behavior of the user to identify whether it is human or not. Malicious bots are getting better at replicating human behavior, however.
The next generation uses supervised machine learning techniques and artificial intelligence engines to classify whether requests are from humans or bots on a per-application basis. Early adopters of this technology are typically large enterprises with strict security and compliance requirements.
Bot management use cases
The main use case of bot management software is to identify and protect websites and applications from malicious bot traffic. The four main types of bot attacks are:
- vulnerability scans and exploits;
- network-based DDoS attacks;
- application layer DDoS attacks; and
- fraudulent purchases, content scraping, and other malicious site activity.
But even when the bot traffic is not malicious, there are times when a site might want to prioritize human traffic over bot traffic. Bot management helps with that as well.
Oracle Dyn has a customer, a rental car company, that has good bot traffic coming to its site from travel agencies that want to check prices and show them to their customers. The rental car company is fine with this traffic but uses our bot management technology to prioritize human traffic over bot traffic, because a human coming to the site to rent a car has a direct impact on business.
The bot management software identifies which traffic is from bots, then takes action based on what the traffic is trying to do and what the company wants to allow it to do.
Getting started with bot management
Proof-of-concept deployments allow organizations to see all the bots on their networks. A lot of the time, organizations are not aware of what is happening to their applications, and so this bot detection is an eye-opener.