For businesses, bots are processes that enable electronic marketing, commerce and customer service transactions and are a foundational part of the fabric of doing business online. Increasingly, botnets use artificial intelligence and analytics to make their functions more effective.
But bots have also become tools for malicious attacks against businesses, according to a mid-2024 report by Akamai Technologies. For example, attackers execute their criminal business model on business sites to cash out loyalty points, place fraudulent orders, or request product returns that create revenue through fraudulent refunds.
Akamai’s analysis found that more than half of the global commerce web traffic is made up of bots, and the bot traffic levels continue to rise.
Akamai segments bots into “Good bot” and “Bad bot” use-cases:
According to Akamai research, 42.1% of overall traffic activity was from bots, with 65.3% of that bot traffic from malicious bots. And a total of 63.1% of the bad bots traffic used advanced techniques.
Scraping the Internet
Botnets that collect information that’s freely available on the internet are called scrapers. Scraping supplies the information that is used by analytics and AI processes, to make bot targeting more effective, and is used both by good and bad bots.
The detection of scraper bots has become more difficult because they, too, use artificial intelligence (AI) botnets and increasingly utilize headless browser technologies.
Why scrape?
Malicious bots can be used to take advantage of scarcity, such as buying out limited events or to buy hot products. Bots can be used to abuse customer acquisition marketing programs by taking advantage of special offerings, which impacts campaign analysis and costs.
Akamai finds that attackers can scrape news articles, blogs, and other content and place it on their own sites, causing the original organization to lose visitors and potential advertising revenue.
Malicious bots can also be trained to harvest private information from an organization’s site to undercut pricing, make changes to their offers, and get a sense of new opportunities and threats.
Bots can even mimic human behavior online to increase clicks and traffic on a website, skewing both the marketing and performance analytics of carefully crafted digital experiences.
In addition, large Distributed Denial-of-Service (DDoS) botnets can overwhelm web-facing applications and cause a poor user experience or the inability to place orders or make reservations, resulting in lost revenue and a negative impact on customer satisfaction that can in turn impact reputation.
Detecting scraping attacks
The first step in reducing bot attacks is to recognize them through detection. Akamai triggers bot alerts when a bot payload is detected within a request to a protected website, application, or API.
Akamai recognizes simpler scraper bots when they make requests advertising older browser or OS versions, use old versions of HTTP, have anomalies in their HTTP header signatures, or appear to originate from thousands of servers. More advanced scrapers advertise current browsers, use more current versions of HTTP, and appear to originate from hundreds of thousands residential and mobile IP addresses.
Another indicator is the pattern of attack traffic; where detection can differentiate between human traffic, which has a day-night rhythm, versus traffic generated by simpler bots, which is regular but may have occasional breaks. Traffic from more sophisticated bots runs continuously.
Bot protection
Akamai advocates a variety of anti-bot protection techniques. Bot protection may include JavaScript fingerprinting, HTTP and TLS fingerprinting (assessing the HTTP headers and TLS handshake), and Internet Protocol (IP) reputation detection. Some of these workflows may include machine learning (ML), especially when gathering statistics on the success rate; adjusting to the cookie strategy, HTTP header, and TLS parameters; and evaluating the JavaScript fingerprinting code.
Further reading:
Scraping Away Your Bottom Line: How Web Scrapers Impact Ecommerce. Report. State of the Internet/Security series. June 2024. Akamai Technologies
Why it matters
Malicious bots can have a significant impact on business. One example cited by Akamai was one of its ecommerce customers, which was unaware that 99% of high-risk traffic had been stopped by scraper bots.
Akamai sees bots being used to enable criminal attacks, fraud schemes, and competitive intel. Recently, Akamai has seen a trend toward the increased use of all bots and a rise in the negative business impacts of scraper bots. This report is intended to share both technical insights and attack methodology to raise awareness of this growing problem across the commerce industry.
