Beware the rise of the robots, as a study from Dianomi indicates that up to 90 percent of clicks generated by some campaigns do not come from humans. But, even though Dianomi warns advertisers to be vigilant of robot clicks and to never pay for them, it also explains that not all robots are necessarily bad.
The report identifies eight different robot types, with the most commonly benign being Feed Fetchers, Search Engine Bots and Commercial Crawlers (spiders used to extract authorized data on behalf of digital marketing tools). Malicious robots include Impersonators, Hacker Tools and Scrapers (bots used for unauthorized data extraction). According to the study’s findings, humans account for 48.2 percent of users, while the bad robots outnumber the good ones at 28.9 and 22.9 percent respectively.
“While the number of robot traffic we detected in 2018 is only 32 percent, down from 60 percent in 2017, that figure varies greatly by month and, as recently as April 2017, was as high as 85 percent,” Dianomi states the report, which goes on to state that robots have averaged 38 percent of clicks since 2013. However, that number varies significantly from year-to-year, and even more wildly from month-to-month.
Robot clicks by publisher varied from 2 to 100 percent from 2013 to 2018, and that’s after disqualifying publishers that generated less than 10,000 clicks during that time. There doesn’t seem to be much correlation between the size of the publisher in terms of clicks delivered and the percent of robot clicks. The same trends appeared when Dianomi looked at publishers that delivered 100,000 clicks.
As for the cause of the bot epidemic, Dianomi cites the Association of National Advertisers’ bot fraud report, which states: “Behind every big bot problem, someone is paying a traffic source,” after observing that sourced traffic has 3.6x the amount of fraud as non-sourced traffic—suggesting that some publishers are buying traffic from questionable sources and are getting robot clicks.
Both the ANA and Dianomi found that bots are becoming much better at mimicking human behavior or at least working with them. In its 2016-17 study, the ANA found that 75 percent of fraud came from computers that had both human and robot users on them at the same time. Dianomi wrote that bots are showing greater sophistication, such as having a browser’s built-in user agent, which performs tasks like optimizing a website to better work on a device, to spread clicks over a longer period of time and over several IP addresses.
Dianomi recommends that publishers carefully measure the amount of robot traffic that may have been delivered when buying clicks. There are standard ways that benign robots identify themselves so that they can be programmed to know where not to click.
Malicious robots that don’t obey these rules can also be detected because they still act in ways that are not humanlike, “like measuring ad viewability of display advertising, measuring the number of robot clicks is critical to achieving ROI on your ad spend,” states Dianomi.
The report has four tips for combating bot fraud:
- Use a third-party platform to analyze the clicks
- Automatically void any clicks coming from high-risk IP addresses.
- Ask your ad partners to provide full transparency of the clicks with time, IP address, user agent and other data, and whether they have validated or voided them.
- Check any IP addresses generating click and impression counts over thresholds for any hour, day or week based on monthly and daily reports.
- For extra certainty, clicks can be passed through a captcha provided by Google, and the user may need to authenticate if the platform deems the click as suspicious.
