The Internet isn’t a single lane of traffic. For a long time, the rule of thumb in web security was that bots are bad, while humans are good. Of course, we’re far past this generalization.
Humans can be fraudulent, and bots can be helpful at different levels. Site owners actively want some automated traffic to interact with our sites to make the Internet functional and discoverable. To complicate things further, the line between "human" and "bot" is blurring more and more.
Now, we have a type of “hybrid” traffic where a single session shifts from human to agentic and back again. (Think of a user browsing a store, and then handing off the checkout process to an automated shopping assistant.) So, how do website owners manage this kind of complexity?
What matters here is assessing behaviors . Is this behavior abusive? Malicious?
What’s the risk presented here, and can I trust this visitor based on their actions? Solving this requires moving beyond static, point-in-time checks. It requires analyzing continuous behaviors to evaluate Trust.
In this post, we’ll share an inside look into the strategy of the Web Integrity & Trust team (covering the bots and fraud problem spaces) around detecting and analyzing good and bad behaviors, providing tools to help site owners tackle emerging challenges in the shifting Agentic Internet. We’ll also share findings around agentic traffic since the launch of Precursor , and a simulation where you can see how your own cursor movements would be assessed as human or bot — plus some exciting launch updates to expect in the near future.
Defining Risk and Trust Let’s talk about the distinction between Risk and Trust , the way we discuss it within the teams at Cloudflare who work in bot detection. These are often viewed as polar opposites of a continuum. At Cloudflare, we look at them as independent, but reciprocal, values.
Trust is the essential ingredient in making informed decisions on what to do about your traffic. Risk is how likely something like a request or action is to be harmful, and it’s often ephemeral. Trust, however, is built up over time, and it’s based on reputation.
We can illustrate this with an example from real life: say that you’re enjoying some evening television at home, when suddenly, you hear the doorbell being rung repeatedly. Besides being annoying, this behavior is strange. Frantic doorbell rings late at night are alarming.
You check through your door camera and see that the person ringing your doorbell is your best friend who lives next door. Of course, you trust your best friend, and we’d bet you would let them in. In this example, it wouldn’t be enough for you to say, “Reject anyone who rings my doorbell at night” or “Reject anyone who rings my doorbell more than 10 times.”
Again, Trust is the essential ingredient. Going back to traffic on the Internet, the strategy as we build products in the bots and fraud space focuses on building an entire ecosystem based on Trust. And our goal is to provide the incentives and primitives for site owners to use to incentivize behavior that makes the Internet safer for everyone: starting with blocking malicious activity at the bottom, to encouraging participation in a safer Internet at the top.
Good behaviors, rooted in transparency Starting at the top: what counts as good behavior? We can draw clear examples from the Verified bots and agents within BotBase. Last month, we announced an updated pragmatic taxonomy for the good bots we track in our system , boiling down the definition of “ Verified ” to two things: 1) you declare yourself honestly, and 2) you don’t abuse the trust you’ve earned.
Transparency between a site owner and a bot operator allows for a symbiotic relationship: site owners can note what behaviors and data uses they want to allow on their websites, and bot operators can be granted access more easily. The transparency allows for Trust in the relationship; if you have nothing to hide, declaring who you are should reduce friction from the sites who want to allow your behaviors.
BotBase is not meant to only make declarations of “who is good”. It is intended to be a directory of all known bots and agents, and provide the facts. Compared to our previous Bots Directory, which only included known good bots, BotBase is also capable of tracking less-than-good bots and agents.
Why? Because our systems track and validate behavior for known good actors, meaning we have the tools to identify when these expectations aren’t met. If you abuse trust on the Cloudflare network, you should not be easily allowed, so you will be unverified.
Bad behaviors: blatant, stealthy, and everything in between A few weeks ago, we announced Precursor , a continuous client-side system to detect even subtly inhuman bot traffic that can fly under the radar when assessing network signals alone. When a customer enables Precursor, the JavaScript detection is CDN-injected, so it doesn’t require sitting at the computer and figuring out where or how to rerun these detections.
What's more, Precursor evaluates user behavior continuously throughout the session , so no more free hall passes for abusive traffic that found a way to pass client and browser-side checks just once. Applying our Risk and Trust framework to these client-side detections, we can point out that CAPTCHAs or one-time hurdles are Risk-based, meaning they lack context .
On the other hand, verification using behavioral tells is Trust-based, since it can capture more context clues from the full user session. Precursor is the tool for us to analyze this behavior. To sum it up, Precursor is so powerful because it: Provides Trust-based detection over the entire user session.
Drives up the cost for bot developers to replicate human behavior over a multipage timeline. By making it economically disadvantageous for bot developers to outrun these detections, we win the adversarial game. Now, what have we learned since we’ve launched?
Originally published at blog.cloudflare.com
