Blocking isn't always the right answer for AI traffic — sometimes you want to allow it, but on your terms. Rate limiting (and metering) lets you keep costs and load under control while still serving agents. Here's how to do it well.
Why naive rate limits fail
Per-IP limits break against agents that rotate through residential proxy networks — each request is a "new" IP. You need a key that survives rotation.
Key on the right identity
- Fingerprint, not IP: rate-limit by a stable network/device fingerprint (JA4/QUIC) so rotation doesn't reset the counter.
- ASN/prefix: for IPv6, limit by prefix (e.g. /64), not the full address.
- Account/API key: for known integrations, meter by credential.
Tiered policy by confidence
- Verified human or partner: normal limits.
- Unknown automation: reduced rate + challenge.
- High-confidence abusive bot: block or hard-throttle.
detectip.ai returns an explainable score and a recommended action, so you can map confidence bands directly to limits.
Monetize instead of block
If AI agents bring value (e.g. assistants buying from you), consider a metered tier for verified agents rather than blocking — detect first, then price. See blocking AI scrapers for the policy spectrum.
FAQ
What's the single most important change? Stop rate-limiting purely by IP; add a fingerprint-based key.
Can I keep good bots and limit the rest? Yes — verify declared crawlers, throttle unknowns, block abusers. Start with a free key.