Anubis: The Guardian Who Blocks AI Bots and Protects Your Servers From Chaos

Sysadmins often have to deal with attacks that drain server resources, increase hosting costs, and compromise data privacy in a digital world increasingly invaded by automated bots and AI scrapers. An innovative and powerful solution is Anubis, an open-source utility inspired by the Egyptian god who “weighs souls”: a Proof-of-Work (PoW)-based reverse proxy that filters incoming connections without using intrusive CAPTCHAs.

To date, Anubis has been adopted by groups such as GNOME, FFmpeg, and UNESCO. It is lightweight and protects privacy. This article will explore some technical details of Anubis, from architecture to installation, to help you integrate it into your infrastructure and protect your web environments from DDoS attacks and unauthorized scraping.

What is Anubis and Why Should You Consider It? Link to heading

Anubis is an open-source Web AI Firewall utility designed to protect websites, APIs, and self-hosted services from AI bots and scrapers. Anubis functions as a reverse proxy that “weighs the risk” of each incoming HTTP request. This sets it apart from traditional defense systems, which often rely on IP blacklists or visual CAPTCHAs that are easily circumvented by advanced artificial intelligence. It identifies and blocks bots before they can consume resources further down the line using a combination of heuristics and PoW challenges.

The software is particularly useful in situations where scrapers generate a large amount of traffic and ignore robots.txt and fake legitimate User-Agents. These scenarios include OpenAI or Perplexity AI crawlers, which account for over 50% of automated visits on many sites. Anubis has been shown to reduce downtime and costs by blocking 90% of unwanted traffic in real-world cases, such as at Duke University libraries, since its launch in 2025.

Why consider it? It’s lightweight, requires no visible user interaction, and maintains privacy without tracking data. For system administrators, it means lower management costs and scalable defense against new AI threats.

How Anubis Works Link to heading

Anubis acts as a bridge between your external reverse proxy, such as Nginx or Caddy, and your target service. The main workflow is as follows:

1. Initial assessment using heuristics: Each HTTP request is examined using metadata such as User-Agent, headers (such as Accept-Encoding, Sec-Ch-Ua), IP address, and behavior patterns. Anubis classifies requests using customizable rules: accept (pass directly), block (block), or challenge (PoW challenge).
2. Proof-of-Work (PoW) challenge: Anubis sends an HTML page containing JavaScript if the request is suspicious. The browser must perform an SHA256-based puzzle: hash a challenge (derived from metadata requests and nonce) until it obtains a hash with a predefined number of leading zeros. This occurs in a Web Worker to avoid blocking the user interface, and on a modern browser it only takes milliseconds.
3. Tokens and verifications: The client sends the solution to the server after solving it. Anubis validates and creates a signed JWT (which is the key that was created at the beginning) that is saved in a cookie valid for one week. Subsequent requests containing a valid cookie are fulfilled; otherwise, a new challenge will be issued. To prevent abuse, the server can randomly revalidate.

Anubis is effective because many bots do not execute JavaScript or support modern features such as ES6 modules and the Web Crypto API, and therefore fail the verification. It does not use reputation databases; everything is handled locally and focuses on privacy.

Step-by-Step Installation and Configuration Link to heading

Anubis is designed to be placed between your reverse proxy (such as Nginx or Caddy) and your target service; it is easy to deploy, with options for containerized or bare-metal environments.

The first steps to configure a basic version of the stack described above are:

1. Set up your own website

2. Configure Anubis to point to your site. Whether you are using the Docker installation or the native package, the only essential variable is:

TARGET="http://localhost:3000"

You can consult the list of environment variables to customize Anubis.

3. Configure the reverse proxy in proxy pass mode

server {
    listen 80;
    location / {
        proxy_pass http://anubis:8923;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

I have already set up a basic configuration in my GitHub repository, built entirely with Docker to enable quick and repeatable test runs.

Next steps:

• Start with low difficulty; monitor logs.
• Create an allowlist containing useful bots (e.g., Googlebot).
• Integrate with alerting (Prometheus + Grafana).
• Test in staging and update regularly to avoid blocking legitimate traffic.

Use Cases and Integrations Link to heading

• API and Static Site Protection: Block scrapers on Forgejo/Gitea, blocking 78,000+ requests per day.
• AI Training Defense: Organizations such as UNESCO use Anubis to prevent data theft for AI models.
• Load Balancer Integration: Place it behind GCP LB or AWS ALB for extra layers.
• Comparisons: reCAPTCHA (has paid models, less privacy), hCaptcha (invasive if using a VPN); Anubis is free, open-source, and self-hosted.
• Honeypots for Tuning: Use decoy servers to log scrapers and update policies, as with DuckDB for SQL queries on patterns (e.g., User-Agent without Sec-Ch-Ua).

Advantages and Limitations Link to heading

Advantages:

• Effective against AI (blocks 90% of bots without JS).
• Scalable, low overhead.
• Open-source, supported by the community.
• Reduces costs: Less outgoing traffic, less disruption.

Limitations:

• Depends on JS: Blocks no-JS users or text browsers.
• Default paranoid: Blocks “good bots”; requires tuning.
• Does not support WebSockets.
• Expensive PoW for bots at scale, but minimal overhead for humans (100-500ms).

Conclusion Link to heading

Anubis marks a shift in web defense, transforming the fight against AI bots into a smart, proactive strategy. It is an essential tool for administrators to keep servers and data secure without compromising the user experience. Anubis is an essential tool for protecting your infrastructure in a future where AI scraping will become even more aggressive.