Benchmarking Agno against other agent frameworks

People keep asking how fast Agno is

It's a fair question. Benchmarks are the honest way to answer it, so we publish ours: the numbers, the method and the code. This post walks through what we measure, why we measure that one thing and how to check our work on your own machine.

We measure one thing

The temptation with benchmarks is to measure everything. Tokens per second, time to first token, some leaderboard score. Most of that measures the model, and the model is the same no matter which framework calls it.

What a framework actually controls is the machinery around the model. So we measure the piece of machinery you pay for most often in production: creating an agent. Time and memory, per agent.

Why creation cost matters

Here's the chain of reasoning.

An agent holds state. Conversation history, memory, whatever a tool cached along the way. Share one agent between two users and that state travels with it. User A's data can show up in user B's session. Nothing warns you when it happens.

Some frameworks suggest you keep state outside the agent and share the object. That holds up until someone on your team writes a tool that caches something. And someone always does. FastAPI solved this years ago with database sessions. Every request gets its own session, and nothing can leak because nothing is shared. Agents deserve the same pattern.

So the safe way to serve agents is a fresh agent per request. That pattern is only practical if creating an agent costs close to nothing. Our benchmark answers one question: does it?

How the benchmark works

Every framework gets its own script, and every script builds the same thing. One agent, with one tool and a model binding. We create that agent 1,000 times, with 10 warmup runs first. Each framework runs in its own process. We record the median time and the median memory.

There are no API keys and no model calls, so the suite finishes in seconds. We run against the published PyPI releases, the same packages you would install.

The results

Measured August 9, 2026 on an Apple M4 Max, macOS 15.7, Python 3.12.8.

What the numbers mean at scale

Small numbers are hard to feel, so let's convert them. At 3.2 microseconds per agent, one CPU core creates about 300,000 agents a second. A gigabyte of memory holds about 200,000 of them. At that price a fresh agent per request is free. You set it up once and never think about it again.

At 18 milliseconds per agent, the cost shows up in your latency graphs. So you start pooling agents to save time. Pooled agents are shared agents, and shared agents bring back the leak we started with. That's the real cost of slow instantiation. It pushes your architecture toward an unsafe pattern.

Where these numbers stop

I want to be clear about the limits. These benchmarks measure construction only. Once an agent calls a model, the model call takes most of the time. That's true in every framework, and a faster constructor will never make your agent answer faster.

Sometimes creation cost is irrelevant too. One tenant? A shared object you can prove holds no state? Then the cost rounds to zero. It starts to count the moment you serve more than one user, because then you pay it on every request, times your request rate.

Run it yourself

Please don't take our word for it. The harness lives in the Agno repository and each script is about a page of code. Build the framework's own quickstart agent, time it, print the median.

git clone https://github.com/agno-agi/agno.git
cd agno
 
# Set up the environment
./scripts/perf_setup.sh
source .venvs/perfenv/bin/activate
 
# Run the benchmarks
python cookbook/09_evals/performance/instantiate_agent_with_tool.py         # Agno
python cookbook/09_evals/performance/comparison/langgraph_instantiation.py  # LangGraph
python cookbook/09_evals/performance/comparison/crewai_instantiation.py     # CrewAI

The comparison folder also has scripts for PydanticAI, OpenAI Agents, Smolagents and AutoGen if you want to go further.

Your absolute numbers will differ from ours. Different CPU, different Python, different package versions. The gap between frameworks stays stable, and the gap is what matters.

One last thing

If your results look different from ours, please open an issue. We would honestly like to see them. Benchmarks are only useful when anyone can rerun them, and we intend to keep ours that way.