# LLM response caching in Agno

> Response caching stores model responses locally to reduce redundant API calls, cutting both costs and response times during development and testing. This guide explores Agno’s two caching strategies: response caching and prompt caching.

- Published: 2025-11-06
- Author: Nancy Chauhan
- Category: Engineering
- Canonical: https://agno-com-nine.vercel.app/articles/llm-response-caching-in-agno
- Markdown: https://agno-com-nine.vercel.app/articles/llm-response-caching-in-agno.md

**Reduce development costs and time by caching model responses to avoid redundant API calls: perfect for testing, iteration, and cost-conscious development.**

## Overview

Response caching eliminates redundant API calls by storing model responses locally, dramatically reducing both costs and response times during development and testing.

Agno provides two complementary caching strategies:

1. **Response Caching**: Caches the entire model response locally to avoid repeated API calls
2. **Prompt Caching**: Caches system prompts on the provider's side to reduce processing costs (Anthropic, OpenAI, OpenRouter)

### When to Use

- Development and testing workflows
- Unit testing with consistent results
- Cost optimization during prototyping
- Rate limit management
- Offline development

Do not use response caching in production for dynamic content or when you need fresh, up-to-date responses for each query.

## Prerequisites

- Python 3.7 or higher
- Agno library (`pip install -U agno`)
- API keys for your model provider
- Basic understanding of Agno Agents

## Response Caching

Response caching is configured at the model level. See the [full documentation](https://docs.agno.com/concepts/models/cache-response) for details.

#### Enable Caching

```python
from agno.agent import Agent
from agno.models.openai import OpenAIChat

agent = Agent(model=OpenAIChat(id="gpt-4o", cache_response=True))

# First call - cache miss (calls API)
response = agent.run("Write a short story about a cat.")
print(f"First call: {response.metrics.duration:.3f}s")

# Second call - cache hit (instant)
response = agent.run("Write a short story about a cat.")
print(f"Cached call: {response.metrics.duration:.3f}s")
```

[Full example in the cookbook.](https://github.com/agno-agi/agno/blob/main/cookbook/agents/caching/cache_model_response.py)

[View full example](https://github.com/agno-agi/agno/blob/main/cookbook/agents/caching/cache_model_response.py) →

**Configuration Options:**

- `cache_response`: Enable/disable caching (default: `False`)
- `cache_ttl`: Cache expiration in seconds (default: never expires)
- `cache_dir`: Custom cache location (default: `~/.agno/cache/model_responses`)

**Example with options:**

```python
agent = Agent(
    model=OpenAIChat(
        id="gpt-4o",
        cache_response=True,
        cache_ttl=3600,        # Cache expires after 1 hour
        cache_dir="./cache"    # Custom cache directory
    )
)
```

## Prompt Caching

Prompt caching reduces costs by caching system prompts on the provider's side. Learn more about [context caching](https://docs.agno.com/concepts/agents/context#context-caching).

#### Anthropic Prompt Caching

System prompts must be at least 1024 tokens. See [Anthropic's documentation](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching).

```python
from agno.agent import Agent
from agno.models.anthropic import Claude

# Your large system prompt (must be 1024+ tokens)
system_message = "Your comprehensive system prompt here..."

agent = Agent(
    model=Claude(
        id="claude-sonnet-4-20250514",
        cache_system_prompt=True  # Enable prompt caching
    ),
    system_message=system_message,
)

# First run - creates cache
response = agent.run("Explain REST APIs")
print(f"Cache write tokens: {response.metrics.cache_write_tokens}")

# Second run - uses cached prompt
response = agent.run("Explain GraphQL APIs")
print(f"Cache read tokens: {response.metrics.cache_read_tokens}")
```

[View full example](https://github.com/agno-agi/agno/blob/main/cookbook/models/anthropic/prompt_caching.py) →

**Extended Cache Time (Beta):** Extend duration from 5 minutes to 1 hour:

```python
agent = Agent(
    model=Claude(
        id="claude-sonnet-4-20250514",
        default_headers={"anthropic-beta": "extended-cache-ttl-2025-04-11"},
        cache_system_prompt=True,
        extended_cache_time=True  # Extend to 1 hour
    ),
    system_message=system_message,
)
```

[View full example](https://github.com/agno-agi/agno/blob/main/cookbook/models/anthropic/prompt_caching_extended.py) →

### OpenAI & OpenRouter

Prompt caching happens automatically when supported. See [OpenAI](https://platform.openai.com/docs/guides/prompt-caching) and [OpenRouter](https://openrouter.ai/docs/prompt-caching) documentation.

```python
from agno.agent import Agent
from agno.models.openai import OpenAIChat

# Prompt caching happens automatically for supported models
agent = Agent(model=OpenAIChat(id="gpt-4o"))
```

## Advanced Usage

#### Caching with Teams

Enable caching on team members and the team leader:

```python
from agno.agent import Agent
from agno.team import Team
from agno.models.openai import OpenAIChat

# Create agents with caching
researcher = Agent(
    name="Researcher",
    role="Research and gather information",
    model=OpenAIChat(id="gpt-4o", cache_response=True),
)

writer = Agent(
    name="Writer",
    role="Write clear and engaging content",
    model=OpenAIChat(id="gpt-4o", cache_response=True),
)

# Create team with caching
team = Team(
    members=[researcher, writer],
    model=OpenAIChat(id="gpt-4o", cache_response=True),
)

# First run - slow, second run - instant from cache
team.print_response("Research AI trends and write a summary")
```

[Full example in the cookbook.](https://github.com/agno-agi/agno/blob/main/cookbook/teams/basic_flows/caching/cache_team_response.py)

[View full example](https://github.com/agno-agi/agno/blob/main/cookbook/teams/basic_flows/caching/cache_team_response.py) →

#### Combining Both Caching Types

Maximize savings by using response and prompt caching together:

```python
agent = Agent(
    model=Claude(
        id="claude-sonnet-4-20250514",
        cache_response=True,        # Local response caching
        cache_system_prompt=True,   # Provider-side prompt caching
        cache_ttl=7200             # 2 hour cache duration
    ),
    system_message="Your large system prompt...",
)
```

## Monitoring

Track cache performance using metrics:

```python
response = agent.run("Your query")

# Response caching metrics
print(f"Duration: {response.metrics.duration:.3f}s")

# Prompt caching metrics (Anthropic)
print(f"Cache read tokens: {response.metrics.cache_read_tokens}")
print(f"Cache write tokens: {response.metrics.cache_write_tokens}")
```

Learn more about [metrics in Agno](https://docs.agno.com/concepts/telemetry).

## Configuration Reference

#### Response Caching Parameters

![Table of the cache_response, cache_ttl and cache_dir parameters with their types, defaults and descriptions](https://agno-com-nine.vercel.app/images/llm-response-caching-in-agno-screen-shot-2025-11-05-11-03-14.png)

#### Prompt Caching Parameters (Anthropic)

![Table of the cache_system_prompt and extended_cache_time parameters with their types, defaults and descriptions](https://agno-com-nine.vercel.app/images/llm-response-caching-in-agno-screen-shot-2025-11-05-11-03-21.png)

## Key Takeaways

1. **Response caching** saves entire responses locally - use during development
2. **Prompt caching** saves system prompts on provider side - use with large prompts (1024+ tokens)
3. **Combine both** for maximum cost savings
4. **Never use in production** for user-facing dynamic content
5. **Monitor metrics** to verify caching effectiveness

## Resources

**Documentation:**

- [Response Caching Guide](https://docs.agno.com/concepts/models/cache-response)
- [Context Caching for Agents](https://docs.agno.com/concepts/agents/context#context-caching)
- [Context Caching for Teams](https://docs.agno.com/concepts/teams/context#context-caching)
- [Tool Result Caching](https://docs.agno.com/concepts/tools/caching)

**Examples:**

- [Anthropic Prompt Caching](https://docs.agno.com/examples/models/anthropic/prompt_caching)
- [OpenAI Response Caching](https://docs.agno.com/examples/models/openai/chat/cache_response)
- [Session Caching](https://docs.agno.com/examples/concepts/teams/session/cache_session)

**Support:**

- [Agno Documentation](https://docs.agno.com/)
- [Community Forum](https://community.agno.com/)
- [Discord](https://discord.gg/4MtYHHrgA8)
- [GitHub](https://github.com/agno-agi/agno)