# Write cache-friendly prompts

> Order prompts so Mortar's prefix cache can reuse them. Prefix caches match from the first token, so stable content goes first and volatile values last — one UUID at position 0 invalidates the whole prefix.

Before a model writes its first token, it processes the whole prompt. That step, prefill, is most of the cost and most of the wait for a short reply. A prefix cache skips it for any part of the prompt the replica has already processed — but only for a prefix: matching starts at the first token and stops at the first difference.

## How prefix matching works

A replica keeps the processed state of prompts it has seen. When a request arrives, the cache compares its prompt with what it holds, token by token from the start. Everything up to the first token that differs is reused. Everything after it is processed again, even text that also appeared in the earlier prompt.

So the position of a change matters more than its size:

- A new question at the end of a long conversation reuses everything before it.
- One character changed at the start reuses nothing.

> [!WARNING]
> One request ID, timestamp or UUID at position 0 invalidates the whole prefix. If your system prompt opens with `Request 3f1c… at 09:30:02`, no two requests share even their first tokens, and every request pays to process its full prompt.

## Put stable content first

Order a prompt from the content that changes least to the content that changes most:

1. Instructions, policies and tool definitions, which are the same for every request.
2. Fixed examples.
3. Reference material, such as retrieved documents, in a deterministic order.
4. The conversation so far, appended to and never rewritten.
5. The new user message.
6. Values unique to this request: IDs, timestamps, the current user's name.

The same request, two ways:

```python tab="Cache-hostile" group="Ordering a prompt"
system = (
    f"Request {request_id} at {datetime.now().isoformat()}. "
    f"You are Acme's support assistant. {POLICY}"
)
messages = [
    {"role": "system", "content": system},
    *history,
    {"role": "user", "content": question},
]
```

```python tab="Cache-friendly"
system = f"You are Acme's support assistant. {POLICY}"
messages = [
    {"role": "system", "content": system},
    *history,
    {
        "role": "user",
        "content": f"{question}\n\nRequest {request_id} at {datetime.now().isoformat()}.",
    },
]
```

In the first version the system message differs on every request, so nothing after the template's opening tokens is ever reused. In the second, the system prompt and the whole history are reused, and only the new message is processed.

## Keep the prefix byte-identical

Text that looks the same isn't always the same. These break a prefix without anyone meaning to:

- **Unstable serialization.** Tool definitions or JSON built from a map with no fixed key order. Sort keys, or build the text once and reuse it.
- **Rewritten history.** Summarising or trimming early turns changes the prompt from that point on. If you must shorten history, do it rarely and in large steps.
- **Reordered documents.** Retrieval results that come back in a different order for the same query. Sort them by a stable key.
- **Whitespace and formatting.** A trailing space or a re-indented template is a different prompt.

## Keep a conversation in one session

Mortar routes each conversation to the replica that already holds its prefix. It recognises a conversation by its session: the `X-Mortar-Session` header, or the `user` field when there's no header. A request with neither starts a session of its own, so nothing it leaves in the cache counts for the next request. Give each conversation a stable session; it belongs to your organization, and Mortar echoes it back in the `X-Mortar-Session` response header:

```bash title="Request"
curl https://api.mortar.one/v1/chat/completions \
  -H "Authorization: Bearer $MORTAR_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Mortar-Session: support-4821" \
  -d '{
    "messages": [
      {"role": "system", "content": "You are a support assistant for Acme."},
      {"role": "user", "content": "How do I rotate an API key?"}
    ],
    "max_tokens": 200,
    "stream": false
  }'
```

Within a session, Mortar measures reuse message by message: the leading messages that are identical, in role and content, to the previous request's are served from the cache. The first message that differs, and everything after it, is processed again — which is why a changing value in the system message costs the whole conversation.

## What caching saves

Each prompt token costs a multiple of the model's input price, depending on what the prefix cache does with it:

| Prompt token | Multiple of the input price |
| --- | --- |
| Read from the cache | 0.1 |
| Processed and written to the cache, for later requests to read | 1.25 |
| Processed with no prefix cache, the baseline every saving is measured against | 1 |

A cold first request reads nothing, so it costs more than it would with no cache. Once more than about 22% of prompt tokens are read from the cache, the reads save more than the writes cost. [`GET /v1/models`](/docs/api/models) publishes each model's cache prices as `costPerMillionCacheRead` and `costPerMillionCacheWrite`. Per-request [telemetry](/docs/api/telemetry#reading-cache-and-cost-events) and the [cost estimate](/docs/api/cost#estimate-a-monthly-bill) both price the cache with them, and the [model catalog](/models) shows each model's input price at a hit rate you choose.

## Measure your hit rate

Follow your session on the [telemetry stream](/docs/api/telemetry), authenticated with the same key, while you send requests. Each request reports:

- `cache.prefix.hit` or `cache.prefix.miss`, with `reusedTokens`, `prefilledTokens` and `reuseRatio`.
- `cost.update`, with `cacheReadTokens`, `cacheWriteTokens` and `savedUsd`, which is negative when a request's cache writes cost more than its reads saved.
- `session.metrics`, with the session's running `cacheHitRate`.

A low hit rate in a conversation that should be warm almost always means something early in the prompt changes on every request. Find it, and move it to the end.

## Check a prompt automatically

The [prompt analyser](/docs/api/cost#analyze-a-prompt) finds these values for you. Send it a prompt, or better, two or more real requests as `variants`: it reports each value that breaks the prefix, how many tokens that wastes on every request and what to move. A prompt that is already in order comes back with no findings.