# Chat completions

> Reference for POST /v1/chat/completions and POST /v1/generations/{id}/stop — request fields, headers, the chat completion object, the server-sent event stream and its chunk format, errors and stopping a generation.

Chat completions generate the next assistant message in a conversation. The endpoint follows the OpenAI chat-completions API, so the OpenAI SDKs work against it; the [compatibility notes](/docs/api#openai-compatibility) list the differences.

## Create a chat completion

`POST /v1/chat/completions`

Send the conversation so far and receive the assistant's reply, either as one JSON object or as a stream of events. Needs an API key.

### Request body

| Field | Type | Required | Description |
| --- | --- | --- | --- |
| `model` | string | No | The model to use. Defaults to `Qwen/Qwen2.5-7B-Instruct`. The [model catalog](/models) lists the others. |
| `messages` | array | Yes | The conversation, oldest message first. Between 1 and 512 messages. |
| `messages[].role` | string | Yes | Who wrote the message: `system`, `user`, `assistant` or `tool`. |
| `messages[].content` | string, array or null | Yes | The message text. An array of content parts is accepted: the `text` of each part is joined with line breaks, and parts without text, such as images, are dropped. Only an assistant message with `tool_calls` may have `null` content, or leave it out. |
| `messages[].tool_calls` | array | No | The calls an assistant message made, as the model returned them. Up to 128. |
| `messages[].tool_calls[].id` | string | Yes | The call's ID. The `tool` message that answers it repeats it in `tool_call_id`. |
| `messages[].tool_calls[].type` | string | No | `function`. |
| `messages[].tool_calls[].function.name` | string | Yes | The function that was called. |
| `messages[].tool_calls[].function.arguments` | string | Yes | Its arguments, as JSON text. |
| `messages[].tool_call_id` | string | On `tool` messages | The ID of the call a `tool` message answers. Its `content` is what the function returned. |
| `stream` | boolean | No | Whether to stream the reply as server-sent events. **Defaults to `true`**, unlike the OpenAI API. |
| `stream_options.include_usage` | boolean | No | When streaming, send one more chunk with token usage before the stream ends. Defaults to `false`. |
| `max_tokens` | integer | No | The most tokens to generate. Must be a positive whole number. Defaults to 1,024, and is lowered when the context window has less room left. A deployment that forwards to a model server passes it on, and the model server ends the reply there. |
| `user` | string | No | Names the session this request belongs to when `X-Mortar-Session` isn't sent. See [cache-friendly prompts](/docs/guides/prompt-caching). |
| `tools` | array | No | Functions the model may call, up to 128. See [call tools](#call-tools). |
| `tools[].type` | string | Yes | `function`. |
| `tools[].function.name` | string | Yes | 1 to 64 letters, digits, underscores or dashes, different for each tool. |
| `tools[].function.description` | string | No | What the function does, for the model to read. |
| `tools[].function.parameters` | object | No | A JSON Schema object describing the arguments. |
| `tools[].function.strict` | boolean | No | Asks a model server that supports it to follow `parameters` exactly. |
| `tool_choice` | string or object | No | `auto`, the default when there are tools, lets the model decide. `none` stops it calling any, `required` makes it call at least one, and `{"type": "function", "function": {"name": "get_weather"}}` makes it call that one. Needs `tools`. |
| `parallel_tool_calls` | boolean | No | Whether the model may make more than one call in a reply. Needs `tools`. |

Mortar ignores any other field, such as `temperature`.

The prompt, the reply and 32 tokens of chat-template overhead must fit in the model's context window — 4,096 tokens for `Qwen/Qwen2.5-7B-Instruct` on this deployment, as reported by [`GET /v1/models`](/docs/api/models). If fewer than 64 tokens would be left for the reply, the request fails with `context_length_exceeded`.

### Request headers

| Header | Required | Description |
| --- | --- | --- |
| `Authorization` | Yes | `Bearer` followed by your API key. |
| `Content-Type` | Yes | `application/json`. |
| `X-Mortar-Session` | No | Names the conversation within your organization: up to 128 letters, digits, `-`, `_`, `.` or `:`, and any other character is dropped. Requests in one session are routed to the replica that already holds their prompt prefix. Defaults to `user`; with neither, the request starts a session of its own. |
| `X-Mortar-Optimize` | No | `off` processes the prompt exactly as sent. Opt-in values include `json-tool-blocks` for JSON-fenced data-tool results and `collapse-spaces` for aligned tool output. `on` asks for the lossless defaults only, overriding an organization default of `off`. Any unknown value is refused with `400`. The deployment's mode still applies: in `shadow` mode, the default, nothing is changed, only measured, and when it is `off` no value turns optimization on. Keep the value the same for a whole conversation. |
| `X-Mortar-Tool-Select` | No | `named` keeps only the function explicitly named by object-form `tool_choice`. This is an opt-in progressive-disclosure fast path for clients that send a full catalogue even though they have already selected one function. Other values, or `named` without a named choice, are refused with `400`. |
| `X-Mortar-Dedupe` | No | `in-flight` coalesces concurrently executing, byte-identical non-streaming requests within the same tenant. It requires `stream: false` and an `Idempotency-Key`. A key reused concurrently with a different request is refused with `409`; no completed response is cached. |
| `Idempotency-Key` | With in-flight deduplication | A 1–128 character operation key using letters, digits, `.`, `_`, `-` or `:`. It is scoped to the authenticated tenant and combined with the exact request and optimization/session headers. |
| `X-Request-Id` | No | Your own ID for the request, echoed back in the response. |

### Response headers

| Header | Description |
| --- | --- |
| `X-Mortar-Request-Id` | The completion's ID, the same as `id` in the body. Pass it to [stop the generation](#stop-a-generation). |
| `X-Mortar-Deployment` | The deployment that served the request. |
| `X-Mortar-Model` | The model that generated the reply. |
| `X-Mortar-Upstream` | Present when the deployment forwards requests to a model server, and names that server. |
| `X-Mortar-Session` | The session the request ran in: the one you named, or the one it started. Follow it on the [telemetry stream](/docs/api/telemetry#stream-serving-telemetry). |
| `X-Mortar-Optimizer` | What the prompt optimizer did: a report such as `ruleset=2; mode=on; bytes=541; saved=77; passes=json-compact/1,json-tool-blocks/1; elapsed_us=212`, where `elapsed_us` is how long optimization took in microseconds, or `ruleset=2; mode=off` while it's off. In `shadow` mode, the default, the prompt was used as sent and `saved` is what optimization would have removed. |
| `X-Mortar-Tool-Selector` | Present when named tool selection ran, for example `mode=named; tools=20->1; tokens=2860->144; saved=2716`. Counts use the same tokenizer as prompt planning. |
| `X-Mortar-Dedupe` | `leader` for the request that reached inference, or `shared` for a concurrent request that reused that exact in-flight result. Shared responses intentionally carry the leader's completion ID. |
| `X-Request-Id` | Your `X-Request-Id`, or one Mortar generated. |

### Example

```bash title="Request"
curl https://api.mortar.one/v1/chat/completions \
  -H "Authorization: Bearer $MORTAR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen2.5-7B-Instruct",
    "messages": [
      {"role": "system", "content": "You are a concise assistant for an inference platform."},
      {"role": "user", "content": "What does max_tokens limit?"}
    ],
    "max_tokens": 200,
    "stream": false
  }'
```

```json title="Response" output
{
  "id": "chatcmpl-9b2f4d6e8a0c4b1d3f5e7a9c",
  "object": "chat.completion",
  "created": 1767225600,
  "model": "Qwen/Qwen2.5-7B-Instruct",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "It caps how many tokens the model may generate for this reply."
      },
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 32,
    "completion_tokens": 13,
    "total_tokens": 45
  }
}
```

### The chat completion object

| Field | Type | Description |
| --- | --- | --- |
| `id` | string | `chatcmpl-` followed by 24 hexadecimal characters. |
| `object` | string | Always `chat.completion`. |
| `created` | integer | When the request was accepted, in Unix seconds. |
| `model` | string | The model that generated the reply. |
| `choices` | array | Always exactly one choice. |
| `choices[].index` | integer | Always `0`. |
| `choices[].message.role` | string | Always `assistant`. |
| `choices[].message.content` | string or null | The reply, or `null` when the reply only calls tools. |
| `choices[].message.tool_calls` | array | Present when the model calls tools: the calls, in order. |
| `choices[].message.tool_calls[].id` | string | The call's ID, to repeat in the `tool_call_id` of the message that answers it. |
| `choices[].message.tool_calls[].type` | string | Always `function`. |
| `choices[].message.tool_calls[].function.name` | string | The function to call. |
| `choices[].message.tool_calls[].function.arguments` | string | Its arguments, as JSON text. Parse them before you trust them: a model can produce invalid JSON. |
| `choices[].logprobs` | null | Always `null`. Log probabilities aren't returned. |
| `choices[].finish_reason` | string | `stop` when the reply is complete, `tool_calls` when it calls tools, and `length` when it was cut off by `max_tokens` or the context window. When the deployment forwards to a model server, that server's own reason is passed through. A server that ignores `max_tokens` is stopped by Mortar at three times it plus 64 tokens, by Mortar's estimate, with `length`. |
| `usage.prompt_tokens` | integer | Tokens in the prompt, including chat-template framing. |
| `usage.completion_tokens` | integer | Tokens in the reply. |
| `usage.total_tokens` | integer | The sum of the two. |

When the deployment forwards to a model server that reports usage, these are that server's own counts, and the `prompt_tokens_details` and `completion_tokens_details` it sends, such as `cached_tokens`, are passed through unchanged. Otherwise, and for a reply Mortar ended itself because the model server ran on past `max_tokens`, they're Mortar's own estimate, the same one its context-window check uses. Either way `prompt_tokens` counts the prompt after the prompt optimizer, which is the prompt the model reads, and the [telemetry stream](/docs/api/telemetry#reading-cache-and-cost-events) says in `usageSource` which kind of count the request was billed on.

## Streaming

With `stream` true — or left out — the response is a `text/event-stream`. Each event is one `data:` line holding a `chat.completion.chunk`, followed by a blank line, and the stream ends with `data: [DONE]`.

This request leaves `stream` out, so the reply streams:

```bash title="Request"
curl -N https://api.mortar.one/v1/chat/completions \
  -H "Authorization: Bearer $MORTAR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "What does max_tokens limit?"}],
    "max_tokens": 200
  }'
```

```text title="Response" output wrap
data: {"id":"chatcmpl-1e3a5c7b9d0f2a4c6e8b0d1f","object":"chat.completion.chunk","created":1767225600,"model":"Qwen/Qwen2.5-7B-Instruct","choices":[{"index":0,"delta":{"role":"assistant"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-1e3a5c7b9d0f2a4c6e8b0d1f","object":"chat.completion.chunk","created":1767225600,"model":"Qwen/Qwen2.5-7B-Instruct","choices":[{"index":0,"delta":{"content":"It caps how"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-1e3a5c7b9d0f2a4c6e8b0d1f","object":"chat.completion.chunk","created":1767225600,"model":"Qwen/Qwen2.5-7B-Instruct","choices":[{"index":0,"logprobs":null,"finish_reason":"stop"}]}

data: [DONE]
```

The chunks arrive in this order:

1. One chunk whose `delta` holds `role: "assistant"` and no text.
2. One chunk per piece of the reply, each with the next text in `delta.content`.
3. One chunk with `finish_reason` set and no `delta`.
4. With `stream_options.include_usage`, one chunk with an empty `choices` array and a `usage` object.
5. `data: [DONE]`.

### The chunk object

| Field | Type | Description |
| --- | --- | --- |
| `id` | string | The completion's ID, the same on every chunk. |
| `object` | string | Always `chat.completion.chunk`. |
| `created` | integer | When the request was accepted, in Unix seconds. |
| `model` | string | The model generating the reply. |
| `choices` | array | One choice, or none on the usage chunk. |
| `choices[].index` | integer | Always `0`. |
| `choices[].delta.role` | string | `assistant`, on the first chunk only. |
| `choices[].delta.content` | string | The next piece of the reply. |
| `choices[].delta.tool_calls` | array | The next piece of the tool calls, when the model calls tools. |
| `choices[].delta.tool_calls[].index` | integer | Which call the piece belongs to, counting from `0`. Pieces of parallel calls can interleave. |
| `choices[].delta.tool_calls[].id` | string | The call's ID, in its first piece. |
| `choices[].delta.tool_calls[].type` | string | `function`, in its first piece. |
| `choices[].delta.tool_calls[].function.name` | string | The function, in its first piece. |
| `choices[].delta.tool_calls[].function.arguments` | string | The next piece of the arguments. Join a call's pieces in order to get its JSON. |
| `choices[].logprobs` | null | Always `null`. |
| `choices[].finish_reason` | string or null | `null` on every chunk except the finish chunk, which carries `stop`, `tool_calls` or `length`. |
| `usage` | object | Only on the usage chunk: `prompt_tokens`, `completion_tokens` and `total_tokens`, counted as for a [chat completion](#the-chat-completion-object). |

### Usage in a stream

Ask for usage and Mortar adds one chunk after the finish chunk:

```bash title="Request"
curl -N https://api.mortar.one/v1/chat/completions \
  -H "Authorization: Bearer $MORTAR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "What does max_tokens limit?"}],
    "max_tokens": 200,
    "stream": true,
    "stream_options": {"include_usage": true}
  }'
```

```text title="Response, last three events" output wrap
data: {"id":"chatcmpl-7c9e1a3b5d0f2c4e6a8b0d2f","object":"chat.completion.chunk","created":1767225600,"model":"Qwen/Qwen2.5-7B-Instruct","choices":[{"index":0,"logprobs":null,"finish_reason":"stop"}]}

data: {"id":"chatcmpl-7c9e1a3b5d0f2c4e6a8b0d2f","object":"chat.completion.chunk","created":1767225600,"model":"Qwen/Qwen2.5-7B-Instruct","choices":[],"usage":{"prompt_tokens":17,"completion_tokens":13,"total_tokens":30}}

data: [DONE]
```

### When a stream ends early

- **A model-server error mid-stream** arrives as one event holding the [error envelope](/docs/api#errors) with `type` set to `upstream_error`, and the stream closes without `[DONE]`. A stream that closes without `[DONE]` is incomplete; treat it as failed.
- **A stopped generation** sends no finish chunk. The usage chunk, if you asked for it, and `[DONE]` still follow.
- **Keep-alive comments.** When the deployment forwards to a model server, Mortar sends a `: ping` comment line every 10 seconds so proxies don't close a quiet connection. Event-stream parsers ignore comments.

The [streaming guide](/docs/guides/streaming) shows how to consume the stream in each SDK.

### Errors

| Status | Code | When |
| --- | --- | --- |
| 400 | | The body isn't valid JSON; `messages` is missing, empty or longer than 512 entries; a message lacks a string `role` or a string-or-array `content`; a tool call, a `tool` message's `tool_call_id`, `tools`, `tool_choice` or `parallel_tool_calls` isn't shaped as described above; `max_tokens` isn't a positive whole number; or `X-Mortar-Optimize` has a value Mortar doesn't recognise. |
| 400 | `context_length_exceeded` | Fewer than 64 tokens of the context window are left for the reply. Start a new conversation or send fewer messages. |
| 404 | `model_not_found` | `model` isn't in the catalog. |
| 503 | `model_not_available` | `model` is in the catalog but not deployed yet. |
| 502, 503, 504 | `upstream_error`, `upstream_unavailable`, `upstream_timeout` | The deployment forwards to a model server, and that server failed, was unreachable or timed out. |

```bash title="Request for an unknown model" status=404
curl https://api.mortar.one/v1/chat/completions \
  -H "Authorization: Bearer $MORTAR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}], "stream": false}'
```

```json title="Response" output
{
  "error": {
    "message": "The model `gpt-4o` does not exist.",
    "type": "invalid_request_error",
    "code": "model_not_found"
  }
}
```

The [API overview](/docs/api#status-codes) lists the errors every endpoint can return.

## Call tools

Describe functions in `tools` and the model can answer with calls to them instead of text. The reply then has `finish_reason` `tool_calls`, `null` content, and the calls in `message.tool_calls`. Run them, and send the conversation back with that assistant message as you received it, followed by one `tool` message for each call: its `tool_call_id` names the call and its `content` holds what the function returned. The model reads the results and answers, or calls again.

```bash title="Request"
curl https://api.mortar.one/v1/chat/completions \
  -H "Authorization: Bearer $MORTAR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "What is the weather in Paris?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Current weather for a city",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    }],
    "stream": false
  }'
```

```json title="Response" output
{
  "id": "chatcmpl-4d6f8a0c2e4b6d8f0a2c4e6b",
  "object": "chat.completion",
  "created": 1767225600,
  "model": "Qwen/Qwen2.5-7B-Instruct",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_8e1f3a5c7b9d2e4f6a8c0b1d",
            "type": "function",
            "function": {"name": "get_weather", "arguments": "{\"city\":\"Paris\"}"}
          }
        ]
      },
      "logprobs": null,
      "finish_reason": "tool_calls"
    }
  ],
  "usage": {
    "prompt_tokens": 61,
    "completion_tokens": 9,
    "total_tokens": 70
  }
}
```

Send the result back with the call that asked for it:

```bash title="Request"
curl https://api.mortar.one/v1/chat/completions \
  -H "Authorization: Bearer $MORTAR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "user", "content": "What is the weather in Paris?"},
      {"role": "assistant", "content": null, "tool_calls": [{
        "id": "call_8e1f3a5c7b9d2e4f6a8c0b1d",
        "type": "function",
        "function": {"name": "get_weather", "arguments": "{\"city\":\"Paris\"}"}
      }]},
      {"role": "tool", "tool_call_id": "call_8e1f3a5c7b9d2e4f6a8c0b1d", "content": "{\"temp_c\": 21, \"sky\": \"clear\"}"}
    ],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Current weather for a city",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    }],
    "stream": false
  }'
```

```json title="Response, abbreviated" output
{
  "object": "chat.completion",
  "choices": [
    {
      "index": 0,
      "message": {"role": "assistant", "content": "It's 21 °C and clear in Paris."},
      "logprobs": null,
      "finish_reason": "stop"
    }
  ]
}
```

Streamed, each call arrives in pieces. The first piece of a call carries its `index`, `id`, `type` and function `name`; the pieces after it carry only the `index` and the next part of the `arguments`. Join each call's `arguments` in order. With `parallel_tool_calls`, the pieces of several calls can interleave, and `index` tells them apart.

```bash title="Request"
curl -N https://api.mortar.one/v1/chat/completions \
  -H "Authorization: Bearer $MORTAR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "What is the weather in Paris?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}
      }
    }],
    "tool_choice": {"type": "function", "function": {"name": "get_weather"}}
  }'
```

```text title="Response" output wrap
data: {"id":"chatcmpl-6b8d0f2a4c6e8a0b2d4f6a8c","object":"chat.completion.chunk","created":1767225600,"model":"Qwen/Qwen2.5-7B-Instruct","choices":[{"index":0,"delta":{"role":"assistant"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-6b8d0f2a4c6e8a0b2d4f6a8c","object":"chat.completion.chunk","created":1767225600,"model":"Qwen/Qwen2.5-7B-Instruct","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call_2b4d6f8a0c1e3a5c7e9b1d3f","type":"function","function":{"name":"get_weather","arguments":""}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-6b8d0f2a4c6e8a0b2d4f6a8c","object":"chat.completion.chunk","created":1767225600,"model":"Qwen/Qwen2.5-7B-Instruct","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\"city\":\"Paris\"}"}}]},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-6b8d0f2a4c6e8a0b2d4f6a8c","object":"chat.completion.chunk","created":1767225600,"model":"Qwen/Qwen2.5-7B-Instruct","choices":[{"index":0,"logprobs":null,"finish_reason":"tool_calls"}]}

data: [DONE]
```

When the deployment forwards to a model server, the tools, `tool_choice` and `parallel_tool_calls` go to it as you sent them, and its calls come back as it made them, streamed pieces included. With no model server, Mortar's scripted model calls tools itself, so an agent loop can be tested without one. It calls the function `tool_choice` names, or every tool for `required`, or only the first when `parallel_tool_calls` is `false`. Otherwise it calls the first tool, until the conversation ends with tool results, which it answers in text. It fills each required argument with an empty value of its type.

## Stop a generation

`POST /v1/generations/{id}/stop`

Stops a chat completion while it is still being generated. Closing the connection stops a generation too; use this endpoint when the request that is generating isn't yours to close, such as a cancel button handled by another server. Needs an API key from the organization that started the generation.

### Path parameters

| Parameter | Type | Description |
| --- | --- | --- |
| `id` | string | The completion's ID, from `id` in any chunk or from the `X-Mortar-Request-Id` header. |

### Example

```bash title="Request" status="200 404"
curl -X POST https://api.mortar.one/v1/generations/chatcmpl-1e3a5c7b9d0f2a4c6e8b0d1f/stop \
  -H "Authorization: Bearer $MORTAR_API_KEY"
```

```json title="Response" output
{
  "stopped": true,
  "id": "chatcmpl-1e3a5c7b9d0f2a4c6e8b0d1f"
}
```

### Response

| Field | Type | Description |
| --- | --- | --- |
| `stopped` | boolean | `true` if a running generation was stopped. |
| `id` | string | The ID you sent. |

The status is `200` when a generation was stopped and `404` when none with that ID is running for your organization: it already finished, the ID is wrong, or another organization started it. Mortar answers all three the same way, so the endpoint can't be used to find out which IDs are live. Both statuses carry the same body; the `404` doesn't use the error envelope.