Serving telemetry

Mortar reports what happened to each request on a separate event stream, so the chat-completions response stays exactly OpenAI-shaped. Subscribe before you send a request and you'll see its routing decision, prefix-cache outcome, latency and cost as they happen.

Both streams use named events: each frame has an event: line naming it and a data: line holding a JSON object. A browser EventSource can listen for each name separately.

Stream serving telemetry

GET /v1/telemetry/stream

Follows the requests in one of your organization's sessions. Send your API key to see them: the stream carries events only from requests made by the same organization in the same session.

A key is optional, so a browser can open the stream. Without one, the stream still opens, but it belongs to that connection alone and no request can publish to it, so it stays empty.

Query parameters

Example

curl -N "https://api.mortar.one/v1/telemetry/stream?session=3e7c9f1a-5b2d-4c8e-9a6f-0d1b2c3e4f5a" \
  -H "Authorization: Bearer $MORTAR_API_KEY"

event: stream.open
data: {"sessionId":"3e7c9f1a-5b2d-4c8e-9a6f-0d1b2c3e4f5a","heartbeatMs":15000,"serverTime":1767225600000}

stream.open arrives at once. After it, Mortar sends one event for each step of every request your organization makes in the session. Every event after stream.open carries at, the time it was published in Unix milliseconds, and events about a completion carry its requestId. A : ping comment every 15 seconds keeps idle connections open.

Events

Reading cache and cost events

reuseRatio is the share of the prompt served from the prefix cache, from 0 to 1. tier says where the cached prefix was found: gpu-hbm, host-memory or storage, and null on a miss.

In cost.update, cacheReadTokens are the prompt tokens read from the prefix cache, priced at the model's costPerMillionCacheRead. cacheWriteTokens are the ones processed and written to the cache for the next request to read, priced at its costPerMillionCacheWrite. Both prices are in GET /v1/models. cacheReadUsd and cacheWriteUsd are what each cost, and inputUsd includes both. billedPromptTokens equals cacheWriteTokens and stays for older clients.

baseline names what baselineUsd prices: Same model, no prefix cache, with every prompt token at the input price. savedUsd is baselineUsd minus totalUsd. A cache write costs more than an input token, so savedUsd is negative when the writes outweigh the reads, as on a cold first request, which reads nothing:

{
  "at": 1767225600000,
  "baseline": "Same model, no prefix cache",
  "baselineUsd": 0.0000441886,
  "billedPromptTokens": 502,
  "cacheReadTokens": 0,
  "cacheReadUsd": 0,
  "cacheWriteTokens": 502,
  "cacheWriteUsd": 0.00001964075,
  "completionTokens": 120,
  "inputUsd": 0.00001964075,
  "outputUsd": 0.000028476,
  "promptTokens": 502,
  "requestId": "chatcmpl-7c1e9a3f5b2d4c6e8a0f1b3d",
  "savedUsd": -0.00000392815,
  "totalTokens": 622,
  "totalUsd": 0.00004811675,
  "usageSource": "estimate"
}

usageSource says where the token counts came from. It's upstream when the deployment forwards to a model server that reported usage: the counts are that server's, and its cached_tokens are the cache reads. It's estimate when Mortar counted them itself: there's no model server, as for the request above, or the server reported none, or it ran on past max_tokens until Mortar ended the reply.

session.metrics adds up the session's requests the same way: its savedUsd is the sum of theirs, reusedTokens equals cacheReadTokens, and prefilledTokens is the prompt tokens not read from the cache, which equals cacheWriteTokens unless a model server reported fewer writes. Its usageSource is upstream or estimate when every request's was, and mixed otherwise. Amounts are US dollars at sample pricing, sent to 12 significant digits. The session adds up its requests' exact amounts and rounds only what it sends, so the rounding never accumulates.

Stream fleet metrics

GET /v1/metrics/stream

Sends fleet-wide counters every 2 seconds, for dashboards such as the one on Mortar's home page. The figures are sample data that move within realistic bounds, not measurements of your traffic. No key is needed.

Example

curl -N https://api.mortar.one/v1/metrics/stream

event: metrics.tick
data: {"requests":4820148,"tokensProcessed":1900081374,"medianTtftMs":210,"blendedCostPerMillion":1.53,"cacheHitRate":0.62,"requestsPerSecond":148,"activeReplicas":6,"at":1767225600000}

Events