Docs
Models Sign in

Models

Last updated View as Markdown Send feedback

On this page
  1. List models
    1. Example
    2. Response
  2. Context windows

The model list tells a client which IDs it can pass as model and how many tokens each one can take. It follows the OpenAI list format, with Mortar's own details in a mortar object on each entry.

List models

GET /v1/models

Lists every model in the catalog, including models that aren't deployed yet. Needs an API key.

Example

Requestbash
curl https://api.mortar.one/v1/models \
  -H "Authorization: Bearer $MORTAR_API_KEY"
Response, first entryjson
{
  "object": "list",
  "data": [
    {
      "id": "Qwen/Qwen2.5-7B-Instruct",
      "object": "model",
      "created": 1767225600,
      "owned_by": "Alibaba Qwen",
      "max_model_len": 4096,
      "permission": [],
      "mortar": {
        "name": "Qwen 2.5 7B Instruct",
        "family": "Alibaba Qwen",
        "size": "7B dense",
        "context": "32K",
        "quant": "BF16 · AWQ INT4",
        "gpu": "1× L4 or A10G",
        "engine": "vLLM",
        "badge": "Popular",
        "available": true,
        "costPerMillionInput": 0.0313,
        "costPerMillionOutput": 0.2373,
        "costPerMillionCacheRead": 0.00313,
        "costPerMillionCacheWrite": 0.039125
      }
    }
  ]
}

Response

Field Type Description
object string Always list.
data array One entry per model in the catalog.
data[].id string The ID to pass as model.
data[].object string Always model.
data[].created integer A fixed timestamp, the same for every model.
data[].owned_by string The model's family, such as Alibaba Qwen.
data[].max_model_len integer The context window this deployment serves, in tokens. Prompt and reply together must fit in it.
data[].permission array Always empty.
data[].mortar.name string The display name.
data[].mortar.family string The model family.
data[].mortar.size string Parameter count and architecture, such as 7B dense.
data[].mortar.context string The longest context the model itself supports, such as 32K.
data[].mortar.quant string The precisions Mortar has tested, such as BF16 · FP8.
data[].mortar.gpu string The GPU the model is sized for.
data[].mortar.engine string The serving engine: vLLM or SGLang.
data[].mortar.badge string or null A catalog label, Popular or Coming soon, or null.
data[].mortar.available boolean Whether the model can serve requests now. Requesting one that can't fails with model_not_available.
data[].mortar.costPerMillionInput number Sample price for a million prompt tokens, in US dollars.
data[].mortar.costPerMillionOutput number Sample price for a million completion tokens, in US dollars.
data[].mortar.costPerMillionCacheRead number Sample price for a million prompt tokens read from the prefix cache: a tenth of the input price.
data[].mortar.costPerMillionCacheWrite number Sample price for a million prompt tokens processed and written to the prefix cache, so a later request can read them: 1.25 times the input price.
Note

Prices are sample pricing, not a quote. Mortar derives them from on-demand GPU list prices and measured throughput at 65% utilization. Prompt caching explains how cache reads and writes add up, and the model catalog shows each model's input price at a cache hit rate you choose.

Context windows

Two fields describe context, and they answer different questions:

  • mortar.context is the most the model can handle.
  • max_model_len is what this deployment serves. Budget max_tokens against it.

They differ when a replica is started with a shorter window than the model allows. Qwen/Qwen2.5-7B-Instruct supports 32K tokens but is served here with 4,096, so a request with a 4,050-token prompt fails even though the model could read it. Chat completions explains how Mortar fits max_tokens into the window.