Choose Managed Cloud or BYOC
On this page
Mortar runs in two places. In Managed Cloud, Mortar runs the whole stack and the GPUs. In Bring Your Own Cloud (BYOC), Mortar's data plane runs in your cloud account and network, and Mortar operates it from outside. Your code calls the same API either way.
Compare the two
| What differs | Managed Cloud | Bring Your Own Cloud |
|---|---|---|
| Where inference runs | Mortar's cloud account | Your cloud account and VPC |
| Who owns the GPUs and the bill for them | Mortar | You |
| Where prompts, outputs and weights are processed | Mortar's cloud | Your environment |
| Encryption keys for model storage | Mortar's | Yours, in your key management service |
| Who operates the software | Mortar | Mortar, under the access you grant |
| The base URL your code calls | Mortar's endpoint | An endpoint inside your network |
| Getting started | Pick a model, set limits, get an endpoint | Connect an account, then deploy |
Choose Managed Cloud to reach production fastest. Choose BYOC when prompts or outputs must not leave your cloud boundary, when you have GPU capacity or commitments to use, or when your security review needs inference to stay in your account.
How BYOC works
Mortar splits into two planes:
- The control plane runs in Mortar's cloud. It holds the console, the catalog, policy and billing.
- The data plane runs in your account. It is the gateway, router, model runtime, prefix cache and autoscaler — everything that touches a prompt.
An agent in your account opens an outbound, mutually authenticated TLS connection to the control plane and pulls versioned deployment specs from it. Nothing needs inbound access to your cluster: no ingress rule, no peering and no allow-list on your side.
Your applications call the data plane directly, at an endpoint inside your network such as https://llm.acme-prod.internal/v1. Inference traffic doesn't pass through Mortar's network.
What leaves your account
In BYOC, only operational numbers reach Mortar:
- Request and token counts, by deployment and model
- Latency percentiles
- GPU node-hours and replica counts
- Deployment and health state, so the control plane can reconcile
Request bodies, responses, prompts, embeddings, retrieved documents and model weights are processed and stored inside your account.
Who does what in BYOC
| Mortar runs | You control |
|---|---|
| Building, releasing and operating the control-plane and data-plane software | The cloud account, regions, GPU quotas and billing |
| Upgrades, observability and incident response, under the access you grant | IAM roles, key policies, networking and allowed endpoints |
| Tenancy, routing, caching and scaling policy | Your data, model assets and application ingress |
If the connection to the control plane drops, deployed models and the local gateway keep serving. You lose the ability to change deployments until it's back, not the ability to run them.
Switch between them
Moving between Managed Cloud and BYOC changes one value in your code: the base URL. Keys, request fields, streaming and errors work the same on both.
import os
from openai import OpenAI
client = OpenAI(
base_url=os.environ["MORTAR_BASE_URL"],
api_key=os.environ["MORTAR_API_KEY"],
)Set MORTAR_BASE_URL to https://api.mortar.one/v1 for Managed Cloud, or to your BYOC endpoint.