Service Tiers

This feature is in Private Preview. For access or more information, contact us or reach out to your account representative.

Prioritize requests on the Cerebras Inference API to balance latency sensitivity and resource allocation across your workloads.

Service tiers are not available with Shared Inference. The examples on this page assume a dedicated endpoint with service-tier support enabled.

Service tiers determine the processing priority of your requests. You can specify a tier using the service_tier parameter in your API requests.

Tier	Description
`priority`	Uses the highest configured processing priority. If the request cannot be served at that priority, it is retried at the request’s standard priority. Use for time-critical, user-facing requests.
`default`	Standard priority processing. Use for standard production workloads with normal latency requirements.
`auto`	Starts with standard quota and priority, then falls back to `flex` when standard rate limits or available capacity cannot serve the request. Use when maximizing completed requests is more important than maintaining a fixed priority.
`flex`	Uses the lowest configured processing priority and separate flex rate limits. Use for overflow, background work, or experiments that can tolerate additional latency.

When no service_tier is specified, requests default to the default tier.

Usage

Add the service_tier parameter to your chat completions request to specify the priority level.

from cerebras.cloud.sdk import Cerebras
import os

client = Cerebras(api_key=os.environ.get("CEREBRAS_API_KEY"))

response = client.chat.completions.create(
    model="gpt-oss-120b",
    messages=[{"role": "system", "content": "Best pastries in San Francisco?"}],
    stream=True,
    max_tokens=20000,
    temperature=0.7,
    top_p=0.8,
    service_tier="auto"
)

import Cerebras from '@cerebras/cerebras_cloud_sdk';

const client = new Cerebras({
  apiKey: process.env['CEREBRAS_API_KEY'],
});

const response = await client.chat.completions.create({
  model: "gpt-oss-120b",
  messages: [{ role: "system", content: "Best pastries in San Francisco?" }],
  stream: true,
  max_tokens: 20000,
  temperature: 0.7,
  top_p: 0.8,
  service_tier: "auto"
});

curl --location 'https://api.cerebras.ai/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer ${CEREBRAS_API_KEY}" \
--data '{
  "model": "llama-3.3-70b",
  "stream": true,
  "max_tokens": 20000,
  "temperature": 0.7,
  "top_p": 0.8,
  "service_tier": "auto",
  "messages": [
    {
      "role": "system",
      "content": "Best pastries in San Francisco?"
    }
  ]
}'

When service-tier handling is applied, the response includes service_tier with the effective tier used to process the request. An auto request returns auto when it is served at its standard priority and flex when it falls back. A priority request returns priority when it is served at the higher priority and auto when it is retried at its standard priority. Requests using the default behavior may omit this field or return null.

Queue Threshold Control

Only applies to requests using the flex or auto service tiers.

The queue_threshold header allows you to set a maximum acceptable queue time for flex tier requests. If the expected queue time exceeds your threshold, the request is preemptively rejected rather than waiting in the queue. Valid range: 50-20000 milliseconds

from cerebras.cloud.sdk import Cerebras
import os

client = Cerebras(api_key=os.environ.get("CEREBRAS_API_KEY"))

response = client.chat.completions.create(
    model="gpt-oss-120b",
    service_tier="flex",
    messages=[{"role": "user", "content": "What are the latest AI trends?"}],
    extra_headers={"queue_threshold": "100"}
)

import Cerebras from '@cerebras/cerebras_cloud_sdk';

const client = new Cerebras({
  apiKey: process.env['CEREBRAS_API_KEY'],
});

const response = await client.chat.completions.create({
  model: "gpt-oss-120b",
  service_tier: "flex",
  messages: [{ role: "user", content: "What are the latest AI trends?" }]
}, {
  headers: { "queue_threshold": "100" }
});

curl https://api.cerebras.ai/v1/chat/completions \
  -H "Authorization: Bearer $CEREBRAS_API_KEY" \
  -H "Content-Type: application/json" \
  -H "queue_threshold: 100" \
  -d '{
    "model": "llama-3.3-70b",
    "service_tier": "flex",
    "messages": [
      {
        "role": "user",
        "content": "What are the latest AI trends?"
      }
    ]
  }'

If no threshold is specified, a system default is used.

FAQ

How do rate limits apply across service tiers?

priority and default rate limits are the same, while flex rate limits are tracked independently and are several multiples of default rate limits.

Are priority, flex, or auto logged differently in usage tracking?

Yes. Log in to cloud.cerebras.ai and click Analytics. Graphs in the analytics tab display usage across different service tiers, allowing you to monitor consumption by priority level.

Are priority, flex, or auto billed differently than default?

No, during the preview launch all service tiers are billed equally.

Will my request ever be processed on a lower service tier if I do not set service_tier to auto?

An auto request can fall back to flex when standard rate limits or capacity are unavailable. A priority request is attempted at its configured higher priority first, but it can be retried at the request’s standard priority if it cannot be admitted there. default and flex requests do not cascade to another tier.

Can I set queue time threshold on other service tiers?

The queue time threshold only applies once a request is being processed on the flex service tier. You can set it on requests using auto or flex, but it will only be evaluated if the request is processed on the flex tier.

Get Started

Models

Capabilities

Dedicated Inference

Compatibility

Cloud Console

Resources

Support

Service Tiers

Service Tiers

Usage

Queue Threshold Control

FAQ

​Service Tiers

​Usage

​Queue Threshold Control

​FAQ

Service Tiers

Usage

Queue Threshold Control

FAQ