> ## Documentation Index
> Fetch the complete documentation index at: https://inference-docs.cerebras.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Service Tiers

> Control request prioritization with service tiers.

<Callout icon="lock" color="#b2b1b1ff" iconType="regular">
  This feature is in [Private Preview](/support/preview-releases). For access or more information, [contact us](https://www.cerebras.ai/contact) or reach out to your account representative.
</Callout>

Prioritize requests on the Cerebras Inference API to balance latency sensitivity and resource allocation across your workloads.

<Warning>
  Service tiers are not available with Shared Inference. The examples on this page assume a dedicated endpoint with service-tier support enabled.
</Warning>

## Service Tiers

Service tiers determine the processing priority of your requests. You can specify a tier using the [`service_tier`](/api-reference/chat-completions#param-service-tier) parameter in your API requests.

| Tier | Description |
| - | - |
| `priority` | Uses the highest configured processing priority. If the request cannot be served at that priority, it is retried at the request's standard priority. Use for time-critical, user-facing requests. |
| `default` | Standard priority processing. Use for standard production workloads with normal latency requirements. |
| `auto` | Starts with standard quota and priority, then falls back to `flex` when standard rate limits or available capacity cannot serve the request. Use when maximizing completed requests is more important than maintaining a fixed priority. |
| `flex` | Uses the lowest configured processing priority and separate flex rate limits. Use for overflow, background work, or experiments that can tolerate additional latency. |

When no `service_tier` is specified, requests default to the `default` tier.

## Usage

Add the `service_tier` parameter to your chat completions request to specify the priority level.

<CodeGroup>
  ```python Python highlight={13} theme={null}
  from cerebras.cloud.sdk import Cerebras
  import os

  client = Cerebras(api_key=os.environ.get("CEREBRAS_API_KEY"))

  response = client.chat.completions.create(
      model="gpt-oss-120b",
      messages=[{"role": "system", "content": "Best pastries in San Francisco?"}],
      stream=True,
      max_tokens=20000,
      temperature=0.7,
      top_p=0.8,
      service_tier="auto"
  )
  ```

  ```javascript Node.js highlight={14} theme={null}
  import Cerebras from '@cerebras/cerebras_cloud_sdk';

  const client = new Cerebras({
    apiKey: process.env['CEREBRAS_API_KEY'],
  });

  const response = await client.chat.completions.create({
    model: "gpt-oss-120b",
    messages: [{ role: "system", content: "Best pastries in San Francisco?" }],
    stream: true,
    max_tokens: 20000,
    temperature: 0.7,
    top_p: 0.8,
    service_tier: "auto"
  });
  ```

  ```bash cURL highlight={10} theme={null}
  curl --location 'https://api.cerebras.ai/v1/chat/completions' \
  --header 'Content-Type: application/json' \
  --header "Authorization: Bearer ${CEREBRAS_API_KEY}" \
  --data '{
    "model": "llama-3.3-70b",
    "stream": true,
    "max_tokens": 20000,
    "temperature": 0.7,
    "top_p": 0.8,
    "service_tier": "auto",
    "messages": [
      {
        "role": "system",
        "content": "Best pastries in San Francisco?"
      }
    ]
  }'
  ```
</CodeGroup>

When service-tier handling is applied, the response includes `service_tier` with the effective tier used to process the request. An `auto` request returns `auto` when it is served at its standard priority and `flex` when it falls back. A `priority` request returns `priority` when it is served at the higher priority and `auto` when it is retried at its standard priority. Requests using the default behavior may omit this field or return `null`.

## Queue Threshold Control

<Note>Only applies to requests using the `flex` or `auto` service tiers.</Note>

The [`queue_threshold`](/api-reference/chat-completions#param-queue-threshold) header allows you to set a maximum acceptable queue time for flex tier requests. If the expected queue time exceeds your threshold, the request is preemptively rejected rather than waiting in the queue.

**Valid range:** 50-20000 milliseconds

<CodeGroup>
  ```python Python highlight={10} theme={null}
  from cerebras.cloud.sdk import Cerebras
  import os

  client = Cerebras(api_key=os.environ.get("CEREBRAS_API_KEY"))

  response = client.chat.completions.create(
      model="gpt-oss-120b",
      service_tier="flex",
      messages=[{"role": "user", "content": "What are the latest AI trends?"}],
      extra_headers={"queue_threshold": "100"}
  )
  ```

  ```javascript Node.js highlight={12} theme={null}
  import Cerebras from '@cerebras/cerebras_cloud_sdk';

  const client = new Cerebras({
    apiKey: process.env['CEREBRAS_API_KEY'],
  });

  const response = await client.chat.completions.create({
    model: "gpt-oss-120b",
    service_tier: "flex",
    messages: [{ role: "user", content: "What are the latest AI trends?" }]
  }, {
    headers: { "queue_threshold": "100" }
  });
  ```

  ```bash cURL highlight={4} theme={null}
  curl https://api.cerebras.ai/v1/chat/completions \
    -H "Authorization: Bearer $CEREBRAS_API_KEY" \
    -H "Content-Type: application/json" \
    -H "queue_threshold: 100" \
    -d '{
      "model": "llama-3.3-70b",
      "service_tier": "flex",
      "messages": [
        {
          "role": "user",
          "content": "What are the latest AI trends?"
        }
      ]
    }'
  ```
</CodeGroup>

If no threshold is specified, a system default is used.

## FAQ

<AccordionGroup>
  <Accordion title="How do rate limits apply across service tiers?">
    `priority` and `default` rate limits are the same, while `flex` rate limits are tracked independently and are several multiples of `default` rate limits.
  </Accordion>

  <Accordion title="Are priority, flex, or auto logged differently in usage tracking?">
    Yes. Log in to [cloud.cerebras.ai](https://cloud.cerebras.ai) and click **Analytics**. Graphs in the analytics tab display usage across different service tiers, allowing you to monitor consumption by priority level.
  </Accordion>

  <Accordion title="Are priority, flex, or auto billed differently than default?">
    No, during the preview launch all service tiers are billed equally.
  </Accordion>

  <Accordion title="Will my request ever be processed on a lower service tier if I do not set service_tier to auto?">
    An `auto` request can fall back to `flex` when standard rate limits or capacity are unavailable. A `priority` request is attempted at its configured higher priority first, but it can be retried at the request's standard priority if it cannot be admitted there. `default` and `flex` requests do not cascade to another tier.
  </Accordion>

  <Accordion title="Can I set queue time threshold on other service tiers?">
    The queue time threshold only applies once a request is being processed on the `flex` service tier. You can set it on requests using `auto` or `flex`, but it will only be evaluated if the request is processed on the flex tier.
  </Accordion>
</AccordionGroup>