GPT-5.6 Tier Selection Playbook for API Routing

A routing policy starts failing before anyone notices. The dashboard still looks normal. The API still returns answers. Then one route gets too much high-risk traffic, one fallback model becomes permanent, and one exception becomes “just how this customer works.”

That is where GPT-5.6 tier selection needs to be treated as production policy, not a model preference. OpenAI’s current GPT-5.6 model guidance names gpt-5.6-sol, gpt-5.6-terra, and gpt-5.6-luna, and says the gpt-5.6 alias routes to Sol. Useful. Still not enough.

Convert Workloads Into Routing Classes

Do not start with model names. Start with workload classes.

A “support” task can be a harmless summary or a refund-sensitive decision. A “coding” task can be a formatting pass or a production patch. Labels hide failure cost. Routing classes expose it.

I use four classes: default interactive, escalated interactive, batch/offline, and human-reviewed. That is enough. More classes usually mean the team is encoding politics into infrastructure.

Default traffic is what runs when nothing unusual is happening. Escalated traffic is triggered by risk, ambiguity, customer value, low confidence, or tool failure. Batch traffic has no user-facing latency requirement. Human-reviewed traffic is where automation can draft, but not decide.

Found the pattern on the third try: routing quality improves when every class has an owner, an eval set, and a rollback rule.

Define Task Value, Failure Cost, Latency Budget, and Review Level

Task value is the upside. Failure cost is the damage when the answer is wrong. They are not the same number.

A high-value upsell summary may still be low risk. A short billing response may be high risk. I paused here because the first instinct is to route “hard tasks” upward. That is too soft for production.

Define four fields before picking a route:

task value
failure cost
latency budget
review level

OpenAI’s accuracy guidance makes this practical: define what accuracy means for the task, then evaluate against examples with known good answers. That applies directly to model routing.

For a refund policy answer, the route may require near-zero critical errors and human review below a confidence threshold. For a taxonomy job, schema validity and low retry cost may matter more. For nightly enrichment, waiting hours may be acceptable if the cost changes materially.

The GPT-5.6 API gives route options. The workload class decides which option is allowed.

Separate Default, Escalated, and Batch Traffic

Default traffic should be boring. Stable quality. Predictable latency. Known cost.

Escalated traffic needs explicit triggers. Examples: low retrieval confidence, regulated language, account-level sensitivity, repeated tool errors, conflicting source evidence, or a user request that changes money, access, or legal exposure.

Batch traffic should be separated early. OpenAI’s Batch API documents lower cost than synchronous APIs, higher rate-limit headroom, and completion within 24 hours. That changes the routing decision. A backfill job should not compete with live user requests.

Batch also changes QA. You can sample before delivery, run schema checks, and compare outputs against prior route baselines. That is not slower. That is cleaner.

Build the Tier-Selection Policy

A routing policy should fit on one page.

It needs a default route, escalation triggers, confidence gates, fallback chain, budget limit, human review rule, and rollback owner. If one of those is missing, the router is guessing.

Route by Measurable Thresholds Rather Than Task Labels Alone

Task labels are allowed as hints. They are not enough.

Route by thresholds:

retrieval confidence
expected token count
output schema risk
tool-call count
customer tier
latency budget
safety category
eval pass rate
recent fallback rate
human correction rate

The policy might say: use the default route when retrieval confidence clears the approved threshold, the request stays under the token budget, and the route-level eval score remains above target. Escalate when confidence drops, the task enters a high-failure-cost class, or the output affects a customer decision.

OpenAI’s GPT-5.6 guidance also separates model selection from reasoning.effort, including none, low, medium, high, xhigh, and max. That matters. Sometimes the right route is not another model. It is a different reasoning setting for the same class.

Cost belongs in the same policy. Current OpenAI API pricing lists different prices across GPT-5.6 tiers and service lanes. A route that improves quality slightly but doubles latency and cost may still be wrong for default traffic.

Good enough is a number. Not a mood.

Add Confidence Gates and Human Escalation Conditions

A confidence gate should stop the route, not decorate it.

Useful gates are blunt:

if retrieval confidence is too low, do not answer from memory
if required citations are missing, route to repair or review
if schema validation fails twice, stop retrying
if tool output conflicts with user claims, escalate
if the answer changes money, access, or entitlement, require review

Log the reason every time. route=sol is not enough. route=sol reason=low_retrieval_confidence is useful. route=human reason=billing_policy_conflict is better.

This is where exception volume starts telling the truth. If the same gate fires all day, the policy is not strict. It is under-specified.

Validate Before Moving Traffic

No routing conclusion should go live because it sounded right in review.

Every material routing policy change needs evidence from team evals. Official docs can describe the models. They cannot validate your product workload.

Shadow Tests, Controlled Rollout, and Route-Level Eval Sets

Start with shadow tests. Send real requests through the candidate route, but do not show those outputs to users. Compare them with the current route.

Keep the prompt, tools, retrieval layer, and post-processing stable unless the test is about changing them. Otherwise the result is noise with a spreadsheet.

Build route-level eval sets. A generic eval set is too smooth. Include edge cases: bad retrieval, ambiguous requests, multilingual inputs, long context, schema pressure, tool failure, and high-value customer scenarios.

OpenAI’s agent evals guide points to traces, graders, datasets, and eval runs for repeatable benchmarking. That is the right shape for route validation too. You need to know not only what the model answered, but which route, tools, guardrails, and handoffs produced it.

Rollout should move in stages: shadow, internal users, 1%, 5%, 25%, then broader release. The percentages can change. The staging should not.

Compare Quality Gain Against Incremental Cost and Latency

A higher route is not justified by being better. It is justified by being better enough.

Compare the candidate route against the current one on:

critical failure rate
eval pass rate
human correction rate
p95 latency
retry rate
fallback rate
cost per successful task
cost per prevented failure

That last line is the one I care about. If a stronger route prevents expensive failures, the cost may be rational. If it improves wording on low-risk traffic while increasing latency, keep it out of default.

For GPT-5.6 tier selection, the answer should be route-specific. Sol Terra Luna should not become a slogan. It should become a tested policy.

Operate the Routing Policy

A routing policy starts aging after release.

Traffic mix changes. Prompts get longer. Retrieval quality shifts. One customer adds an exception. Then another. The router still runs, but it is no longer routing the workload it was tested on.

Override Rules, Fallback Chains, and Rollback Authority

Overrides need owners.

Name who can approve a temporary override, who can make a permanent policy change, who can roll back immediately, and where the incident note lives. Without that, the team will improvise during the only moment when improvisation is expensive.

Fallback chains should be explicit. Timeout, rate limit, schema failure, safety refusal, and tool failure should not share one vague retry path.

OpenAI’s rate limits guide notes that unsuccessful requests still count against per-minute limits, so blind retries can make an incident worse. Use backoff, jitter, and route-aware retry limits.

Priority processing is also a separate decision. OpenAI’s Priority processing guide documents service_tier="priority" for lower and more consistent latency, with its own operational considerations. I would reserve it for high-value user-facing traffic where latency is part of the product promise.

A fallback model should protect trust first. If it cannot complete the full task safely, return a narrower answer, ask for review, or defer. Silent degradation is not resilience. It is debt.

Monitor Routing Drift and Exception Volume

Watch route distribution, escalation reasons, fallback usage, override count, eval regression, latency, cost, and customer-specific exceptions.

Also sample decisions manually. Ask two questions every week:

Why did this request route upward? Why did this request stay default?

The second question catches more problems. Exception volume is a policy health metric. If 20% of traffic needs exceptions, the classes are wrong. If one customer has a private routing tree, they need a separate contract. If a fallback route runs every day, it is no longer fallback.

This is where my data ends: routing is not a setup task. It is an operating system for model choice.

FAQ

Which team signs off on routing-policy changes?

Engineering signs off on implementation, observability, fallback behavior, and rollback safety. Product signs off on user experience, latency tolerance, and business tradeoffs. Risk, operations, or support signs off on failure cost and human review rules.

No material model routing change should ship without route-level eval evidence.

How should customer-specific routing exceptions be recorded?

Record them as policy objects.

Each exception should include customer ID, workload class, route change, reason, approver, start date, review date, cost impact, fallback behavior, and removal condition. If it affects production behavior, it should appear in monitoring.

Hidden exceptions create fake averages.

What should an incident review retain after an emergency override?

Keep the route state before the override, trigger condition, approver, affected traffic share, fallback model behavior, latency impact, cost impact, customer impact, and the condition that ended the override. Also keep samples. Logs show where traffic went. Samples show whether the answer was acceptable.

The review should end with one decision: keep the policy, change the threshold, or expand the eval set. For GPT-5.6 tier selection, anything softer will become folklore.

Keep the route state before the override, the trigger, the approving person, traffic percentage affected, fallback model behavior, cost and latency impact, customer impact, and the condition that ended the override. Also keep samples. Logs tell you where the request went. Samples tell you whether the answer was acceptable.

For GPT-5.6 tier selection, the incident review should end with one of three decisions: keep the policy, change the threshold, or add eval coverage. Anything softer turns into folklore.

Previous posts: