Routing contract: hard boundaries and soft preferences

The routing contract separates hard application boundaries from soft preferences that vary as pool, context, and operating conditions change.

Design applications around the stable task and terminal-result contract, separating hard tenant boundaries from soft routing preferences that vary at runtime.

Only routes enabled in the selected tenant configuration form the hard outer routing boundary.
Every RoutingPolicy field is a soft preference, not a percentage target, SLO, or control boundary.
The SDK result does not identify provider, model, attempt order, fallback, or routing reason.
503 LLM_CONFIG_UNAVAILABLE is non-retryable; 503 LLM_PROVIDERS_EXHAUSTED is marked retryable.
No fixed traffic split, identical outputs after pool change, or immediate cancellation is promised.

Summaries were generated by AI. Generative AI is experimental.

Design the application around a stable task and terminal-result contract, not around an assumed provider or model. The routes and number of provider calls can vary as the request, effective pool, routing evidence, cache state, and operating conditions change.

Know what can change

Keep application logic on the left, and test the items on the right as variables:

Treat as an application contract	Expect to vary at runtime
Task instruction and typed response schema	Provider endpoint and model used for each attempt
Agent tool interface and permissions	Output wording, reasoning path, and tool choice
Deadline and terminal success, stop, or error handling	Latency, estimated cost, retries, and attempt pattern
Tenant and deployment selected by application configuration	Relative ranking inside the effective model pool
Business acceptance and human-review rules	Which route completes each model attempt

Treat as an application contract
Task instruction and typed response schema
Expect to vary at runtime
Provider endpoint and model used for each attempt
Treat as an application contract
Agent tool interface and permissions
Expect to vary at runtime
Output wording, reasoning path, and tool choice
Treat as an application contract
Deadline and terminal success, stop, or error handling
Expect to vary at runtime
Latency, estimated cost, retries, and attempt pattern
Treat as an application contract
Tenant and deployment selected by application configuration
Expect to vary at runtime
Relative ranking inside the effective model pool
Treat as an application contract
Business acceptance and human-review rules
Expect to vary at runtime
Which route completes each model attempt

Only routes enabled in the selected tenant configuration that the task runtime has applied can be used. This is the hard outer routing boundary. A routing policy cannot add a route from another tenant or deployment, and Duale AI does not add a default route. A saved disable can take effect later, and the last applied configuration can remain in use during a configuration refresh failure. Use provider credential or network revocation when a cutoff must fail closed.

Request compatibility is not a second hard boundary. Suitability checks for remaining time, context, and skills can relax rather than empty the pool; the absolute deadline does not. Media narrows the pool only when another route accepts it. If none does, the task can continue without the images. Qualify every enabled route for every reachable workload and define the safe result when required input is unavailable.

A change to the enabled pool can change the result of every new task that reaches the tenant. Once a route is enabled, any such task can select it. Treat a material pool change as a production change.

Provider capability and compatibility states what each provider type accepts and how you declare it.

Separate hard requirements from preferences

Classify a requirement before you configure routing:

Requirement	Enforce it through	Do not rely on
Approved provider, account, or model	A tenant pool that contains only approved routes, accounts, credentials, and model identities	A routing preference or skill label
Region or processing location	A qualified regional deployment, endpoint and account, network, storage, telemetry, tools, terms, and access	A tenant, token, route label, or preference
Maximum spend	Application admission, budgets, provider limits, billing controls, and stop rules	`cost_sensitivity`
Exact model or version	Standalone routes with a pinned provider identity and version guarantee	`target_accuracy`, a preset-backed route, or rolling alias
Output shape or tool contract	A strict response schema, compatible routes, tool permissions, and application validation	Benchmark quality alone
End-to-end response time	A realistic deadline, capacity, quota, network, and measured workflow performance	`speed_preference` alone
Business safety or human approval	Deterministic application controls, review gates, and operating procedures	`target_permissiveness` or `required_skills`

Requirement
Approved provider, account, or model
Enforce it through
A tenant pool that contains only approved routes, accounts, credentials, and model identities
Do not rely on
A routing preference or skill label
Requirement
Region or processing location
Enforce it through
A qualified regional deployment, endpoint and account, network, storage, telemetry, tools, terms, and access
Do not rely on
A tenant, token, route label, or preference
Requirement
Maximum spend
Enforce it through
Application admission, budgets, provider limits, billing controls, and stop rules
Do not rely on
cost_sensitivity
Requirement
Exact model or version
Enforce it through
Standalone routes with a pinned provider identity and version guarantee
Do not rely on
target_accuracy, a preset-backed route, or rolling alias
Requirement
Output shape or tool contract
Enforce it through
A strict response schema, compatible routes, tool permissions, and application validation
Do not rely on
Benchmark quality alone
Requirement
End-to-end response time
Enforce it through
A realistic deadline, capacity, quota, network, and measured workflow performance
Do not rely on
speed_preference alone
Requirement
Business safety or human approval
Enforce it through
Deterministic application controls, review gates, and operating procedures
Do not rely on
target_permissiveness or required_skills

Every RoutingPolicy field is a soft preference. A numeric value expresses relative emphasis inside the current pool. It is not a percentage target, service-level objective, route identifier, or control boundary.

Field	Public meaning
`target_accuracy`	`0` or `None` adds no capability emphasis; higher values increasingly favor the highest available capability.
`target_permissiveness`	For sensitive but allowed requests, `0` favors withholding or qualified answers, `0.5` balances, `1` favors direct answers, and `None` adds no preference. It is not a safety control.
`cost_sensitivity`	`0` or `None` adds no base-tariff preference; higher values favor cheaper routes and can trade capability for a large price reduction. Known cache savings can affect every value. It is not a budget.
`speed_preference`	`0` gives measured latency no preference, `None` or `0.5` applies the normal balance, and `1` gives it the strongest preference.
`priority_level`	`-10` to `10` adjusts the latency preference. It does not set queue priority or extend the deadline.
`required_skills`	Prefers evidence for every listed skill, then relaxes to the best available route when the full request cannot be met.
`preferred_skills`	Improves preference for listed evidence without excluding another route.

Field
target_accuracy
Public meaning
0 or None adds no capability emphasis; higher values increasingly favor the highest available capability.
Field
target_permissiveness
Public meaning
For sensitive but allowed requests, 0 favors withholding or qualified answers, 0.5 balances, 1 favors direct answers, and None adds no preference. It is not a safety control.
Field
cost_sensitivity
Public meaning
0 or None adds no base-tariff preference; higher values favor cheaper routes and can trade capability for a large price reduction. Known cache savings can affect every value. It is not a budget.
Field
speed_preference
Public meaning
0 gives measured latency no preference, None or 0.5 applies the normal balance, and 1 gives it the strongest preference.
Field
priority_level
Public meaning
-10 to 10 adjusts the latency preference. It does not set queue priority or extend the deadline.
Field
required_skills
Public meaning
Prefers evidence for every listed skill, then relaxes to the best available route when the full request cannot be met.
Field
preferred_skills
Public meaning
Improves preference for listed evidence without excluding another route.

Routing does not infer skill requirements from the task text. Set them explicitly. Skill and benchmark evidence can change with the pool and published evidence; a skill label is not a certification or an allowed-use decision.

Omit RoutingPolicy, or send the whole policy as None, to use the service default policy. RoutingPolicy() is an explicit empty policy. Explicit None is valid for the four nullable numeric fields; omit priority_level and the skill lists instead of assigning None. The service default does not add a route, and its numeric values are not part of this public contract.

What varies between two identical requests

Plan for these changes even when the application sends the same task twice:

The SDK result does not identify the provider, model, attempt order, fallback, pool revision, or routing reason. Documented analytics can show provider and model later, but application logic must not depend on them.
The same policy can select a different eligible route when context or operating conditions change.
A continuation preserves the root request’s exact policy, including an explicit empty policy or no policy. When no policy was sent, each turn can resolve the service default again. A continuation does not freeze the model pool.
cost_sensitivity does not cap retries, fallback, tools, or other work performed for the task.
speed_preference and priority_level only affect latency preference. They do not replace or extend the task deadline.
The absolute deadline bounds obtaining a complete model response. Durable persistence and delivery can finish later, so it is not a strict end-to-end return-time limit. A child cannot extend its parent’s accepted deadline; a shorter child deadline also constrains later descendants.
Streaming content can reset during a replacement attempt. Clear the rendered preview on every reset. Stream exhaustion is not success: always call model() to obtain or raise the terminal outcome.
Stopping a task does not undo a provider or tool call that has already started, and it does not recover consumed cost.

Keep retries, display state, and business side effects independent of route identity and streamed preview.

Handle pool failures

Handle each public terminal problem through the owner named below:

Terminal problem	Meaning	Application action
`503 LLM_CONFIG_UNAVAILABLE`	The tenant has no usable model configuration. It is non-retryable and needs owner action.	Stop automatic retry and send the item to the configuration owner.
`503 LLM_PROVIDERS_EXHAUSTED`	Route execution ended without a result. It is marked retryable.	Apply the bounded retry and idempotency policy for the business item.

Terminal problem
503 LLM_CONFIG_UNAVAILABLE
Meaning
The tenant has no usable model configuration. It is non-retryable and needs owner action.
Application action
Stop automatic retry and send the item to the configuration owner.
Terminal problem
503 LLM_PROVIDERS_EXHAUSTED
Meaning
Route execution ended without a result. It is marked retryable.
Application action
Apply the bounded retry and idempotency policy for the business item.

LLM_PROVIDERS_EXHAUSTED does not prove that every configured route ran. A blocking error, the deadline, or available retry capacity can stop later attempts.

What the routing contract does not promise

The routing contract does not promise:

a fixed traffic split between old and new models;
sequential or parallel execution across route attempts;
an attempt against every enabled route;
identical outputs after a model-pool change;
immediate cancellation of in-flight provider work after a configuration change.

If a workload needs one exact model version, use only standalone routes for that approved provider model and version. Detaching current fields does not make a preset-backed route a complete snapshot because other preset-owned fields can change. Keep several approved endpoints for the same version only when the account, region, credential, and endpoint boundaries permit them. Define what the application does when no approved route is available.

A configured provider model_id can refer to a rolling alias, gateway, or hosted deployment. An exact-version boundary also needs a pinned provider identity and a provider or operator version guarantee.

Protect the business contract

An unchanged response schema proves only that the result has the expected structure. It does not prove that the result is correct or acceptable for the business process. Add domain checks, approval rules, outcome monitoring, and a safe failure path where a wrong answer can cause material harm.