--- title: "Routing contract: hard boundaries and soft preferences" description: "The routing contract separates hard application boundaries from soft preferences that vary as pool, context, and operating conditions change." lang: en status: public-preview lastUpdated: 2026-08-31 url: https://dev.duale.ai/en/docs/model-routing/contract-and-limits --- ## AI-generated summary Design applications around the stable task and terminal-result contract, separating hard tenant boundaries from soft routing preferences that vary at runtime. - Only routes enabled in the selected tenant configuration form the hard outer routing boundary. - Every RoutingPolicy field is a soft preference, not a percentage target, SLO, or control boundary. - The SDK result does not identify provider, model, attempt order, fallback, or routing reason. - 503 LLM_CONFIG_UNAVAILABLE is non-retryable; 503 LLM_PROVIDERS_EXHAUSTED is marked retryable. - No fixed traffic split, identical outputs after pool change, or immediate cancellation is promised. Summaries were generated by AI. Generative AI is experimental. --- Design the application around a stable task and terminal-result contract, not around an assumed provider or model. The routes and number of provider calls can vary as the request, effective pool, routing evidence, cache state, and operating conditions change. ## Know what can change Keep application logic on the left, and test the items on the right as variables: | Treat as an application contract | Expect to vary at runtime | | ----------------------------------------------------------- | ----------------------------------------------------- | | Task instruction and typed response schema | Provider endpoint and model used for each attempt | | Agent tool interface and permissions | Output wording, reasoning path, and tool choice | | Deadline and terminal success, stop, or error handling | Latency, estimated cost, retries, and attempt pattern | | Tenant and deployment selected by application configuration | Relative ranking inside the effective model pool | | Business acceptance and human-review rules | Which route completes each model attempt | Only routes enabled in the selected tenant configuration that the task runtime has applied can be used. This is the hard outer routing boundary. A routing policy cannot add a route from another tenant or deployment, and Duale AI does not add a default route. A saved disable can take effect later, and the last applied configuration can remain in use during a configuration refresh failure. Use provider credential or network revocation when a cutoff must fail closed. Request compatibility is not a second hard boundary. Suitability checks for remaining time, context, and skills can relax rather than empty the pool; the absolute deadline does not. Media narrows the pool only when another route accepts it. If none does, the task can continue without the images. Qualify every enabled route for every reachable workload and define the safe result when required input is unavailable. > [!WARNING] > A change to the enabled pool can change the result of every new task that reaches the tenant. Once a route is enabled, > any such task can select it. Treat a material pool change as a production change. [Provider capability and compatibility](https://dev.duale.ai/en/docs/model-routing/provider-capabilities.md) states what each provider type accepts and how you declare it. ## Separate hard requirements from preferences Classify a requirement before you configure routing: | Requirement | Enforce it through | Do not rely on | | ------------------------------------ | ------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------- | | Approved provider, account, or model | A tenant pool that contains only approved routes, accounts, credentials, and model identities | A routing preference or skill label | | Region or processing location | A qualified regional deployment, endpoint and account, network, storage, telemetry, tools, terms, and access | A tenant, token, route label, or preference | | Maximum spend | Application admission, budgets, provider limits, billing controls, and stop rules | `cost_sensitivity` | | Exact model or version | Standalone routes with a pinned provider identity and version guarantee | `target_accuracy`, a preset-backed route, or rolling alias | | Output shape or tool contract | A strict response schema, compatible routes, tool permissions, and application validation | Benchmark quality alone | | End-to-end response time | A realistic deadline, capacity, quota, network, and measured workflow performance | `speed_preference` alone | | Business safety or human approval | Deterministic application controls, review gates, and operating procedures | `target_permissiveness` or `required_skills` | Every `RoutingPolicy` field is a soft preference. A numeric value expresses relative emphasis inside the current pool. It is not a percentage target, service-level objective, route identifier, or control boundary. | Field | Public meaning | | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `target_accuracy` | `0` or `None` adds no capability emphasis; higher values increasingly favor the highest available capability. | | `target_permissiveness` | For sensitive but allowed requests, `0` favors withholding or qualified answers, `0.5` balances, `1` favors direct answers, and `None` adds no preference. It is not a safety control. | | `cost_sensitivity` | `0` or `None` adds no base-tariff preference; higher values favor cheaper routes and can trade capability for a large price reduction. Known cache savings can affect every value. It is not a budget. | | `speed_preference` | `0` gives measured latency no preference, `None` or `0.5` applies the normal balance, and `1` gives it the strongest preference. | | `priority_level` | `-10` to `10` adjusts the latency preference. It does not set queue priority or extend the deadline. | | `required_skills` | Prefers evidence for every listed skill, then relaxes to the best available route when the full request cannot be met. | | `preferred_skills` | Improves preference for listed evidence without excluding another route. | Routing does not infer skill requirements from the task text. Set them explicitly. Skill and benchmark evidence can change with the pool and published evidence; a skill label is not a certification or an allowed-use decision. Omit `RoutingPolicy`, or send the whole policy as `None`, to use the service default policy. `RoutingPolicy()` is an explicit empty policy. Explicit `None` is valid for the four nullable numeric fields; omit `priority_level` and the skill lists instead of assigning `None`. The service default does not add a route, and its numeric values are not part of this public contract. ## What varies between two identical requests Plan for these changes even when the application sends the same task twice: - The SDK result does not identify the provider, model, attempt order, fallback, pool revision, or routing reason. Documented analytics can show provider and model later, but application logic must not depend on them. - The same policy can select a different eligible route when context or operating conditions change. - A continuation preserves the root request's exact policy, including an explicit empty policy or no policy. When no policy was sent, each turn can resolve the service default again. A continuation does not freeze the model pool. - `cost_sensitivity` does not cap retries, fallback, tools, or other work performed for the task. - `speed_preference` and `priority_level` only affect latency preference. They do not replace or extend the task deadline. - The absolute deadline bounds obtaining a complete model response. Durable persistence and delivery can finish later, so it is not a strict end-to-end return-time limit. A child cannot extend its parent's accepted deadline; a shorter child deadline also constrains later descendants. - Streaming content can reset during a replacement attempt. Clear the rendered preview on every reset. Stream exhaustion is not success: always call `model()` to obtain or raise the terminal outcome. - Stopping a task does not undo a provider or tool call that has already started, and it does not recover consumed cost. Keep retries, display state, and business side effects independent of route identity and streamed preview. ## Handle pool failures Handle each public terminal problem through the owner named below: | Terminal problem | Meaning | Application action | | ----------------------------- | ----------------------------------------------------------------------------------------- | --------------------------------------------------------------------- | | `503 LLM_CONFIG_UNAVAILABLE` | The tenant has no usable model configuration. It is non-retryable and needs owner action. | Stop automatic retry and send the item to the configuration owner. | | `503 LLM_PROVIDERS_EXHAUSTED` | Route execution ended without a result. It is marked retryable. | Apply the bounded retry and idempotency policy for the business item. | `LLM_PROVIDERS_EXHAUSTED` does not prove that every configured route ran. A blocking error, the deadline, or available retry capacity can stop later attempts. ## What the routing contract does not promise The routing contract does not promise: - a fixed traffic split between old and new models; - sequential or parallel execution across route attempts; - an attempt against every enabled route; - identical outputs after a model-pool change; - immediate cancellation of in-flight provider work after a configuration change. If a workload needs one exact model version, use only standalone routes for that approved provider model and version. Detaching current fields does not make a preset-backed route a complete snapshot because other preset-owned fields can change. Keep several approved endpoints for the same version only when the account, region, credential, and endpoint boundaries permit them. Define what the application does when no approved route is available. A configured provider `model_id` can refer to a rolling alias, gateway, or hosted deployment. An exact-version boundary also needs a pinned provider identity and a provider or operator version guarantee. ## Protect the business contract An unchanged response schema proves only that the result has the expected structure. It does not prove that the result is correct or acceptable for the business process. Add domain checks, approval rules, outcome monitoring, and a safe failure path where a wrong answer can cause material harm. ## Related content - [Model routing architecture patterns](https://dev.duale.ai/en/docs/model-routing/architecture-patterns.md) - [Integrate model routing in an application](https://dev.duale.ai/en/docs/model-routing/application-integration.md) - [Plan the deployment and model portfolio](https://dev.duale.ai/en/docs/model-routing/plan-deployment.md) - [SDK task lifecycle, routing, and streaming](https://dev.duale.ai/en/docs/sdk/concepts.md) - [Understand model routing and the stable application contract](https://dev.duale.ai/en/docs/model-routing.md) - [Ownership and approvals for routing decisions](https://dev.duale.ai/en/docs/model-routing/ownership-and-approvals.md) --- ## Sitemap See the full [Markdown sitemap](https://dev.duale.ai/sitemap.md) for all pages.