--- title: "Operate model routing and manage risk" description: "Operating model routing as a business-process service requires monitoring health across business, task, and control boundaries." lang: en status: public-preview lastUpdated: 2026-08-31 url: https://dev.duale.ai/en/docs/model-routing/operations-and-risk --- ## AI-generated summary Operate model routing as a business service by monitoring health, preparing controls, testing resilience, responding to incidents, and reviewing the approved pool. - Defines three health views: business process, task and route, and control boundary. - Requires stop rules, tested failure paths, and named owners before production starts. - Lists incident types with first safe actions and follow-up resolution steps. - Distinguishes business records, route analytics, and provider invoices as non-replacing evidence sources. - Covers resilience test outcomes including fallback failure, quota exhaustion, and boundary violations. Summaries were generated by AI. Generative AI is experimental. --- Operate routing as a service that supports business processes, not only as a list of reachable model endpoints. A healthy endpoint can still produce an unacceptable business result. A successful business task can also hide a failing fallback or a boundary violation. ## Monitor three kinds of health Define the evidence for each decision, then confirm which signals your deployment exposes. The task result does not contain route, attempt, fallback, or cost data. Use only documented route analytics; provider invoices remain the source for provider-billed spend. | Health view | Questions | Evidence and limit | | ---------------- | ------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | | Business process | Are results acceptable, timely, safe to use, and correctly reviewed? | Customer task records, acceptance, manual-review, domain-rule and incident data are authoritative for the workflow. | | Task and route | Do requests return an acceptable terminal result, and do later eligible routes work when needed? | Error codes, end-to-end latency, controlled tests, and documented analytics for each provider and model. The result has no attempt sequence. | | Control boundary | Does desired configuration still match approvals and data obligations? | Secret-redacted config evidence, customer change record, access review, locality evidence, and exceptions. A hash is not runtime proof. | Compare configured-price data in route analytics, where available, with the provider invoice and any separate Duale AI billing record. Routing preferences do not create a hard budget, and retries or supporting work can change total cost. Analytics can be delayed or cached. Record the query window and generation time, allow for data arrival, and never treat a missing row as proof that no route ran or no incident occurred. Route analytics are aggregate evidence for the requested configuration ID and hash and accepted outputs. It can exclude failed attempts and does not provide per-task, cohort, fallback, or how widely a configuration change has taken effect. Use the business record to decide whether an outcome is acceptable, route analytics to explain aggregate patterns, and provider invoices to confirm billed spend. None replaces the other two. ## Prepare operational controls Production starts only after the workload has a stop rule, a tested failure path, and named owners for every external dependency. Record and test these controls before the first production task: - verify endpoint, network, identity, credential, quota, and rate-limit behavior; - set application deadlines and define what happens when no approved route succeeds; - confirm that later eligible routes support the same essential response, tool, context, and input contracts; - rotate credentials and test the emergency cutoff without exposing secrets in logs; - define capacity and quota owners for each deployment and provider account; - restrict secret-bearing configuration reads and review material changes independently; - treat telemetry as content-bearing. When platform telemetry is enabled, it can retain text-only model inputs and outputs, tool arguments, and tool results. Confirm collection, location, access, retention, deletion, and support use; - record provider, regional, retention, backup, tool, and support data paths; and - schedule failure tests and pool reviews. The control set is ready when each item has an owner, an evidence date, and a scheduled next test. ## Test resilience from the application boundary Use an evaluation tenant and an approved controlled fault. Submit representative work, require `await response.cache_hit()` to be `False`, observe the terminal application outcome, and retain evidence appropriate to the claim. Documented route analytics can support provider and model evidence, but they do not prove the provider account or processing location. The normal result has no route selector or attempt sequence. If the deployment provides no supported attempt evidence, the test proves application recovery only; it does not prove which fallback path ran. Exercise these outcomes, not only endpoint probes: - the first-ranked route fails, a retry adds cost or delay, and a later route succeeds; - a shared endpoint, account, or regional quota affects several routes; - a blocking error, deadline, or retry limit prevents another attempt; - a later route cannot satisfy an input, tool, or response contract; - streaming output resets before a replacement or terminal error; - a route is disabled while work is active; - the application receives a valid schema with an unacceptable business value; - a regional dependency fails without sending work outside the approved boundary; and - network evidence, backup restore, and support access match the approved data flow. Confirm the terminal application behavior, customer communication, evidence record, and recovery action for each case. For an account or locality boundary, require provider-side account and processing-location evidence in addition to captured network evidence. Provider analytics, model analytics, and DNS data are insufficient on their own. Do not infer a service-level guarantee from one successful fallback test. ## Respond by incident type An incident needs two decisions: the first action that limits harm, then the resolution that restores the approved boundary. Use the row for the observed incident instead of treating the first action as the fix. | Incident | First safe action | Follow-up | | ---------------------------------------------- | ----------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | | Model-provider outage | Keep the workload inside its approved pool and invoke the application failure path | Follow the service incident process, confirm recovery evidence, and review shared failure domains | | Customer account, quota, or rate-limit failure | Stop or admit work within the available quota; do not add an unapproved route | Restore the customer-owned quota or account and test bounded retry and later-route behavior | | Business-output regression | Stop the affected application cohort and disable the route when the risk threshold is met | Switch new roots to the stable tenant, evaluate affected records, and reopen only after acceptance | | Unexpected route, alias, or preset drift | Stop the affected cohort and capture a secret-redacted effective snapshot | Assess the affected work, provider identity, model version, processing location, notice requirements, and independent evidence for reopening | | Unexpected cost | Apply the business stop rule or workload limit | Compare application records, documented analytics, retries, tools, configured prices, and provider invoices | | Credential exposure or provider compromise | Revoke the credential or block the network path | Replace it, inspect use, assess affected data and work, verify desired configuration, and record the incident | | Suspected data-boundary violation | Stop the affected transfer or workload under the incident procedure | Preserve evidence, involve privacy and security owners, assess notices, and verify every data path | | Configuration error | Stop the affected cohort or use the approved emergency cutoff | Reapply desired state, assess propagation limits and affected tasks, then review access and change controls | Do not add a fallback route during an incident if that route has not passed the required provider, data, functional, and operational approvals. Use the control that matches the stop scope. Application admission stops new work. Disabling a route affects later selection after configuration refresh and is not a fail-closed revocation control. Provider credential or network revocation blocks later provider access. Only the submitting agent can request a stop for active work, and acceptance of that request is not terminal proof; verify the stopped result. ## Review the service A service review decides whether the approved pool can stay in use. Run it at a scheduled interval and after a material incident or model change, then review: - workload acceptance, manual review, errors, latency, quota, use, and cost; - provider contracts, regions, subprocessors, deprecations, and prices; - credentials, access, change records, exceptions, and expiry dates; - capacity, recovery tests, and independent failure domains; - application deadlines, failure paths, tools, and business controls. Record the review date, evidence window, findings, owners, and next review. If evidence is missing, record the gap instead of treating it as success. [Ownership and approvals](https://dev.duale.ai/en/docs/model-routing/ownership-and-approvals.md) defines the record. ## Related content - [Understand model routing and the stable application contract](https://dev.duale.ai/en/docs/model-routing.md) - [Model routing architecture patterns](https://dev.duale.ai/en/docs/model-routing/architecture-patterns.md) - [Control model changes across the production lifecycle](https://dev.duale.ai/en/docs/model-routing/control-model-changes.md) - [Integrate model routing in an application](https://dev.duale.ai/en/docs/model-routing/application-integration.md) - [Ownership and approvals for routing decisions](https://dev.duale.ai/en/docs/model-routing/ownership-and-approvals.md) - [Routing contract: hard boundaries and soft preferences](https://dev.duale.ai/en/docs/model-routing/contract-and-limits.md) --- ## Sitemap See the full [Markdown sitemap](https://dev.duale.ai/sitemap.md) for all pages.