Operate model routing and manage risk

Operate routing as a service that supports business processes, not only as a list of reachable model endpoints. A healthy endpoint can still produce an unacceptable business result. A successful business task can also hide a failing fallback or a boundary violation.

Monitor three kinds of health

Define the evidence for each decision, then confirm which signals your deployment exposes. The task result does not contain route, attempt, fallback, or cost data. Use only documented route analytics; provider invoices remain the source for provider-billed spend.

Compare configured-price data in route analytics, where available, with the provider invoice and any separate Duale AI billing record. Routing preferences do not create a hard budget, and retries or supporting work can change total cost.

Analytics can be delayed or cached. Record the query window and generation time, allow for data arrival, and never treat a missing row as proof that no route ran or no incident occurred. Route analytics are aggregate evidence for the requested configuration ID and hash and accepted outputs. It can exclude failed attempts and does not provide per-task, cohort, fallback, or how widely a configuration change has taken effect.

Use the business record to decide whether an outcome is acceptable, route analytics to explain aggregate patterns, and provider invoices to confirm billed spend. None replaces the other two.

Prepare operational controls

Production starts only after the workload has a stop rule, a tested failure path, and named owners for every external dependency. Record and test these controls before the first production task:

• verify endpoint, network, identity, credential, quota, and rate-limit behavior;
• set application deadlines and define what happens when no approved route succeeds;
• confirm that later eligible routes support the same essential response, tool, context, and input contracts;
• rotate credentials and test the emergency cutoff without exposing secrets in logs;
• define capacity and quota owners for each deployment and provider account;
• restrict secret-bearing configuration reads and review material changes independently;
• treat telemetry as content-bearing. When platform telemetry is enabled, it can retain text-only model inputs and outputs, tool arguments, and tool results. Confirm collection, location, access, retention, deletion, and support use;
• record provider, regional, retention, backup, tool, and support data paths; and
• schedule failure tests and pool reviews.

The control set is ready when each item has an owner, an evidence date, and a scheduled next test.

Test resilience from the application boundary

Use an evaluation tenant and an approved controlled fault. Submit representative work, require await response.cache_hit() to be False, observe the terminal application outcome, and retain evidence appropriate to the claim. Documented route analytics can support provider and model evidence, but they do not prove the provider account or processing location. The normal result has no route selector or attempt sequence. If the deployment provides no supported attempt evidence, the test proves application recovery only; it does not prove which fallback path ran.

Exercise these outcomes, not only endpoint probes:

• the first-ranked route fails, a retry adds cost or delay, and a later route succeeds;
• a shared endpoint, account, or regional quota affects several routes;
• a blocking error, deadline, or retry limit prevents another attempt;
• a later route cannot satisfy an input, tool, or response contract;
• streaming output resets before a replacement or terminal error;
• a route is disabled while work is active;
• the application receives a valid schema with an unacceptable business value;
• a regional dependency fails without sending work outside the approved boundary; and
• network evidence, backup restore, and support access match the approved data flow.

Confirm the terminal application behavior, customer communication, evidence record, and recovery action for each case. For an account or locality boundary, require provider-side account and processing-location evidence in addition to captured network evidence. Provider analytics, model analytics, and DNS data are insufficient on their own. Do not infer a service-level guarantee from one successful fallback test.

Respond by incident type

An incident needs two decisions: the first action that limits harm, then the resolution that restores the approved boundary. Use the row for the observed incident instead of treating the first action as the fix.

Do not add a fallback route during an incident if that route has not passed the required provider, data, functional, and operational approvals.

Use the control that matches the stop scope. Application admission stops new work. Disabling a route affects later selection after configuration refresh and is not a fail-closed revocation control. Provider credential or network revocation blocks later provider access. Only the submitting agent can request a stop for active work, and acceptance of that request is not terminal proof; verify the stopped result.

Review the service

A service review decides whether the approved pool can stay in use. Run it at a scheduled interval and after a material incident or model change, then review:

• workload acceptance, manual review, errors, latency, quota, use, and cost;
• provider contracts, regions, subprocessors, deprecations, and prices;
• credentials, access, change records, exceptions, and expiry dates;
• capacity, recovery tests, and independent failure domains;
• application deadlines, failure paths, tools, and business controls.

Record the review date, evidence window, findings, owners, and next review. If evidence is missing, record the gap instead of treating it as success. Ownership and approvals defines the record.