Model routing architecture patterns

Model routing architecture patterns show how to structure model pools based on hard boundaries and failure outcomes rather than preferred provider names.

Model routing architecture patterns show how to structure model pools by hard boundary and failure outcome, covering customer separation, strict isolation, fallback, and deployment splits.

Select patterns from hard boundary and failure outcome, not from a preferred provider name.
Model pools prevent completion outside the pool but not ingress, identity, or telemetry.
Strict workloads use a dedicated tenant with pinned provider identity and a version guarantee.
Cloud and on-premises deployments are separate environments unless the deployment contract states otherwise.
Use a separate tenant for experimental routes so production tasks cannot select them.

Summaries were generated by AI. Generative AI is experimental.

Select a pattern from the hard boundary and failure outcome, not from a preferred provider name. These examples show how to structure model pools. They do not confirm product availability, provider suitability, or data residency.

A tenant and model pool are logical controls. Do not claim local processing, regional residency, or no cross-border access until a dated evidence pack covers the complete live data path. It must also cover every external recipient, every human-access path, and the failure case. Define the claim’s scope and exclusions.

Separate customers or regions

Create a separate tenant when a customer needs a tenant-scoped provider configuration, cost view, conversation, Library, or logical access boundary. Provision distinct provider accounts and credentials when the customer boundary requires them. Use a separate regional landing zone when the full platform processing path must remain in that region.

The following map is an illustrative enterprise design:

Boundary	Example approved pool	Application configuration
Region A	Routes and independent endpoints approved for processing in Region A	Region A endpoint and separately provisioned token
Region B	Routes and independent endpoints approved for processing in Region B	Region B endpoint and separately provisioned token
Strict customer boundary	Routes, provider accounts, and credentials approved for one customer	Customer tenant identity in the approved deployment

Boundary
Region A
Example approved pool
Routes and independent endpoints approved for processing in Region A
Application configuration
Region A endpoint and separately provisioned token
Boundary
Region B
Example approved pool
Routes and independent endpoints approved for processing in Region B
Application configuration
Region B endpoint and separately provisioned token
Boundary
Strict customer boundary
Example approved pool
Routes, provider accounts, and credentials approved for one customer
Application configuration
Customer tenant identity in the approved deployment

The application can keep one source integration only when Duale AI has qualified the SDK and platform version pair for every function it uses. Each deployment still needs its own endpoint, token, identifiers, grants, cache, and telemetry configuration. The token provides identity inside its deployment; it is not locality evidence. The deployment handoff must identify the supported SDK and platform pair.

Separate model pools prevent model completion against a route outside the pool. They do not control platform ingress, identity, internal indexing or supporting models, Libraries and media paths, tools and their external targets, telemetry, queues, caches, backups, billing, support, or human access. Review every path before a local-processing or no-cross-border claim.

Isolate a strict workload

Use a dedicated tenant that contains only standalone routes for the approved provider model and version when a contract, evaluation, or policy requires it. Require a pinned provider identity and a provider or operator version guarantee: a configured model_id can name a rolling alias, gateway, or hosted deployment. A preset-backed route is not a complete snapshot, even when its current fields are detached. The application still uses the normal SDK contract.

You can include several approved endpoints for the same pinned identity when the account, region, and credential boundaries permit them. This pattern removes fallback to another model and can also remove endpoint fallback. Define a manual path, delayed processing, or safe failure when no approved route is available. Do not add a general fallback that violates the strict requirement.

Use independent multi-provider fallback

Place two or more functionally compatible routes in one approved pool when the workload can use several providers and resilience is more important than fixed route identity. Verify that the routes do not share critical failure domains.

Test output compatibility and the complete failure path. Attempts can add provider calls, quota use, cost, and delay. Routing contract and limits owns the attempt-order and traffic-split limits.

Separate cloud and on-premises work

Treat cloud and on-premises deployments as separate application environments unless the supported deployment contract states what they share. Hosting location alone does not prove the full processing boundary.

Use separate deployments when cloud and customer-hosted workloads have different data, network, provider, or change-control boundaries. Customer-hosted deployment is available to eligible customers. Duale AI supplies the supported release, deployment specification, upgrade guidance, and product support. The customer operates its infrastructure, network, storage, and infrastructure access. The deployment agreement defines the exact split.

Keep the same application contract only where Duale AI has qualified the SDK, platform, API topology, identity functions, and route capabilities. Give each deployment its own application configuration and SDK client. Use separate processes when the deployment, token-derived tenant identity, telemetry endpoint, or telemetry token differs.

A private endpoint does not create DNS, TLS, proxy, firewall, VPN, or private-link connectivity. Provide and test that path. An on-premises platform can still send data to an external model or tool service, so hosting location alone does not prove local processing.

Verify in each deployment:

model, tool, task API, /libraries, and presigned-storage connectivity;
network, identity, tenant, agent, grant, and credential controls;
route support for every application function;
document, cache, telemetry, backup, monitoring, and support data paths; and
capacity, recovery, compatibility, and upgrade procedures.

Treat a move as new provisioning because the public interface has no general state export or import:

create the target identities, credentials, grants, and configuration;
re-establish the approved document, tool, telemetry, backup, and recovery paths; and
submit a new root task with only the context you are authorized to carry and a durable business idempotency key.

A source task_id cannot continue in the target. Verify the target independently before you stop the source environment.

Protect stable and experimental work

Use a separate tenant for experimental routes so production tasks cannot select them. Use separate credentials or a separate deployment when the evaluation also needs a provider-account or physical boundary. Control model changes gives the complete evaluation, enablement, observation, and rollback sequence.

Separate customers or regions Isolate a strict workload Use independent multi-provider fallback Separate cloud and on-premises work Protect stable and experimental work

Last updatedAugust 31, 2026