Architecture principle: Models are replaceable intelligence services. Organizational workflows, knowledge, controls, and evidence are durable assets.
The lock-in problem
AI applications are often built directly around a provider API. Prompts, tool schemas, retrieval behavior, safety controls, and observability become coupled to that provider. A model change then becomes an application rewrite.
This creates commercial, operational, and risk concentration. It also prevents the organization from matching the best intelligence to each workload.
Separate the operating layers
A multi-model AI architecture separates four concerns:
- Experience: the interface through which people and systems request work.
- Orchestration: the agents, skills, workflows, approvals, and state that define how work is performed.
- Intelligence: the approved models selected for each task.
- Control: the policies, security, evaluation, cost, and evidence applied across every route.
Channels and applications
↓
Chakali agents and workflows
↓
Policy-aware model router
↙ ↓ ↘
Private Regional SaaS
models providers models
The workflow calls an approved intelligence capability rather than hard-coding a provider.
What should drive model selection
Capability
Reasoning, tool use, language, modality, context length, and domain performance must match the task.
Risk and data sensitivity
Restricted data may require an on-premises model. Public research may be suitable for an external provider. Policy should make the choice before execution.
Cost and latency
Not every step needs the most capable model. Routing can use smaller models for classification and extraction while reserving advanced models for complex synthesis.
Availability and resilience
Fallback routes can reduce dependency on a single provider. A fallback is only safe when it meets the same policy and quality requirements.
Residency and contractual constraints
The route should account for permitted regions, provider data handling, retention, and organizational agreements.
Policy-aware routing
Model routing should produce a decision record containing:
- Requested capability
- Data classification
- Candidate models
- Policy conditions
- Selected route
- Reason for selection
- Cost and latency expectation
- Fallback route
- Evaluation profile
This turns model selection from invisible application logic into a governable platform decision.
Deployment models
SaaS
Suitable when the organization wants rapid adoption and centrally managed operations. The architecture must still provide tenant isolation, identity integration, encryption, retention controls, and provider transparency.
On-premises
Suitable for restricted data, isolated networks, regulated environments, or full infrastructure control. Models, vector stores, tools, and orchestration can remain inside the organization's boundary.
Hybrid
A hybrid model uses policy to keep sensitive workloads private while allowing approved external intelligence for suitable tasks. The control plane provides one operating model across both.
Evaluation is part of routing
A model should not enter production because it performed well in a demonstration. Define task-specific evaluation sets, quality thresholds, safety tests, cost limits, and regression checks.
When a provider releases a new version, the organization can evaluate it against the same workload before changing the route. The workflow and business process remain stable.
Architecture review checklist
- Are workflows independent of provider-specific interfaces?
- Is model routing based on policy and task requirements?
- Can sensitive data be prevented from leaving an approved boundary?
- Are fallbacks evaluated and governed?
- Are model versions visible in execution records?
- Can a model be disabled without editing every workflow?
- Are cost, latency, quality, and risk measured together?
- Can the same operating model run in SaaS and on-premises environments?
Chakali provides the harness in which models can compete on value while the organization retains ownership of its processes, controls, and evidence.