AI Insights
Multi-tenant AI SaaS architecture is the set of design decisions that let one deployment serve many customers while guaranteeing that no customer can ever see another customer’s data. In a conventional SaaS product those decisions live in the database. In an AI product they also live in the vector store, the prompt, the cache, the logs, the evaluation set and the fine-tuning corpus — six new places where a tenant boundary can quietly fail. Getting this right is the difference between a product you can sell to an enterprise and one you can only demo.
This is a builder’s guide, not a vendor pitch. It covers the isolation models worth considering, the AI-specific leak paths that the standard SaaS playbook does not cover, what per-tenant cost control actually requires, and the Australian obligations that apply once real customer data is involved.
Multi-tenancy means a single running system serves multiple customers — tenants — from shared infrastructure. Tenant isolation is the mechanism that keeps their data, compute and identity separate. AWS’s SaaS Lens frames isolation as a foundational element of SaaS: every system delivering a multi-tenant solution should take deliberate measures to ensure tenant resources are isolated, and that isolation is distinct from authentication and authorisation. Authenticating a user tells you who they are. Isolation is what stops a bug in your retrieval code from returning someone else’s document to them anyway.
In an AI product, the tenant boundary has to be enforced in more layers than most teams expect:
Miss any one of them and you have a cross-tenant disclosure path. The OWASP Top 10 for LLM Applications 2025 moved Sensitive Information Disclosure from sixth place to second, and calls out the multi-tenant case explicitly: in a shared environment, one customer can attempt to induce the model to disclose another customer’s information, and insufficient access controls on vector stores can expose data across tenant boundaries.
Three reasons, and they compound.
First, retrieval is a second database that nobody treats like one. Teams apply careful access control to Postgres and then write a vector search that filters on a metadata field passed in from the application layer. If that filter is ever omitted — a new endpoint, a background job, an agent tool call — the query silently returns the nearest neighbours across every tenant in the collection. There is no equivalent of a foreign key constraint to catch it. We cover the retrieval layer itself in our RAG architecture guide.
Second, the model is a confused deputy. The model has no concept of tenancy. It will faithfully summarise whatever you put in its context window, and it cannot tell that one of the six retrieved chunks belongs to a different customer. Every isolation guarantee has to be enforced before the model sees the data, never by instructing the model to behave. Prompt-level instructions are not a security control — a point we go into in our AI app security guide.
Third, cost is now tenant-attributable and highly variable. In classic SaaS, one heavy tenant costs you some CPU. In AI SaaS, one heavy tenant can cost you real money per request, and a single customer running long-context workloads can consume more inference spend than your entire flat-rate revenue from them. Isolation and metering are the same problem viewed from two angles.
Honest cost benchmarks, the hidden costs vendors don’t quote, and a 10-line scoping worksheet.
Get the free Australian AI MVP Cost Guide 2026 — we’ll email it straight to you.
AWS’s SaaS Lens describes the two poles and the pragmatic middle. Full stack (silo) isolation gives each tenant a dedicated environment — sometimes an account per tenant — which eliminates noisy-neighbour effects and makes the isolation story trivially easy to explain to a security reviewer. It is also the most expensive model to run and the slowest to deploy changes to. Pool isolation shares compute and storage across all tenants, which is cheaper and simpler to operate but puts the entire burden of separation on your application logic.
The useful insight from the SaaS Lens is that this is not a single decision. Targeted isolation means each microservice and each resource it touches can use a different model, and tier-based isolation means you can package different flavours of isolation as different commercial tiers.
For an AI product, the mix that tends to work is:
Sell the silo tier. Enterprise buyers will pay for dedicated infrastructure, and offering it as a paid tier turns your hardest architectural problem into a revenue line rather than a cost centre.
Every serious vector database now documents a multi-tenancy pattern, and the guidance is remarkably consistent. Qdrant’s multitenancy documentation recommends a single collection with payload-based partitioning as the default for services serving many independent users. The operational rules that matter:
tenant_id as a keyword payload with a pre-created index, so filtered queries do not degrade into full scans.payload_m in the HNSW config to a non-zero value and m to zero, so the index is built per tenant group rather than globally — better performance for many-tenant workloads with no query-level changes.On the relational side, PostgreSQL row-level security has been available since version 9.5 and remains the strongest available backstop. As Crunchy Data’s write-up on RLS for tenants sets out, a policy such as CREATE POLICY tenant_isolation ON tenant_data FOR ALL USING (tenant_id = current_tenant_id()) pushes the filter into the engine so a forgotten WHERE clause fails closed. Two footguns are worth memorising: table owners and superusers bypass RLS unless you also run FORCE ROW LEVEL SECURITY and connect as a non-owner role; and you must set the tenant context with SET LOCAL inside a transaction, because a plain SET will leak one tenant’s context into the next request through a connection pooler.
These are the failures that do not show up in a database review, because no database is involved.
Caches. Semantic caching is one of the most effective inference cost levers available — and the single most dangerous shared resource in a multi-tenant AI product. If the cache key is a hash of the prompt, two tenants asking the same question will share an answer that was generated from one tenant’s private documents. The tenant ID must be part of the cache key, always, with no exceptions for “generic” queries. The same discipline applies to embedding caches and provider-side prompt caches. Our guide to cutting AI API costs covers the cost side of this trade-off.
Traces and logs. AI observability tooling captures full prompt and completion text by default, because that is what makes it useful for debugging. That means your trace backend now holds every tenant’s data in one index, usually with broader internal access than production. Decide explicitly: redact at the SDK boundary, or treat the trace store as production data with the same isolation and retention rules. Do not let it be an accident. We cover what to capture and how in our AI observability guide.
Agent tools. If your product uses tool-calling agents, every tool is a new path to your data that bypasses your API’s authorisation middleware. The tenant context must be bound to the execution session and injected server-side into every tool invocation — never passed as a model-supplied argument, because a model-supplied tenant ID is an argument an attacker can influence through prompt injection. Our overview of AI agent design patterns goes deeper on where those boundaries belong.
Model artefacts. Fine-tuning on pooled tenant data bakes one customer’s information into weights that serve everyone, and it cannot be undone by a delete request. If you fine-tune, do it per tenant or on data you own outright. Few-shot examples embedded in a shared system prompt are the same problem in miniature.
Cost control in multi-tenant AI is an architectural requirement, not a finance exercise. Three things to build in from the start:
If you are building for Australian customers, three things bear directly on this architecture.
The Privacy Act and the Australian Privacy Principles. A cross-tenant disclosure is an unauthorised disclosure of personal information, and if it is likely to result in serious harm it is notifiable. The OAIC reported that data breach notifications reached an all-time high in 2025 — 1,205 notifications, up 8% on 2024 and the highest annual figure since the scheme began in 2018, with 716 attributed to malicious or criminal attack. APP 8 also applies the moment your inference provider or vector store sits offshore: you generally remain accountable for what an overseas recipient does with the data. Our guide to data sovereignty for AI in Australia unpacks the hosting decision.
Automated decision-making transparency. From 10 December 2026, APP entities that use personal information in automated decision-making capable of significantly affecting an individual’s rights or interests must disclose that in their privacy policy. The OAIC is consulting on guidance ahead of commencement. Architecturally, this means you need to know which of your tenants’ workflows are decisions rather than suggestions — and be able to say so per tenant.
APRA CPS 230. If you sell to APRA-regulated entities, CPS 230 has been in force since 1 July 2025, and the transitional arrangements that applied to pre-existing contracts ended on 1 July 2026. Your customers are now expected to manage you as a material service provider, which in practice means tenant isolation, incident notification timelines and exit plans will show up in their due diligence questionnaire. Build the answers before the questionnaire arrives.
A defensible default for a new multi-tenant AI product:
SET LOCAL per transaction.That last item is the one teams skip and later wish they had not. An isolation guarantee you do not test on every deploy is an isolation hope. It belongs in the same suite as the rest of your pre-launch checks — see our production-readiness checklist and our guide to running AI evals.
Neomeric, a Melbourne-based AI product and consulting company — and the team behind NeoMind, Australia’s onshore AI teammates platform — builds and ships multi-tenant AI products for Australian founders and businesses, so these are the decisions we make weekly rather than theorise about.
Yes, provided the tenant key is a required indexed field on every vector and the filter is enforced below the application layer rather than passed in by each caller. Vector database vendors including Qdrant recommend a single collection with payload-based partitioning as the default for services with many independent users. The risk is not the shared collection — it is a code path that can construct a query without the filter.
Usually not for every customer. Full stack isolation removes noisy-neighbour effects and simplifies the security story, but it is the most expensive model to operate and the slowest to update. The common pattern is to pool most tenants and offer dedicated infrastructure as a paid enterprise tier, which AWS’s SaaS Lens describes as tier-based isolation.
Not on tenant data without explicit rights to do so. Fine-tuning bakes information into weights that serve every tenant, and it cannot be reversed by a deletion request. If you fine-tune, do it per tenant or on data you own. The same caution applies to few-shot examples placed in a shared system prompt.
In the layers that are not the database: shared semantic or prompt caches keyed only on prompt content, observability traces that capture full prompt text into one index, and agent tool calls that accept a tenant identifier as a model-supplied argument. Each bypasses the authorisation middleware that protects your main API.
Yes. A cross-tenant disclosure of personal information can be a notifiable data breach, and APP 8 makes you accountable for personal information sent to overseas recipients such as offshore inference providers or vector stores. From 10 December 2026, entities using personal information in automated decision-making that can significantly affect a person’s rights or interests must also disclose that in their privacy policy.
Tag every model call with a tenant ID at a single gateway, then enforce per-tenant rate limits and spend ceilings at that same choke point before the provider call is made. Add tier-based model routing so routine requests use a smaller model. Reporting after the fact tells you what happened; only a gateway control stops it happening.
Neomeric is a Melbourne AI product studio — 7+ products shipped, including our own. Start with a free 15-minute scoping call, or a 2-week Build Sprint at A$6,900 fixed, fully credited toward your pilot.
What an AI MVP really costs in Australia in 2026 — line-item budgets, the traps that blow them out, and how to scope a build that pays for itself.
AI Insights
Build an AI data pipeline for RAG that stays fresh: change detection, chunking, contextual retrieval, hybrid search and reranking. A 7-stage how-to guide.
AI Insights
AI for veterinary clinics in Australia: what it should handle, what must never touch it, onshore hosting, and how fast setup really is. Read the guide.
AI Insights
The 10 AI development tool categories that matter in 2026, what each is genuinely for, and the four you should not launch without. A builder's shortlist.