Runs in your infrastructure: what that means in practice

A phrase everyone uses

"Runs in your infrastructure" appears on most enterprise software pages and means something different on each. Sometimes it means a connector runs in your network while the product runs in the vendor's cloud. Sometimes it means a single-tenant instance that the vendor still operates. Occasionally it means what it says. The phrase is worth defining precisely, because the difference decides who can reach your data and under whose keys, and that is the question a security review is actually asking.

What it should mean

The whole product. Not a front end with a hosted back end, and not an agent that ships data elsewhere. The index that holds the copy of your records, the dashboards that read it, the assistant that answers from it, the MCP server that exposes it: all of it deployed on servers you control, on-premise or in a cloud account that is yours.

Nothing hosted by the vendor. No tenant on the vendor's platform, no copy of your data on their side for support, no telemetry that carries records. The vendor's involvement is deploying, configuring and upgrading, inside your boundary, with your people present.

Your keys. The key to the LLM provider is yours, so what the assistant sends goes under your account. The keys AI clients use are issued from your deployment and revocable by you. Your users are managed by you. A product that runs in your infrastructure but holds its own keys to outside services has moved the boundary without telling you.

What can still leave, and why that is fine

Running inside the boundary does not mean nothing ever crosses it. An assistant that uses a language model sends that model the information the request needs; an MCP client receives the results it asked for. The point of running inside your infrastructure is that those crossings are the only ones, that they go to parties you chose, and that they happen under keys you hold. A vendor who says "nothing leaves" of a product with an AI feature is either not using a model or not telling you where it is. The honest statement is per flow, which is what Data flow map provides.

Support without a copy

The usual objection is support: how does a vendor help with a product it cannot see? The answer is the way infrastructure vendors have always done it - with your people in the room. Logs are read on your side, configuration is changed in a session you control, and an upgrade is a package you install on your schedule. What the vendor does not get is a standing copy of your records for convenience, and a product that needs one to be supportable is a product that is not quite running in your infrastructure. The question to ask is simply whether support ever requires data to leave.

Deployment models

Private cloud and on-premise are the same promise in two locations: your account or your building. Vendor-hosted is a different promise - convenience in exchange for the boundary - and it can be the right trade for software that does not hold your records. For a product whose job is to hold a copy of your orders, customers and invoices, it usually is not.

What it asks of you

Servers or a cloud account, a network boundary, and a team willing to coordinate upgrades. In exchange: the index is on your hardware, the copy of your data never had a second home, data residency is wherever you put the servers, and a security review can be answered by pointing at a diagram rather than at a contract. For most organizations with regulated or simply sensitive data, that is the trade they wanted all along. One-way data flow covers what the deployment does and does not do to the systems around it.

Six questions to verify it

1. Which components run inside our infrastructure, and which, if any, do not?
2. Is any copy of our records held on the vendor's systems, for any reason, including support?
3. Who holds the key to the language model, and whose account is billed?
4. Who issues the keys that AI clients use, and can we revoke one without the vendor?
5. What crosses the boundary, per feature, and to whom?
6. Can we run it with no outbound connection to the vendor at all?

The answers should be short and specific. A vendor who means the phrase can give them in a few minutes, because they are describing a diagram they have drawn many times.

Related

EXPLAINER Data flow map: what leaves your environment, per product One boundary, four flows. What goes in, what stays, what goes out, to whom, and who controls it - for each product separately, because the honest answer differs. EXPLAINER One-way data flow Data goes from your systems into the index and never back. That one rule is why an indexed layer is easy to add, easy to audit and easy to remove. EXPLAINER Access keys and revocation No key, no connection. Keys issued from your own deployment, one per team or tool, rotated on a schedule and revoked in a moment. The whole access model, in one page.