· privacy
Why private AI is not a checkbox
“Private AI” is useful only when it names a boundary and a control. A model running on a local server may still expose prompts in logs, allow broad administrator access or return data through an unreviewed integration. A cloud deployment can be tightly controlled, but a private network label alone does not establish who can inspect data or how long it is retained.
Map the complete data path
Start with one workflow and trace information from source to deletion. Include the user’s prompt, uploaded files, retrieved passages, embeddings, model requests and responses, application logs, traces, backups, analytics, support tools and downstream records. For each step, record the system owner, people and services with access, location, retention period and whether the data can leave the intended environment.
Hypothetical example: an employee asks questions about internal procedures. The workflow may read a document store, create or reuse embeddings, send a retrieved excerpt for generation, log errors and store user feedback. A “private model” decision does not answer whether retrieval respects document permissions or whether support logs retain the excerpt. The team needs to define and test those controls at each hop. This is a boundary-mapping example, not a claim about a specific system.
Turn “private” into verifiable requirements
- Which data classes may enter the system, and what should be masked or excluded?
- Which roles can read documents, issue prompts, change policy or inspect logs?
- What does the provider retain, use for, disclose to subprocessors or expose to support staff?
- Where do backups, telemetry, model artifacts and administrative control planes live?
- How can access be revoked, a document removed, or an incident investigated?
- What is the fallback if the private endpoint or its identity service is unavailable?
Choose controls from the threat model. A restricted cloud service may be sufficient for some workflows; a self-managed or disconnected environment may be necessary for others and brings added patching, capacity and incident-response duties. Do not pay for isolation that fails to cover the actual data path, and do not call a deployment private until access and retention have been checked.
Use local GPU versus VPC perimeters to compare hosting choices, then review AI threat-model questions. See Private AI Services or share the data class and deployment constraint for a boundary review.