“Runs locally” answers one important question:

Where does model inference happen?

It does not answer every other privacy question.

A local model can process a prompt on your PC while the surrounding workflow still uses cloud sync, online search, backups, hosted tools, remote access, or shared storage.

The useful rule is simple:

Judge privacy by the path the data takes, not by the word local on the product page.

Follow one prompt from start to finish

For a local-only session, ask where each stage happens.

1. You type the prompt

It begins on your computer.

But synced clipboards, remote desktop software, browser extensions, or accessibility tools can create other paths.

2. The application prepares the request

For a local session, that should remain on the computer.

The boundary changes if you select a hosted model, cloud mode, online plug-in, remote server, or other networked feature.

3. The model generates the answer

With a local model, inference happens using the computer’s CPU/GPU and memory.

If a hosted model is selected instead, the prompt must be sent elsewhere for generation.

4. Chat history is saved

The model can be local while its chat files are stored in a folder that is backed up or synchronized.

That may be acceptable. It simply means “never leaves the computer” is not an accurate description.

5. You attach a document

A document can be read locally while the folder containing it is also synchronized through OneDrive, Dropbox, or another service.

Local inference does not cancel an existing sync relationship.

Five places people commonly forget

Cloud-synced folders

If the source document already lives in a synced folder, another copy may already exist off-device.

Decide whether your goal is:

  • not sending the document to an AI inference provider; or
  • keeping the file entirely on the device.

Those are different privacy goals.

Backups

Chat histories, exported answers, prompt libraries, and working documents may enter normal cloud backup.

Backups can be valuable and appropriate, but they change the privacy description.

Browser helpers and extensions

A browser extension can access page content independently of the local model.

Assess the whole workflow, not only the AI application.

Web tools, plug-ins, remote APIs, and connected services

Any feature that calls an external service can move data outside the local application.

Enable tools one at a time and understand what is sent before using sensitive material.

Shared computers

A chat can be private from an AI provider but visible to another person using the same Windows account.

If privacy between users matters, use separate accounts and appropriate permissions.

The six-question boundary test

Before putting genuinely sensitive information into a local-AI setup, answer:

  1. Is the active model local or hosted?
  2. Does the feature I am using require internet access?
  3. Where are chats and attached documents stored?
  4. Is that location synchronized or backed up?
  5. Have I enabled any tools that can send content elsewhere?
  6. Who else can sign in to the computer or read the same folders?

If you cannot answer one of those questions, treat that part of the boundary as unknown until you can.

Local inference is not the same as offline operation

Some desktop local-AI applications can run downloaded models without the internet while still needing connectivity for:

  • model search;
  • model downloads;
  • runtime downloads;
  • software updates;
  • optional cloud features.

A practical workflow can therefore be:

  1. connect;
  2. download the application, runtime, and model;
  3. verify the files and setup;
  4. disconnect for a local-only session when that is part of your requirement.

If your actual requirement is only “do not send this private document to an AI cloud inference service,” blocking every update check may be unnecessary.

Define the requirement before choosing controls.

Use an offline test as evidence, not a guarantee

To test one workflow:

  1. download everything required while connected;
  2. run a normal local prompt;
  3. close the application;
  4. disconnect networking;
  5. reopen it;
  6. select the downloaded local model;
  7. repeat the prompt;
  8. test one local document if document chat is part of the workflow;
  9. record what worked and what failed;
  10. reconnect.

If the workflow still works, you have evidence that that tested path can run without a network connection.

You have not proved that the whole application makes no requests when connected, that the operating system is isolated, or that the computer is secure against other software.

Keep online features off until you need them

A pure local chat does not need web search, a hosted model, or a remote tool.

Leaving unnecessary features disabled has two advantages:

  • it reduces the number of possible data paths;
  • it makes the setup easier to explain.

A privacy-conscious setup should be understandable enough that you can describe it in a minute.

Be precise when describing privacy to other people

Avoid broad claims such as “your data never leaves our office” unless you have checked:

  • backups;
  • synced folders;
  • email;
  • remote support;
  • browser features;
  • networked tools;
  • every other part of the workflow.

A narrower statement can be more accurate, for example: model inference for a defined workflow is performed on a managed local workstation and the documents are not sent to an external AI inference provider.

That still requires the rest of the storage and access path to be assessed separately.

Local does not mean compliant

For regulated, legally privileged, highly sensitive, or safety-critical material, follow the organization’s approved systems, policies, access controls, retention rules, and legal obligations.

A consumer local-AI setup is not a shortcut around governance.

If private documents are your main use case, continue with how to use local AI with private documents.