Send the region, not the whole screen.
The short answer
Decision one: where the agent executes, meaning whose machine, browser, session and credentials. Decision two: where the model runs, meaning a provider's API, your own cloud account, or your own hardware. Answer them separately.
Four architectures
| Setup | Agent runs | Model runs | What leaves your network |
|---|---|---|---|
| Cloud agent, cloud model | Vendor's machine with your credentials | Vendor's provider | Credentials, every screenshot, results, logs |
| Local agent, public model API | Your machine | Provider's API | Only what each model call needs |
| Local agent, your cloud tenant | Your machine | Your Azure or AWS account | Model inputs, inside your contract and region |
| Local agent, self-hosted model | Your machine | Your hardware | Nothing |
The first row moves two things off site that the others keep home: your credentials and the full record of the work.
Ask providers two different questions
"We don't train on your data" and "we don't keep your data" are different promises. Ask for both, with retention in days, and confirm them in your signed agreement, not a web page.
Also ask where processing happens. Most major providers now let you pin a region or keep data within a zone.
Send the model less
The model only needs what the step needs.
Send text instead of pixels when it reads reliably.
Mask sensitive fields like SSN and bank account before the call.
Give exact-answer steps to code. A threshold check needs no model.
Never let the model see a password.
Regulated data
Export-controlled drawings (ITAR) and controlled unclassified information (CUI) raise the bar. A model has to decrypt a screenshot to read it, so sending one to a third-party model may not fit the rules. Talk to your compliance officer before the first run. Until then, keep those screens on rule-based steps, a human review step, or a model they have approved. The rest of the workflow can still run.
Local is not safe by default
A local agent is as secure as the machine it runs on. Encrypt the disk, lock the screen, patch the OS, enroll the machine so it can be revoked, and give workflows access only to the apps and folders they need. Protect the evidence too. AI Agent Audit Trail →
- Disk encrypted
- Screen lock
- Patched
- Enrolled
- Least access
- Evidence protected
Decide step by step
Steps
- 01
Classify each screen and document: public, internal, confidential, regulated.
- 02
Ask whether each step needs a model at all. If not, nothing leaves.
- 03
Regulated data on a model step: self-hosted, an approved enclave, or no model.
- 04
Confidential: no training, short or zero retention, pinned region, signed data agreement.
- 05
Internal and public: standard API terms usually suffice.
- 06
Run the agent locally when the work depends on the user's session, office IP or credentials you won't hand over.
Six questions for any agent vendor
Which machine clicks, and whose IP does the site see?
Where are credentials stored, and can vendor staff get them?
Which model provider sees screen content, under whose contract?
What does the vendor itself receive?
Where is the evidence stored, and who can delete it?
Can a step use a specific model, or none?
How LaunchAI answers
LaunchAI agents run on your machine. Screens and documents are handled there, and nothing goes to LaunchAI for processing. Model steps send only the region being read, to the model account you choose: OpenAI, Anthropic, Azure, Bedrock or self-hosted. Rule-based steps make no model call. Product →
Questions
Can one workflow use different models?
Yes, and it should when sensitivity varies. Navigation on a public API, a controlled drawing on a self-hosted model, exact checks on none.
Is a self-hosted model accurate enough?
Test it on your own screens with a small labeled set before deciding.
Does local keep us out of privacy law?
No. It reduces how many processors touch the data. Every model provider you use still belongs on your data map.
Keep learning
- Why Browser Agents Get Blocked Sites score every visit on signals a scripted browser gets wrong: a datacenter IP address, a connection fingerprint that doesn't match the browser it claims to be, automation flags, robotic input timing and request rates no person produces. Read the guide
- What Is a Computer-Use Agent A computer-use agent is AI software that operates a computer through its screen. Read the guide
- What Is an ADK An agent development kit (ADK) is a packaged set of building blocks for creating, testing and running AI agents: a model interface, tools that act on systems, orchestration, guardrails, human checkpoints, evaluation, tracing and a runtime. Read the guide
No prompts. No babysitting. No Integrations.