Fundamentals

Local LLM vs. Cloud LLM for Coding Agents

Not a verdict, because there isn't a universal one — this blog runs both local models and cloud-backed agents in isolated VMs, and the right choice genuinely depends on the workload. What actually determines it, laid out plainly rather than argued toward a predetermined answer.

Where cloud models still clearly win

Raw capability, first: the largest frontier cloud models remain meaningfully stronger than anything that fits in consumer unified memory, on the harder end of coding tasks — multi-file refactors, subtle bug diagnosis, reasoning through an unfamiliar codebase's architecture. That gap has narrowed considerably as local models and quantization have improved, but it hasn't closed, and pretending otherwise sets up a frustrating experience on genuinely hard problems. Context window is the second, related advantage: cloud models commonly offer context windows well beyond what's practical to hold in local unified memory at the same time as the model weights themselves — see the RAM math post for exactly why KV cache, not weights, is what actually runs out first on a long local session.

Zero local setup is the third: point an agent at a cloud API key and it works immediately, with no model download, no quantization decision, no hardware ceiling to think about — genuinely the lower-friction path if you're not already committed to a local workflow.

Where local models win

Cost at volume is the most concrete: a local model's marginal cost per token, once downloaded, is electricity — no per-request billing, which matters a great deal for an agent that might make dozens of tool calls and self-corrections per task, each one a separate model invocation. This blog's own benchmarks quantify the isolation overhead side of that equation, but the API-cost side is the more consequential one for anyone running agents constantly rather than occasionally.

Privacy is the second, and it's categorical rather than a matter of degree for some workloads: a local model never sends your code anywhere, which matters directly for proprietary codebases, client work under an NDA, or anything regulatorily sensitive — the AI Bridge data privacy post covers exactly what does and doesn't leave the Mac in Velo's own architecture. Offline reliability is the third: a local model keeps working on a plane, on a bad connection, or during a cloud provider's outage, none of which is hypothetical for anyone who's actually depended on a cloud API during an incident.

The isolation question applies to both, differently

Worth being clear about a common conflation: running a model locally and running an agent in an isolated VM are separate decisions, not the same one. This blog's own agent-isolation architecture sandboxes the agent's actions — filesystem access, command execution — regardless of whether the model behind it runs on your GPU or a cloud API. A cloud-backed agent still benefits from VM isolation, for exactly the reasons covered in the AI agent security risks post; a local model doesn't automatically make an ungoverned agent safer, because the risk lives in what the agent does, not in where the model's weights happen to live.

A practical pattern, not a rule

Plenty of real workflows split the difference rather than picking one exclusively: a local model for the high-volume, lower-stakes loop — linting fixes, small refactors, routine boilerplate, anything run constantly where cost and latency compound — and a cloud model reserved for the harder, less frequent problem that actually benefits from frontier capability. The AI Bridge architecture supports pointing at either kind of backend from the same isolated agent setup, which makes switching a configuration change rather than a rebuild — worth designing for that flexibility even if you start out leaning one way.

The actual decision

Cost-sensitive, high-volume, or offline-dependent → lean local, and see the quantization post for getting the most capability out of the memory you have.
Privacy-critical or regulatorily sensitive → local isn't optional, it's the requirement — start there regardless of the capability tradeoff.
Hard, infrequent, high-stakes problems → lean cloud, and isolate the agent regardless of which backend it's calling.
Not sure yet → build the agent setup to support both from the start, and let real usage tell you where the balance actually sits.