Two numbers, not one

A model's memory footprint isn't a single static figure — it's two genuinely different things added together: the weights, which are fixed the moment you pick a model and a quantization level, and the KV cache, which grows as a conversation gets longer. Sizing for weights alone is the mistake that makes a model work fine in a quick test and then run out of memory in a real session.

Weights: the simple part

Weight memory is roughly parameter count times bytes-per-parameter. A model's name tells you the first number — "7B" means 7 billion parameters — and quantization tells you the second: 4-bit quantization is roughly 0.5 bytes per parameter, so a 7B model at 4-bit lands around 3.5GB just for weights. Double the bit-width and you roughly double that figure. (For what's actually behind that bytes-per-parameter number — the different quantization schemes, and why they're not interchangeable at the same bit width — see LLM quantization explained for Mac users.) This is the part every model card and download page already tells you, directly or indirectly, and it's the part most "how much RAM" answers stop at — which is exactly the gap that causes surprises later.

KV cache: the part that grows on you

The key-value cache stores intermediate attention state for every token in the current context, and unlike weights, it scales with how long the conversation gets. The actual formula: 2 × layers × KV-heads × head-dimension × tokens × bytes-per-element — each of those numbers is in the model's own config, and most inference tools will report the resulting figure directly rather than making you compute it by hand. The practical upshot is what matters here: for a short exchange, weights dominate and KV cache is nearly irrelevant. For a long agentic session with a large context window — exactly the kind of multi-turn, tool-calling conversation an AI coding agent produces — KV cache can grow to rival or exceed the weights themselves. That's the mechanism behind "it worked when I started, then got slow or crashed later": nothing changed about the model, the context just kept growing.

The other consumer: macOS itself

Unified memory means the model isn't the only thing drawing from the pool — the OS and display compositor need their share too, which is why the existing model-sizing table reserves roughly 20–25% of total RAM for that overhead before budgeting the rest. That reserve isn't a made-up safety margin; it's covering real, measured system usage that doesn't show up in a naive "weights + KV cache" calculation.

Putting it together

A workable estimate for any model, not just the ones already tabulated on this blog: take the parameter count times bytes-per-parameter for weights, add a KV cache estimate scaled to how long a session you actually expect (short chat: small; long agentic session: meaningfully more), then make sure the total leaves that 20–25% headroom for macOS. If a model's weights alone already eat most of your unified memory, there's no room left for KV cache to grow into — that's the model to size down, not the one to fight with swap.

What it looks like when you get it wrong

Memory pressure on a Mac doesn't usually announce itself as a hard crash — it shows up as swapping, and an inference engine that was fast a minute ago suddenly isn't, with no error message pointing at the actual cause. If a model that felt snappy at the start of a session gets progressively slower rather than staying consistent, check context length and KV cache growth before assuming something's wrong with the model or the engine itself.

Related reading: the tier-by-tier model table this post explains the math behind, the broader landscape of local LLM tools, and how much memory to give a Linux VM for non-LLM workloads, including the memory-balloon device that handles the VM side of memory allocation. Or download Velo Workspaces and try it yourself.