---
title: "An 80-Billion-Parameter Model Now Runs on a Mac in 4.3 GB"
description: "Swiftlet runs an 80B-parameter Qwen model on a Mac using 4.3 GB of RAM by streaming expert weights from SSD. How the trick works, and what it costs."
dek: "A new open-source runtime called Swiftlet keeps most of a large language model on the hard drive and fetches the pieces it needs mid-sentence. The same trick puts a 35-billion-parameter model on an iPhone — with one honest catch."
published_at: "2026-08-05T04:59:26.434Z"
updated_at: "2026-08-05T04:59:26.434Z"
tags:
  - "AI"
  - "Consumer"
  - "Infrastructure"
source_url: "https://github.com/leonickson1/Swiftlet"
source_domain: "github.com"
canonical: "https://hex37.com/80-billion-parameter-model-runs-on-a-mac-in-4-3-gb"
---

A [large language model](https://en.wikipedia.org/wiki/Large_language_model) is, physically speaking, a very large table of numbers. Those numbers are called parameters, and the count of them is the number that gets attached to a model's name: 8 billion, 70 billion, 80 billion. Each one has to be stored somewhere, and each one has to be read by the chip doing the arithmetic.

That leads to a piece of arithmetic that anyone can do at home. An 80-billion-parameter model, stored in a compressed form, occupies about 42 gigabytes. A well-specified MacBook has 16 or 32 gigabytes of RAM — the fast working memory a processor can reach instantly. An iPhone has rather less. So the model does not fit, and the conclusion follows: this class of model belongs in a data centre, on hardware that costs more than a car.

Which makes the numbers published by a new open-source project called [Swiftlet](https://github.com/leonickson1/Swiftlet) look like a typo. Running an 80-billion-parameter Qwen model on an M5 Mac, it reports a peak memory use of 4.3 gigabytes, and produces text at roughly five words' worth of output per second. A smaller 35-billion-parameter model from the same family runs in 2.6 gigabytes, at seven to eleven tokens per second — and runs on an iPhone 17 in about 2.5 gigabytes, slowly, at around one token a second.

Nothing has been compressed away. The 42 gigabytes are still 42 gigabytes. They are simply somewhere else.

## Most of the model is idle most of the time

The assumption that breaks is the one nobody states out loud: that generating a word requires touching every parameter.

For a conventional model, it does. But the Qwen models Swiftlet targets are built as a [mixture of experts](https://en.wikipedia.org/wiki/Mixture_of_experts). Instead of one enormous block of numbers that every input passes through, each layer of the network contains hundreds of smaller specialised blocks — experts — plus a small routing component whose only job is to decide which of them a given piece of text should be sent to.

The routing is aggressive. In the 80-billion model, each layer holds 512 experts and picks 10. In the 35-billion model, it picks 8 out of 256. Multiply that sparsity across the whole network and the working figure is that about 3 billion parameters — under four per cent of the total — actually do any work for any given token, the chunk of roughly a word that the model emits at each step.

The other 96 per cent are not being used. They are merely *available*, in case the next token routes somewhere else.

## Fetching the parts as they are called for

That gap between "stored" and "used" is the opening. Swiftlet splits the model in two. The parts that every token needs regardless of routing — the attention machinery, the routers themselves, the shared components, the vocabulary tables — stay permanently in RAM. That dense core is about 1.3 gigabytes for the smaller model and 2.5 for the larger one. Everything else lives on the solid-state drive and is read in only when a router asks for it.

Doing that quickly is the engineering. Text generation proceeds one token at a time, layer by layer, in a strict order, which means the system always learns which experts it needs a fraction of a second before it needs them. To make each fetch cheap, Swiftlet repacks the tens of thousands of experts into a single container file where every expert occupies a slot of identical size, at a predictable position. Retrieving one is a single direct read from the drive — not a hunt through a file, not a request to the operating system's general-purpose caching machinery, which tends to thrash when asked to juggle gigabytes it cannot hold. Recently used experts are kept in a fixed-size pool in memory, evicted on a mix of how often and how recently they were called.

The telling detail is how little that cache matters. Across measured hit rates ranging from 43 to 70 per cent, throughput barely moved. Apple's SSDs are fast enough that missing the cache and going to disk costs almost nothing in practice.

## The bottleneck is now the scheduler, not the disk

Which relocates the problem entirely. By the project's own account the decode loop is now limited not by storage but by dispatch — the fixed overhead of handing each small piece of work to the graphics processor and waiting for it to be scheduled. When the amount of arithmetic per operation is tiny, as it is when only 3 billion parameters are active, that per-operation tax dominates. The whole forward pass runs on Apple's GPU programming layer, with the shader programs compiled at runtime so the same code ships to phones as to Macs. Being dispatch-bound is, from a developer's point of view, good news: it means the current speeds are a floor, not a ceiling.

One more architectural choice keeps memory flat as conversations get longer. Most models must retain a growing cache of intermediate values for every token processed so far, which is why long documents blow up memory even when the model itself fits. Three-quarters of the layers here use a form of attention that carries a fixed-size running state instead — it does not grow with context length at all.

Stack these together and the shape of the answer is clear. Capacity was never the real constraint; it only looked that way because we insisted on holding the whole model at once. Route sparsely enough, lay the weights out for fast retrieval, and an impossible memory requirement turns into an ordinary bandwidth requirement — and bandwidth, unlike RAM, is something a $200 drive already has.

## What the trick costs

It is not free, and the project says so plainly. A model that activates 3 billion parameters per token writes and converses like a large model, because fluency comes from the shared machinery. But it recalls facts like a small one, because factual knowledge is distributed across all those experts that are not consulted. You get the manner of an 80-billion-parameter model with something closer to the memory of a 3-billion one — a trade that is fine for drafting and reasoning aloud, and considerably less fine if you wanted a local encyclopedia.

There is also the disk itself: 18 gigabytes for the smaller model, 42 for the larger. On a 128-gigabyte phone, that is a serious ask.

Swiftlet is Apache-licensed and works end to end today, as a command-line tool, a local server that imitates the OpenAI interface, a Swift package for app developers, and an iOS app. Every layer has been checked against reference implementations, and the project is explicit that an expert produces identical output whether it came from memory or from disk. Whether the knowledge trade-off is one ordinary users will accept is the question the next year answers.