Fine-tuning a model on your phone

Training a language model is supposed to require a data center. This is not a marketing claim by data center owners; it is a memory problem, and memory problems are unforgiving.

QVAC Fabric claims to fine-tune a billion-parameter model on a Samsung Galaxy S25 in about eighty minutes, and to push to roughly thirteen billion parameters on an iPhone 16. Both claims are surprising enough to be worth taking apart, because if they hold, the privacy argument for local AI changes shape entirely.

Why training is hard and inference is not

Running a model — inference — means holding the weights in memory and pushing numbers through them. A four-bit quantized seven-billion-parameter model is about four gigabytes. A modern phone has eight to sixteen. Tight, but it fits.

Training is different, and the difference is not a small multiple. To update a weight you need the weight, the gradient with respect to that weight, and the optimizer’s running state for that weight — Adam keeps two additional values per parameter. You also need the activations from the forward pass, held in memory until the backward pass consumes them. In full-precision training, the working set is commonly quoted at somewhere around sixteen bytes per parameter against the two bytes inference needs.

For a billion parameters, that is sixteen gigabytes of working memory to update a two-gigabyte model. On a phone this is not a tight fit. It is not a fit.

The two tricks

Fabric gets there by attacking both terms.

LoRA attacks the number of parameters that need updating. The insight, now several years old and thoroughly validated, is that fine-tuning does not require moving every weight. You freeze the original model entirely and inject small low-rank matrices alongside it, training only those. The frozen weights need no gradients and no optimizer state — they are read-only, like inference. Instead of updating a billion parameters you update a few million. The optimizer state, which dominated the memory budget, nearly vanishes.

BitNet attacks the size of each remaining number. It is an architecture from Microsoft Research that compresses model weights far below the usual quantization floor, and the frozen base model — which is most of what is sitting in memory — compresses with it. Tether reports the combination reduces memory requirements by roughly 78%.

Neither technique is Tether’s. What Tether did was make them work together on hardware nobody had targeted.

The hardware problem, which is the real problem

Here is what is actually novel, and it is not the memory arithmetic.

Training has been effectively locked to NVIDIA. Not because the mathematics requires it, but because the software stack — CUDA, cuDNN, the whole accumulated tooling — was built for one vendor’s hardware, and everyone else’s GPUs were left to run inference or nothing. This is a moat made of software, and it has held for a decade.

Fabric is hardware-agnostic. It switches between Vulkan, CUDA, and ROCm depending on the GPU in front of it, which means AMD, Intel, Qualcomm’s Adreno, ARM’s Mali, and Apple’s A- and M-series silicon all become training targets. Vulkan is the interesting one: it is a graphics API, supported essentially everywhere, and using it as a compute backend is what lets the same code run on a desktop RTX card and a phone GPU without a separate implementation.

The remaining obstacle on mobile GPUs is that they choke on large matrix operations — the memory required for a single big multiply exceeds what the chip will allocate. Fabric’s answer is a dynamic tiling algorithm that segments those operations into pieces the hardware will accept, trading a little overhead for the ability to run at all.

That is the actual breakthrough. Not “we made training smaller,” but “we made training run on the GPU that is already in everyone’s pocket.”

Why this matters more than it sounds

The privacy case for local inference has always had a hole in it, and people who make the case tend not to mention it.

You run the model on your device, so your prompt stays put. Fine. But the model was trained on a corpus you did not choose, by a company you did not select, and it adapts to you not at all. It is a stranger’s model, running locally. Privacy is preserved and personalization is impossible, and personalization was half the reason to want the thing.

The alternative was worse. If you wanted a model that knew your writing style, your medical history, your codebase, you uploaded all of it to somebody who would fine-tune on your behalf — which is to say, you solved personalization by abandoning privacy completely, and more thoroughly than a single prompt ever would.

On-device fine-tuning closes the hole. The model adapts to your data because the training happens where the data already is. Nothing is uploaded, because there is nowhere to upload it to. You get a model that has read your notes without anyone else’s model having read your notes.

For a health application this is the whole ballgame. A model fine-tuned on years of your own biometrics, correlating your sleep against your glucose against your training load, is worth substantially more than a general model reasoning about a population you belong to statistically. And the data required to build it is the data you would least like to send anywhere.

What to hold lightly

Eighty minutes on a flagship phone. Not a mid-range one, not one that is three years old, and presumably not one you are also trying to use. Battery and thermal behavior over an eighty-minute GPU saturation are not addressed in the material, and anyone who has watched a phone throttle under sustained load knows the answer is not nothing.

The thirteen-billion-parameter iPhone figure is described as pushing the limits, which is the language of a demonstration rather than a shipping configuration.

And LoRA fine-tuning is not full fine-tuning. It adapts a model. It does not teach it a domain it has never seen. The gap between those two is real and is sometimes exactly the gap you need to cross.

These are the caveats you would apply to any first-of-its-kind result. They do not make it less first-of-its-kind. A regular smartphone can now learn from data that never leaves it, on hardware from five vendors rather than one, using an open-source framework under Apache 2.0.

The interesting part is not the eighty minutes. It is that a category of software which was previously impossible — an application that quietly gets better at your particular life, without any of your particular life going anywhere — has just become something a developer can build this afternoon.

QVAC Health is the closest thing to this pattern shipping today, correlating your own biometrics on the device that collected them.