Soup CLI
trysoup.devFine-tune Llama-3.1-8B on a 4 GB laptop GPU, and align on the same card. Soup is at v0.74.0, tagged 4 September 2026, and 116 of the 120 pull requests merged into it came from somebody other than the maintainer, by 25 people. What outside eyes found is the release: the frozen base, the one part of a LoRA run that never receives an optimizer step, was being loaded in fp32 on every supervised path and in twelve more trainers, so an unchanged config paid twice its checkpoint precision for nothing. Measured on an H100 with Llama-3.1-8B and LoRA, peak memory falls from 48,241 MiB to 18,658 MiB, 2.59x, byte-identical across three repeats. A trainable base still loads fp32 deliberately. The same release moved the training stack to Transformers 5.16.1, TRL 0.29 and PEFT 0.20, which also makes soup-cli[train,mlx] resolvable for the first time, and it repaired the free notebook tier a second time: after v0.73.1 fixed the bf16 assumption, the whole pre-Ampere tier still could not stream at all, because PEFT builds LoRA adapters in the checkpoint's dtype while the fp16 gradient scaler needs fp32 gradients. Layer streaming admits a tenth architecture, Qwen4-Exp text, alongside Qwen3.5 dense and MoE aliases, and it now shards an untied embedding and LM head separately into one shared buffer instead of keeping both resident. The MCP server grew network transports, sse and http, both behind a mandatory bearer token and DNS-rebinding protection, with gated execution refused over a listener on purpose. Unknown config keys, silently dropped since forever, now warn with a suggestion and will be rejected from v0.75. Breaking: soup serve exits 2 rather than warning when bound to a non-loopback host without a tool auth token, because the shell endpoint it protects now really executes, and the SGLang backend finally obeys trust-remote-code instead of loading every model with it on. Four obfuscated IPv4 spellings that reached the telemetry, webhook and tracing validators are closed. Underneath, the v0.73 line is unchanged and still carries the flagship. Quantizing the streamed base to NF4 shrinks it about fourfold, so Llama-3.1-8B trains at 119.6 tok/s in 3.32 GB on a 4 GB card, a figure measured before v0.73.0's correctness repair and not re-run on that hardware since, and DPO, ORPO, SimPO and KTO stream too, with DPO's reference model costing no extra weights. v0.73.0 validated all of that on borrowed H100s against resident references at 8B through 72B, and disclosed the NF4 gradient defect that only such a reference could find. It ships BETA and every measurement behind it is published, the discarded numbers and the disclosed defect included. Plus soup ship, one SHIP or DON'T-SHIP verdict you can commit next to the weights, the Reward Forge verifier pair, the semantic data moat, the compliance pack, LoRA task arithmetic and Whisper fine-tuning. 163 recipes, all offline. Lean PyTorch-free install, then [train]. Python 3.10 to 3.12. On your own GPU.
US
1 evidence
Report
95/ 100 relative fit