AI
A 16GB Gaming GPU Now Runs a Three-Model Homelab
A 16GB RTX 4070 Ti Super now runs research, code, and images with no AI plan, as a three-model box that still calls a search API.
A 16GB RTX 4070 Ti Super can now run a research model, a coding model, and an image generator with no cloud AI subscription. It does that as a three-worker box, not as a single ChatGPT stand-in. One mixture-of-experts search model parks leftover experts in system RAM, a dense 27B coder spills six layers off the card, and an image pipeline swaps two helper models through VRAM in stages.
The card in the measured setup is the same RTX 4070 Ti Super that was bought for 1440p games, paired with a Ryzen 5 7600X. The AI bill on that desk is $0. The operating cost is RAM, a free-tier search key, a revenue-capped image license, and the patience to keep three weights in rotation.
Three Experts Share One 16GB Card
Local AI in late 2026 is no longer a hunt for one do-it-all weight file. The useful pattern is a small farm: a web-research specialist, a denser coder, and a separate image stack, all sitting on the same NVMe drive and taking turns on 16GB of GDDR6X. On this box the three are BigBang-v1 for search, Qwen3.8-27B for code, and FLUX 3 Image Klein for pictures.
BigBang-v1 is a fine-tune of Alibaba’s Qwen3.6-35B-A3B, released under Apache 2.0. The IQ4_XS file at 19.3 GB is larger than the card, which is why llama.cpp’s –n-cpu-moe flag exists. Qwen3.8-27B is a dense 27B model, also Apache 2.0; Unsloth’s UD-IQ4_XS cut of it is 14.3 GB and can sit in VRAM until the context cache shoves layers back to RAM. Klein is the odd one out: 5.6 billion parameters, not Apache, and it will not run from a prompt alone.
THE 16GB THREE-MODEL STACK
| Model | Job | On-disk weights | Measured pace | License |
|---|---|---|---|---|
| BigBang-v1 | Research | 19.3 GB IQ4_XS | 68-78 tokens a second | Apache 2.0 |
| Qwen3.8-27B | Coding | 14.3 GB UD-IQ4_XS | 15-17 tokens a second | Apache 2.0 |
| FLUX 3 Image Klein | Images | 5.6B INT8, plus two helpers | About 16 seconds at 1024×1024 | Kommunity, under $5 million |
Those speeds are from this 16GB box, not from a 24GB 4090. Klein’s 16-second figure is a five-run average for a 1024×1024 image and includes the prompt-upsampler step. Sampling itself was about 4 seconds. Keep that split in mind when vendor charts quote a faster GPU-only time on a different GeForce 4070.
The Research Model Still Calls the Web
BigBang-v1 is sold as an agentic 35B-A3B model. The endless-frontier repo describes it as 35B total, 3B active per token, which is why a 19.3 GB file can still write 68 to 78 tokens a second at a 128K context on this card. About 7GB of expert weights go to system RAM. Attention stays on the GPU. Only a handful of experts fire for each token, so the RAM copy is idle most of the time and the card does not stall the way a dense 19 GB model would.
Its headline score is 76.5 on BrowseComp, a long-horizon web-search test that uses OpenAI’s encrypted set and a five-attempt cap. On the same card it also posts 54.2 on SWE-Bench Pro. Both numbers come from the project’s own table. Closed models on that table sit much higher on BrowseComp, with Gemini 3.1 Pro at 85.9 and Claude Opus 4.8 at 84.3, so 76.5 is a 35B-class result, not a frontier one.
A local research model with a 2026 training cutoff still cannot answer a question about a fresh Windows 11 26H2 bug list from weights alone. The workaround on this desk was llama.cpp’s web UI wired to Exa over MCP with a free key. Five searches later, the model had Microsoft’s confirmed issues, dates, and KB numbers. It also invented a KB number and a fix that could not be traced back, which is why every citation still has to be opened by a person.
That search hop is the part the “$0 local” line skips. Exa’s starter plan gives $10 of search credits a month plus a $10 onboarding bonus, with no card required, and it exposes an MCP server. Lite search is $4 per 1,000 requests once the credits run out. The weights never leave the PC. The queries do. A research worker that cannot fetch pages is a summarizer of last year’s internet, so the homelab stays honest only while that key still works.
Klein Is Fast Under a $5 Million Cap
Black Forest Labs released FLUX 3 Image Klein on October 15, 2026, as the first downloadable image model in the FLUX 3 family. It is a 5.6B flow-matching transformer that does text-to-image, instruction edits, bounding-box layout, and up to 10 reference images from one set of weights. A base checkpoint is meant for LoRA work at 50 steps. The distilled checkpoint, the one this 16GB box actually runs, is built for 16 steps with no CFG.
BFL’s own figures put generator weights at 11.2 GB in BF16 and 5.6 GB in FP8. The INT8 build is the one aimed at GeForce cards with as little as 8GB. NVIDIA’s ComfyUI timings, quoted by BFL, are about 4 seconds for a 1MP image on an RTX 5090 and under 9 seconds on an RTX 4070. Those are GPU generation times. They do not include the upsampler, and they are not this 4070 Ti Super. On the Ti Super, wall-clock time for a 1024×1024 image averaged about 16 seconds across five runs, with about 4 seconds in the sampler and the rest in a 9B Qwen3.5 prompt rewriter. A 4B Qwen3-VL encoder then turns that expanded caption into embeddings Klein will accept. ComfyUI loads each INT8 build when that stage runs, then unloads it, which is the only reason three extra models fit next to a 16GB ceiling.
Labs that publish Klein tell you not to skip the upsampler. Short prompts produce generic frames because the transformer was trained on full scene captions. ComfyUI 0.41.0 or later is required, and the Hugging Face weights are gated behind a license click. There is no per-image watermark on a local run, and there is no usage cap of the kind hosted image tools still enforce. Speed is the settled part of the story. Testers on other 16GB cards, including an RTX 5080 running the distilled NVFP4 build at 16 steps, have seen 1K frames in about 8 seconds and still walked away from the look, especially on Chinese prompts, in favor of other open image models.
The weights are available under the FLUX Kommunity License, which allows free commercial use for individuals and companies with annual revenue under $5M, alongside research and non-commercial use.
Black Forest Labs, FLUX 3 Image Klein announcement
That sentence is the other half of “open weights.” A Qualifying User is anyone whose gross annualized revenue, affiliates included, stays under $5 million. Those users may generate outputs for business and may run the model in production for that purpose. Distributing the weights as a hosted product, or using them once the revenue line is crossed, needs a separate deal. BFL’s own API starts at $0.009 per 1K image for people who would rather not argue with the license. Apache 2.0 this is not, and the free commercial use under $5 million line is the part a small studio has to read twice.
Why the Dense Coder Is Worth the Wait
Keeping Qwen3.8-27B on the same drive as a 35B-A3B research model looks wasteful until the coding scores are next to each other. Qwen reports 61.7 on SWE-bench Pro for Qwen3.8-27B, using the Claude Code harness at a 256K window. BigBang-v1’s own card lists 54.2 on the same benchmark with the official mini-SWE-agent harness. Both figures are vendor-reported. The 27B model also posts 90.3 on LiveCodeBench v6 and 73.0 on Terminal Bench 2.1 in Qwen’s table. Those are not reasons to throw the MoE model away. They are reasons a dense coder still earns a slot when the job is a repo, not a web hunt.
The cost is speed, and it is mechanical. BigBang-v1 activates 3B parameters per token, so –n-cpu-moe can hide unused experts in RAM. Qwen3.8-27B is dense: all 27B parameters work on every token, across 64 layers. The UD-IQ4_XS file is 14.3 GB, which fits a 16GB card until you add cache. A 32K window needs about 2GB more, 14.3 plus 2 is 16.3, and llama.cpp then moves 6 of those 64 layers into system RAM. Generation falls to 15 to 17 tokens a second. Unsloth’s own guide says 4-bit quants on 16-19GB VRAM are the expected band, with 1-2GB extra if you turn on MTP draft decoding.
The native context on Qwen3.8-27B is 262,144 tokens, extendable toward 1 million with YaRN. This 16GB box does not run that window. It runs 32K because the cache would not otherwise fit. For Pygame-scale toys and short agent loops, 15 tokens a second is slow and still usable. For a full-day coding agent that wants 64K or 98K resident, owners of 16GB cards already report that leaving VRAM headroom matters more than maxing the context slider. The denser model is the quality pick. It is also the one that turns a 16GB card back into a tight box.
RAM Holds the Experts the GPU Cannot
The flag that makes a 19.3 GB MoE file runnable on 16GB is –n-cpu-moe, added to llama.cpp in PR 15077. It does not offload whole layers the way -ngl does. It keeps the mixture-of-experts feed-forward tensors for the first N layers in system RAM and leaves attention on the GPU. Higher N means more RAM and fewer tokens a second. Lower N means more VRAM use and more speed, until the process dies. On a 16GB card the right N is the smallest one that loads, not a number copied from a 24GB build.
That is why a 35B-A3B research model can feel faster than a 27B dense coder on the same GeForce. The MoE file is larger on disk and smaller in active compute. The dense file is smaller on disk and fully hot on every token. RAM is doing the job a second GPU used to do, and it only works if the desktop actually has tens of gigabytes of system memory to hold the parked experts. A 16GB laptop with 16GB of RAM is a different machine from a 16GB desktop with 64GB of DDR5, even when the sticker on the card is the same.
WHAT THE ZERO-DOLLAR STACK STILL NEEDS
- A live search key: BigBang-v1’s research job depends on Exa over MCP, with $10 of free credits a month before Lite search is $4 per 1,000 requests.
- System RAM as overflow: about 7GB of BigBang-v1 experts, plus 6 of Qwen3.8-27B’s 64 layers at 32K, sit off the GPU.
- Two extra image models: a 9B Qwen3.5 upsampler and a 4B Qwen3-VL encoder, loaded in stages inside ComfyUI.
- A revenue test on pictures: Klein’s Kommunity license is free for business only while affiliates stay under $5 million a year.
- A human on citations: the same research run that nailed Windows 11 26H2 KBs also fabricated one, so sources still get opened by hand.
None of those line items is a ChatGPT Plus invoice. Together they are why “the GPU I already own” is the cheap part of the story and the rest of the box is the product. People who treat local inference as a drop-in cloud replacement hit the search gap, the license line, or ComfyUI’s node graph first, then assume the models are the problem.
A Gaming PC Now Competes With GPU Hosts
The hosting consequence is sitting in that 16GB floor. Hobbyists who would have rented an entry GPU instance for weekend image jobs, or paid a monthly AI plan for code and search, now have a path that never issues an invoice, as long as the card has 16GB, the case has RAM, and someone is willing to run llama.cpp and ComfyUI. That does not empty a data center. It does nibble the bottom of the market that cheap GPU clouds sell to people who already own a 1440p gaming PC.
It also redraws who is left out. BFL’s INT8 Klein build can start on 8GB. The three-model stack cannot. An 8GB card can make pictures and still has to send research and coding to a host. A 16GB card can keep all three jobs in the case, with the caveats above. Operators who are still reserving cloud B300 GPUs are selling a different tier: big training and multi-tenant inference, not a single coder’s weekend batch. The local 4070 Ti Super is not in that fight. It is in the fight with $20-a-month chat plans and with the smallest rented 16GB slices.
HOW THE PIECES ARRIVED
- April 15, 2026: Qwen publishes Qwen3.6-35B-A3B on Hugging Face under Apache 2.0, 35B total and 3B active, the base BigBang-v1 later fine-tunes.
- August 3, 2026: the endless-frontier/BigBang-v1 repo goes up, with the BrowseComp 76.5 and SWE-Bench Pro 54.2 table attached.
- August 7, 2026: bartowski uploads the IQ4_XS GGUF, 19.3 GB, the file this 16GB box actually loads.
- October 15, 2026: Black Forest Labs releases FLUX 3 Image Klein, 5.6B, Kommunity license, day-one ComfyUI support.
- October 25, 2026: the three-model rotation on a 16GB RTX 4070 Ti Super and Ryzen 5 7600X is documented as a $0-subscription daily driver.
The weight files on that drive will not be the ones still there at Christmas. Open models at this size are turning over every few weeks, and the same 16GB card will load the next IQ4_XS without a new invoice. What does not turn over as fast is the shape of the box: one GPU, extra RAM, a search key, a license with a dollar line, and three specialists instead of one generalist. That is the homelab, and it is already cheaper than renting the same VRAM by the hour.
-
Cloud Hosting4 weeks agoUtho Asks India to Book B300 GPUs Before They Arrive
-
NEWS1 month agoRank Math PRO Joined a 2026 Pattern of Paid Plugin Backdoors
-
AI1 month agoGoDaddy Lets AI Agents Run Hosting, Email and Stores
-
NEWS1 month agoElementor Pro’s Form Bug Left Hosts Holding the Cleanup
-
NEWS4 weeks agoFirstVDS Disk Theft Exposes a Live VMmanager Flaw
-
Email Hosting1 month agoWindows Updates Cap RPC Calls and Stall Exchange Mail
-
Cloud Hosting1 month agoYandex Cloud Bots Moved to Selectel After the Strikes
-
Email Hosting1 month agoTitan Puts a $3 Inbox Beside Google on Squarespace
