Moonshot AI published the full weights for Kimi K3 late on July 26, 2026, roughly a day ahead of its stated July 27 target, making the largest open-weight model yet released something anyone can download. The coverage since has mostly stopped at that sentence. Free to download, historic milestone, the open-closed gap is closing. All true, and all skipping the question that actually decides whether this matters to your organization: self-hosting Kimi K3 requires hardware most companies do not have, and "you can download it" is not the same as "you can run it."
This piece is the boring, useful follow-up. What actually shipped, why mixture-of-experts sparsity does not rescue you the way people assume, what the memory floor really is, what that hardware costs against simply paying for the API, and who should genuinely consider running this in-house. The specific numbers will age as hardware improves, but the shape of the math will not.
What actually shipped
Moonshot released the weights on Hugging Face in the moonshotai/Kimi-K3 repository. The headline specifications, which we covered when the model launched in our explainer on Kimi K3, are a mixture-of-experts model at 2.8 trillion total parameters, activating 16 of 896 experts per token, with a 1,048,576 token context window and native vision support.
Two details matter more for self-hosting than any benchmark. First, the release is quantised to MXFP4, a 4-bit format, and the published weight shards come to roughly 1.56TB on disk, which is about 1.42TiB if your tooling reports binary units. For comparison, the same model at BF16 would be around 5.6TB, so the 4-bit format is doing enormous work here. It is worth being precise about what is quantised: only the mixture-of-experts expert weights drop to MXFP4, activations run at MXFP8, and the attention projections, routers, and shared experts stay at higher precision, which is why the total lands above a naive four-bits-per-parameter calculation. Second, and more usefully, Moonshot’s co-founders stated in a July 27 AMA that this is the same quantisation the company runs for its own hosted API. If that holds, it removes the usual worry that the public weights are a degraded version of the served model: self-hosted output quality should match what the API produces, because it is the same artefact.
One housekeeping note before you plan around a number. Secondhand download figures for this release have varied wildly, partly because 1.56TB in decimal units and 1.42TiB in binary units are the same quantity described two ways, and partly because community re-quantisations and pruned variants circulate under similar names. Check the repository itself and confirm which artefact you are pulling.
The mixture-of-experts misconception
Here is the assumption that derails most first attempts at sizing this. K3 activates only 16 of its 896 experts per token, so people reason that they only need to hold about 2 percent of the model in memory at any moment. That is not how it works, and getting it wrong is the difference between a workable plan and a wasted quarter.
Sparsity in a mixture-of-experts model reduces the compute spent per token, not the memory that has to be resident. The router picks which experts to consult per token, and it is not predictable in advance. Any expert can be needed for the next token, so all of them have to be loaded and immediately reachable. That is what makes MoE models efficient to serve at scale and stubbornly expensive to host: you pay full price in memory capacity for the entire parameter count while only getting the arithmetic savings of the active slice.
The practical translation is that the whole 1.56TB of MXFP4 weights has to sit in fast memory. Not a fraction of it. All of it. The sparsity buys you arithmetic, and the model activates roughly 104 billion parameters per token out of the 2.8 trillion it holds, but it buys you nothing at all on capacity.
The real memory floor
Start from the weights and work up. Roughly 1.56TB is the resident floor before the model does anything. On top of that you need memory for the KV cache, which holds the attention state for the tokens currently in flight, plus activations and framework overhead.
Here K3 does something genuinely clever, and it cuts against what you would expect. In a conventional transformer the KV cache grows linearly with context, so a 1,048,576 token window would dominate memory and make long context ruinously expensive to serve. K3 avoids that. Three of every four attention layers use Kimi Delta Attention, a linear-attention mechanism that carries a fixed-size recurrent state rather than an ever-growing cache, and only the one-in-four Gated MLA layers hold a global KV, in compressed latent form. The result is that the KV footprint at very long context is a fraction of what a full-attention model this size would demand. So the 1M context window is not the memory problem people assume. The binding constraint on this deployment is simply fitting the weights.
Put together, a realistic single-node target starts around 2TB of GPU memory: 1.56TB of weights plus working room for cache, activations, and framework overhead. For reference, a single NVIDIA DGX Spark carries 128GB of unified memory, which is roughly a twelfth of what the weights alone require. This model does not fit on a workstation, and no amount of clever configuration changes that. Our guide to hardware for AI agents covers why memory capacity and bandwidth, rather than raw FLOPs, are usually the binding constraint on inference.
What hardware actually clears the bar
This is where a lot of the early sizing advice went wrong, including some published in the first days after the release. An 8-way NVIDIA H200 node offers roughly 1,128GB of GPU memory, and that was the number many people reached for. It does not fit. The weights alone exceed it by a comfortable margin, so the previous generation’s standard inference node is simply not a candidate.
The realistic single-node entry point is eight NVIDIA B300 accelerators at 288GB each, roughly 2.3TB in total, which holds the weights with genuine room for cache and activations. The alternative is 16 B200s split across two nodes, which works but requires a high-bandwidth interconnect between them rather than two independent servers sitting in the same rack. MXFP4 also runs natively on Blackwell and AMD MI355-class parts, so on older hardware you fall back to slower emulated paths even if you can find the capacity. Moonshot’s own guidance points at clusters larger still.
On cost, Blackwell-class capacity prices above the H200 generation it replaces, and newest-generation supply fluctuates enough that any figure quoted today is a snapshot. Rather than repeat a rate that will be stale within weeks, the more durable approach is the formula operators actually use: your effective cost per million output tokens is the node’s hourly rate divided by sustained output tokens per second, divided by 3,600, multiplied by a million. The one input you cannot assume is sustained throughput, because it depends on your batch sizes, context lengths, and whether speculative decoding is enabled. Measure it on your own traffic. Buying hardware outright rather than renting shifts the problem to capital expenditure, data-centre space, power, and cooling, which is a different conversation and rarely a cheaper one at a single-node scale.
The comparison nobody runs
Set that against the alternative. K3’s hosted API is priced at $15 per million output tokens, with input at $3 per million on a cache miss and $0.30 on a cache hit, and several day-zero hosting partners matched that pricing rather than undercutting it. To justify a continuously-running Blackwell node, you need to be consuming an awful lot of tokens, and you need to be consuming them steadily rather than in bursts, because an idle GPU node bills exactly the same as a busy one.
Crossover estimates circulating for models in this class tend to land in the low tens of thousands of dollars per month in API spend before dedicated hosting starts to make financial sense, and those estimates typically fold in power, cooling, and the staff time to operate the thing. The exact threshold depends heavily on your utilisation and on hardware rates that are moving quickly, so treat it as a direction rather than a number. The direction, though, is consistent: unless your usage is both large and continuous, the API wins on cost. Our explainer on input versus output tokens is useful for estimating where your own spend actually lands, since output volume drives most real bills.
The uncomfortable arithmetic for a small or mid-sized business is that you would need to be spending more on AI inference every month than many companies spend on their entire cloud footprint before self-hosting K3 pencils out on price alone.
Quantisation will not save you
The instinct at this point is to reach for more aggressive quantisation. It is worth understanding why that lever is mostly already pulled.
MXFP4 is a 4-bit format, and Moonshot is shipping the model at the same precision it serves in production. That is already an aggressive compression of a model that would occupy several times the space at 16-bit. Community re-quantisations in other formats will appear, as they always do, and some will squeeze the footprint further. But you are optimising against a floor that has already been lowered once, and every additional step down trades away quality in ways that are hard to predict per workload and that vendors rarely quantify honestly. Going below the precision the model’s own authors ship is a decision to accept unmeasured degradation in exchange for a memory saving that, at this scale, still will not get you onto a workstation.
The honest framing is that quantisation determines whether you need one server node or several. It does not determine whether you need a server node at all. You do.
So who should actually self-host this
Cost is the wrong reason to run K3 yourself. There are good reasons, and they are all about control rather than economics.
Data sovereignty is the strongest one. If your inputs cannot leave your infrastructure for regulatory, contractual, or competitive reasons, then the calculation changes entirely, because the API is not an option at any price. Running weights you hold yourself, on hardware you control, is the whole point. That is the case where a K3-class open-weight model genuinely unlocks something no closed API can offer, and it is worth noting that a hosted Chinese API carries its own data-governance considerations for many organizations, which makes the self-hosted path more than a preference.
Customisation is the second. Open weights let you fine-tune, inspect, and adapt the model in ways an API forbids. If your differentiation depends on that, you are buying capability, not saving money.
Predictable heavy volume is the third. If you genuinely run continuous, high-volume inference, dedicated capacity can beat metered pricing, and the crossover math above starts working for you rather than against you.
For everyone else, and that is most organizations, the realistic answer is to use K3 through a hosted endpoint, whether Moonshot’s own or one of the four day-zero providers that stood the model up on release day, and we have costed that path separately in what Kimi K3 hosting actually costs, and treat the release as good news about pricing pressure rather than an infrastructure project. Our overview of local AI models covers the smaller open-weight models that genuinely do run on hardware you already own, which is a far more practical starting point for most teams.
What the release actually changes
None of this diminishes the milestone. Nathan Lambert of Interconnects calls K3 the strongest open model ever released and argues the lag between open weights and closed frontier models has compressed from roughly six to nine months down to three to five. Artificial Analysis, which is useful precisely because it runs every model on the same harness at the same settings, places K3 fourth on its Intelligence Index, behind Claude Fable 5 and GPT-5.6 Sol.
That same testing surfaces a tradeoff worth carrying: K3’s measured hallucination rate on AA-Omniscience sits around 51 percent, up from about 39 percent for K2.6, while factual accuracy rose from 33 to 46 percent. The model knows more and is also confidently wrong more often. For calibration, Artificial Analysis puts Claude Fable 5 at 54.9 percent on the same measure, so K3 is not an outlier, but it is not a model to deploy without verification either.
Keep the leaderboard claims in proportion. The roughly 2.5x scaling-efficiency improvement over K2 is Moonshot’s own figure, and the first-place finish in Arena frontend-code rankings is a preference ranking rather than a capability measure. Where this release genuinely bites is on price: every organization paying frontier rates now has a credible, inspectable, downloadable alternative to point at in a procurement conversation, and that is worth more to most businesses than the ability to run it in their own rack. Our pillar on open versus closed AI models covers where open weights sit on the wider spectrum, and why "open" is a range rather than a binary.
Frequently Asked Questions
Can I run Kimi K3 on a workstation or a high-end desktop?
No. The MXFP4 weights alone are roughly 1.56TB and all of them must be resident in fast memory, because mixture-of-experts sparsity reduces compute per token rather than the memory that has to stay loaded. For scale, a single NVIDIA DGX Spark carries 128GB of unified memory, about a twelfth of what the weights require before any cache or activation overhead. This is a server-class deployment, not a desktop one.
Why does a model that activates only 16 of 896 experts still need so much memory?
Because the router selects experts per token and its choices are not predictable in advance. Any expert may be needed for the next token, so every expert has to be loaded and immediately reachable. Sparsity saves arithmetic, not capacity. You pay full memory price for the whole parameter count while getting compute savings only on the active slice.
What hardware do I actually need?
Blackwell-class capacity. The realistic single-node entry point is eight NVIDIA B300 accelerators at 288GB each, roughly 2.3TB in total, which holds the 1.56TB of weights with room for the KV cache and activations. Sixteen B200s across two nodes also works, but needs a high-bandwidth interconnect between them rather than two separate servers. Note that the previous generation’s standard inference node, an 8-way H200 at roughly 1,128GB, cannot hold these weights at all.
What does it cost to self-host compared with using the API?
The honest answer is that it depends on hardware rates that are moving quickly, so the durable version is the formula: cost per million output tokens equals the node’s hourly rate divided by sustained output tokens per second, divided by 3,600, times a million. K3’s API is $15 per million output tokens and $3 per million input on a cache miss. Crossover estimates for models in this class land in the low tens of thousands of dollars per month in API spend, so unless your usage is large and continuous, the API is cheaper.
Will more aggressive quantisation make this affordable?
Not meaningfully. Moonshot ships at MXFP4 and applied quantisation-aware training from the supervised fine-tuning stage onward rather than compressing a finished checkpoint, so the model was trained to tolerate these numerics and the compression lever has largely been pulled already. Community re-quantisations and pruned variants will shrink things somewhat, but going below the precision the authors ship trades quality in ways that are rarely measured honestly. Quantisation decides whether you need one node or several, not whether you need a node at all.
Does self-hosting give the same quality as the hosted API?
It should, if Moonshot’s statement holds. The company’s co-founders said in a July 27 AMA that the published MXFP4 weights use the same quantisation as its production API, which means the self-hosted artefact is the served artefact rather than a degraded public version. That is a meaningful and unusually transparent commitment, though as with any vendor claim it is worth validating on your own workloads.
Who should genuinely self-host Kimi K3?
Organizations with data-sovereignty requirements that rule out sending inputs to a third-party API, teams whose differentiation depends on fine-tuning or inspecting the model directly, and operations running continuous high-volume inference where dedicated capacity beats metered pricing. For everyone else, use a hosted endpoint and treat the release as pricing leverage rather than an infrastructure project.
How big is the download, exactly?
The published MXFP4 weight shards come to roughly 1.56TB, which is about 1.42TiB in binary units. Much of the confusion in secondhand reporting comes from those two numbers describing the same quantity, and from community re-quantisations and pruned variants circulating under similar names. The same model at BF16 would be around 5.6TB. Check the Hugging Face repository directly and confirm which artefact you are pulling.
Correction, 28 July 2026: an earlier version of this article gave the MXFP4 download as roughly 594GB and identified an 8-way H200 node as sufficient hardware. The published weights are approximately 1.56TB, which an 8-way H200 node cannot hold. The memory-floor, hardware, and cost sections have been corrected, and the discussion of the KV cache has been revised to reflect that K3’s Kimi Delta Attention layers keep the cache small at long context rather than large. The correction raises the hardware floor rather than lowering it.
