Key figures

Assessment

At 4-bit, 671B parameters need about 336 GB for weights alone and roughly 419 GB once runtime overhead is included, which is about 9 professional cards. The first gate to fail is hardware feasible. Needs about 419 GB of accelerator memory at 4-bit, roughly 9 professional cards. Available: 200 GB. Mixture-of-experts models pay memory for TOTAL parameters because every expert must be resident, even though only a few activate per token. Self-host route: not available. Hosted route: available.

Questions for the founder

1. Is the client asking because of a headline, or because they have a workload the current model cannot serve?
2. Who patches, monitors and re-benchmarks this model once it is self-hosted, and is that billable?
3. Has anyone read the licence past the word "open"?
4. What data class will actually reach this model in month three, not in the pilot?
5. If the hosted route is used instead, which jurisdiction processes the request?

Assumptions & method

• Memory covers WEIGHTS ONLY at 4-bit. KV cache, activations, batching and long context add materially on top.
• Runtime overhead multiplier of 1.25 applied to weight memory *
• Card count assumes 48 GB professional cards *
• Mixture-of-experts sized on TOTAL parameters, because every expert must be resident.