When a model page says “1T parameters”, it is describing one trillion learned numerical values. It is not saying that every answer uses a trillion calculations, that the model needs exactly one terabyte of memory or that it will outperform a smaller model on your task.
That distinction is especially important for mixture-of-experts models. Kimi K2, for example, is described as a one-trillion-parameter model with 32 billion active parameters. The full checkpoint is very large, but each token is routed through only part of it.
The buyer’s translation is simple: total parameters help estimate how much model data must be stored and loaded; active parameters help explain how much of an MoE model works on each token. Neither number is a deployment plan by itself.
B, M and T in model names
Model names often abbreviate the parameter count:
| Label | Number | Written out | |---|---:|---:| | 1M | 1,000,000 | one million | | 1B | 1,000,000,000 | one billion | | 1T | 1,000,000,000,000 | one trillion |
A 7B model has roughly seven billion parameters. A 70B model has roughly ten times as many. A 1T model has roughly 143 times as many as a 7B model.
The marketed figure may be rounded. Model cards sometimes separate the main model from auxiliary modules, embeddings or other weights, so use the official technical table when memory margins are tight.
What a parameter is
During training, a neural network adjusts many numerical values so that its output better matches the training objective. Those values are parameters. They are organised into matrices and other tensors rather than stored as a human-readable database of sentences.
A parameter does not hold one fact. The model’s behaviour comes from patterns distributed across many values and from the way the architecture processes tokens.
This is why parameter count alone cannot tell you whether a model is accurate, safe, current or suitable for engineering, medicine, law or code. Training data, architecture, post-training, tokenizer, context handling and evaluation all matter.
Dense models and MoE models
A dense transformer uses most of its model parameters for each token. If a dense model has 70B parameters, the main computation path broadly involves that 70B-scale network each time it generates another token.
A mixture-of-experts (MoE) model contains many expert blocks and a routing mechanism. The router selects a subset of experts for each token. This gives the model a large total parameter pool without using every expert on every step.
Current examples make the notation easier to read:
| Model | Total parameters | Active parameters per token | Architecture note | |---|---:|---:|---| | Qwen3-30B-A3B | 30B | 3B | MoE; the A3B suffix signals about 3B active | | Qwen3-235B-A22B | 235B | 22B | MoE | | gpt-oss-20b | 21B | 3.6B | MoE | | gpt-oss-120b | 117B | 5.1B | MoE | | Llama 4 Scout | 109B | 17B | MoE with 16 experts | | Llama 4 Maverick | 400B | 17B | MoE with 128 experts | | DeepSeek-V3 | 671B main model | 37B | MoE | | Kimi K2 | 1T | 32B | MoE |
These figures come from the model publishers. They describe architecture, not measured speed on a particular GPUMachines system.
Why active parameters do not solve the memory problem
It is tempting to read “1T total, 32B active” and size the machine like a 32B dense model. That is usually wrong.
The inactive experts still exist in the checkpoint. A straightforward deployment must keep the full set of model weights somewhere: accelerator memory, coherent system memory, host RAM or storage used for offload. Only the selected experts participate in the token’s main compute path, but the runtime still needs access to whichever experts the router chooses next.
Offloading experts to slower memory can make a model load on a smaller GPU pool. It can also make token generation depend on CPU memory bandwidth, PCIe transfers or storage access. “Runs” may then mean “produces an answer eventually”, not “serves people at an acceptable rate”.
Turning parameters into rough weight memory
The first-pass calculation is:
parameters × bits per weight ÷ 8 = bytes for weights
Using decimal gigabytes and ignoring format overhead:
| Model size | 16-bit weights | 8-bit weights | 4-bit weights | |---|---:|---:|---:| | 7B | 14 GB | 7 GB | 3.5 GB | | 32B | 64 GB | 32 GB | 16 GB | | 70B | 140 GB | 70 GB | 35 GB | | 120B | 240 GB | 120 GB | 60 GB | | 1T | 2,000 GB | 1,000 GB | 500 GB |
The table is arithmetic, not a promise. Real files contain quantisation scales, metadata and tensors that may use different precisions. The runtime also needs a KV cache, compute buffers and other allocations.
OpenAI’s gpt-oss releases are a useful reminder. The 120B name refers to a model with 117B total parameters. OpenAI supplies it natively quantised in MXFP4 and says it fits within 80 GB, rather than the simple 58.5 GB implied by exactly four bits per parameter. The extra margin is not wasted; it reflects the actual format and runtime needs.
For the wider precision decision, read Quantised Models vs Full Precision Models. The next guide in this series builds a complete memory budget.
Does a larger model give better answers?
Sometimes, but not as a universal rule. Parameter count is one design variable among several.
A smaller, newer model can beat an older, larger model on a target task because its training data, architecture or post-training is better suited. A code model may outperform a larger general model on code. A compact multilingual model may be the better choice for a narrow language set. Retrieval can improve factual grounding without replacing the base model.
The right comparison uses your prompts, your documents and an agreed scoring method. Public benchmarks are useful for shortlisting, but they do not reproduce your users, retrieval pipeline, safety rules or latency target.
What parameter count does not tell you
It does not tell you:
- the maximum useful context in your application;
- memory consumed by the KV cache and concurrent requests;
- tokens per second on your runtime and hardware;
- whether the model supports images, audio or tool calls;
- the licence conditions;
- the quality of a particular quantisation;
- how much memory is needed for fine-tuning;
- whether the model’s answers are acceptable for your task.
Training also needs much more memory than inference because it stores gradients, optimiser states and activations. A machine that serves a model comfortably may not be able to fine-tune the same model with the method you selected.
What a 1T model means for GPUMachines hardware
A one-trillion-parameter checkpoint is not a normal single-GPU workstation workload. Four-bit weight arithmetic starts at about 500 GB before runtime allocations.
The W775-V10-L01 GB300 workstation is listed with 748 GB of coherent memory. That puts some heavily quantised very-large-model experiments within the capacity conversation, subject to runtime support and measured performance. It does not guarantee that a 1T checkpoint, long context and useful concurrency will all fit or run at the required speed.
For a shared service, PCIe GPU servers can divide replicas or selected model-parallel workloads across several cards. HGX servers are the stronger starting point when one model depends on high-bandwidth GPU-to-GPU communication. A hosted pilot can also reveal whether the workload justifies purchasing the full memory pool.
Do not buy 500 GB of nominal model capacity from the formula alone. Obtain the exact checkpoint, runtime and quantisation, then measure peak allocated memory, prompt processing, generation speed and concurrent requests.
A beginner’s model-reading checklist
When you see a model name, translate it in this order:
1. Is it dense or MoE? 2. What are the exact total and active parameter counts? 3. Which checkpoint is this: base, instruction-tuned, reasoning, vision or a fine-tune? 4. What weight format and precision will be deployed? 5. What context length will the application actually use? 6. How many requests must run at once? 7. Which runtime supports this exact version? 8. Does the licence permit the intended use? 9. What evidence will decide whether its output is good enough?
That list turns an impressive model name into a testable infrastructure requirement.
Questions beginners ask
Does 1T mean one terabyte?
No. It means one trillion parameters. The memory depends on the number of bits used for each weight, the checkpoint format and runtime allocations. At 8 bits per weight, the weight arithmetic is about one terabyte; at 4 bits it is about 500 GB.
What does A22B mean in Qwen3-235B-A22B?
The model has 235B total parameters and about 22B activated for each token. The A value is an active-parameter figure, not the complete checkpoint size.
Is a 32B-active MoE as fast as a 32B dense model?
Not automatically. Routing, expert placement, memory bandwidth, quantisation, kernels and communication all affect speed. Active count helps explain compute, but only a benchmark on the planned stack answers the service question.
Can I split a large model across several GPUs?
Yes, if the model and runtime support tensor, pipeline, expert or other forms of parallelism. The GPUs’ interconnect and the placement strategy can then affect latency and throughput.
Should I always choose the model with the most parameters that fits?
No. Choose the smallest model that passes the task, quality, context and response-time tests. Spare memory can support longer context, more users or a simpler operating margin.
Sources and Further Reading
- OpenAI: Introducing gpt-oss
- Qwen Team: Qwen3
- DeepSeek: DeepSeek-V3 repository and model summary
- Meta: Llama 4
- Moonshot AI: Kimi K2 model card
- Hugging Face Transformers: Optimising LLMs for speed and memory
The practical answer
Read total parameters as a clue to checkpoint size. Read active parameters as a clue to the per-token path in an MoE model. Then ignore the urge to rank models by either number.
The useful purchase question is whether the exact checkpoint, precision, context and user load fit a machine that delivers acceptable responses. GPUMachines can size that test against a workstation, coherent-memory system, PCIe server, HGX platform or hosted deployment.