“Open model” can mean anything from a downloadable checkpoint to a system whose training code and data recipe can be inspected. That difference matters before a company places sensitive data on the machine, modifies the model or builds a commercial service around it.
The short answer is this: open-weight means you can obtain the learned parameters, but it does not by itself promise an open licence, disclosed training data or reproducible training process. Open Source AI is a stricter claim. The Open Source Initiative (OSI) says the system must give people the freedom to use, study, modify and share it, supported by the preferred form for making changes.
For a buyer, “can we download it?” is only the first question. The exact licence, model card, acceptable-use terms, runtime support and hardware requirement decide whether it belongs on a local workstation or production server.
The parts of a local AI model
A language model download is not a complete application. It normally sits inside a stack with several separate parts:
| Part | What it does | What to check | |---|---|---| | Architecture | Defines how the neural network is arranged | Does the chosen runtime support it? | | Weights or parameters | Hold the numerical values learned during training | What precision and file format are supplied? | | Tokenizer | Converts text into tokens the model processes | Is it the correct tokenizer for this checkpoint? | | Model card | Describes intended use, limits, training and evaluation information | Are the claims specific enough for your use case? | | Licence and policies | Set the legal conditions for use, modification and redistribution | Is commercial use permitted? Are there use restrictions? | | Runtime | Loads the model and generates outputs | Does it support the GPU, operating system and model format? | | Serving layer | Exposes the model to users or applications | How will access, logging, quotas and updates work? |
The weights are the expensive numerical component that usually drives memory capacity. They do not tell you where the training data came from or how the finished system should be operated.
What open-weight means
An open-weight model makes its trained parameters available for download. Depending on the release, you may be able to run it locally, inspect its architecture, fine-tune it, convert it to another format or serve it behind your own API.
OpenAI, for example, describes gpt-oss-20b and gpt-oss-120b as open-weight models. Their weights are available under Apache 2.0 alongside an OpenAI usage policy. Qwen describes the Qwen3 family as open-weight and releases its listed models under Apache 2.0. Meta calls Llama 4 Scout and Maverick open-weight, but distributes them under its own model licence.
Those examples show why the label is only a starting point. Two open-weight releases can give the operator different rights and duties.
Before adopting one, record:
- the exact model and version;
- the model licence and any linked use policy;
- whether commercial use, redistribution and derivative models are allowed;
- attribution or notice requirements;
- geographic, scale or field-of-use restrictions;
- the source of the checkpoint and its file hash;
- who will monitor later security, licence and model updates.
This is operational guidance, not legal advice. Where licence terms affect a commercial product, ask a qualified legal adviser to read the actual documents rather than relying on a model catalogue label.
What Open Source AI means
The OSI’s Open Source AI Definition 1.0 applies the familiar freedoms to AI systems: use, study, modify and share. It also says the preferred form for modification needs more than final weights. It includes sufficient training-data information, the code used to train and run the system, and the model parameters under suitable terms.
That is why “source-available”, “open model” and “open-weight” should not be treated as synonyms for Open Source AI. A provider can release useful weights without publishing enough of the training process for another skilled team to rebuild a substantially equivalent system.
This distinction does not make an open-weight model a poor choice. It makes the choice more precise. Many organisations want control over inference, fine-tuning and data location, not the ability to reproduce pre-training from scratch.
What local means
“Local” means the inference process runs on infrastructure you control rather than sending every prompt to the model provider’s hosted API. The machine may sit under a desk, in your server room or in a rack operated on your behalf.
Local deployment can give you:
- control over where prompts, retrieved documents and outputs are processed;
- a stable model version while you test updates separately;
- the option to place the service on a private network;
- direct access to runtime settings, quantisation and fine-tuning;
- predictable capacity when the system is reserved for your team.
It also transfers work to you. Someone must patch the operating system and runtime, control access, scan downloaded files, evaluate model behaviour, retain or delete logs, back up fine-tunes and respond when an application starts producing unsafe or incorrect output.
Owning the weights does not remove privacy or security risk. It changes who is responsible for it.
Open-weight does not mean small
Some open-weight models fit on a single workstation GPU. Others need several accelerators, large unified-memory systems or distributed servers.
OpenAI says its natively quantised gpt-oss-20b can run within 16 GB of memory, while gpt-oss-120b is designed to fit within 80 GB. Qwen3 ranges from dense 0.6B models to a 235B mixture-of-experts model. DeepSeek-V3 has 671B main-model parameters, of which 37B are active for each token. Kimi K2 has one trillion total parameters and 32B active parameters.
The next article in this series explains why total and active parameters answer different questions. For hardware sizing, start with total stored weights, their precision, the context requirement and the number of simultaneous users.
A sensible first local deployment
For a first project, do not begin with the largest model your budget can load. Begin with one task and a pass/fail test.
Suppose the goal is an internal assistant that answers questions from approved engineering documents. A useful pilot would define:
1. A small set of representative questions, including questions the assistant should refuse to answer without evidence. 2. A candidate model licence that permits the intended business use. 3. The retrieval source, access rules and document update process. 4. Response-quality checks performed by people who understand the subject. 5. Time-to-first-token and generation-speed targets for the real users. 6. A memory test at the planned context length and user count. 7. Logging, feedback, model-change and rollback procedures.
A smaller model that passes those tests is a better purchase than a larger model selected from a leaderboard and never made dependable.
Which GPUMachines route fits?
A compact small-form-factor local LLM system suits private development and one-person or small-team experiments where immediate desk-side access matters. A tower GPU workstation gives more scope for discrete GPUs, local storage and expansion.
Shared services need a different design. A PCIe GPU server can host several model replicas or split independent services across GPUs. Models that must span several tightly connected accelerators may be better placed on an HGX server. If power, cooling or rack operations are the obstacle, Buy & Host keeps the hardware dedicated while GPUMachines operates it in a data-centre environment.
The hardware choice comes after the model licence, memory calculation and service target. A machine cannot turn an unsuitable licence or an unevaluated model into a production system.
Questions beginners ask
Is an open-weight model free to use commercially?
Not necessarily. Read the exact licence and every incorporated policy. “Free to download” and “permitted in our commercial product” are separate facts.
Can an open-weight model work without the internet?
Yes, once the weights, runtime and dependencies are present, many models can run on an isolated machine. The surrounding application may still call online tools unless those connections are removed or controlled.
Does local AI keep all data private automatically?
No. Local processing gives you control over the path, but logs, telemetry, retrieval systems, backups, plugins and user permissions can still expose data. Test the whole application, not only the model process.
Is open-weight the same as open source?
No. Open-weight describes access to trained parameters. The OSI definition of Open Source AI also requires the freedoms and materials needed to study, modify and share the system.
Do I need to train a local model from scratch?
Usually not. Most teams start from a released instruction-tuned checkpoint, test it as supplied, then consider retrieval, adapters or fine-tuning only when evidence shows a gap.
Sources and Further Reading
- Open Source Initiative: Open Source AI Definition 1.0
- Open Source Initiative: Open Weights
- OpenAI: gpt-oss model card
- OpenAI: Introducing gpt-oss
- Qwen Team: Qwen3
- Meta: Llama 4
The practical answer
Choose an open-weight model when local control, customisation or private deployment matters and its licence permits the work you plan to do. Call it Open Source AI only when the release meets that higher standard.
For a first project, settle the licence and task before ordering hardware. GPUMachines can then match the checkpoint, precision, context and user count to a compact system, workstation, PCIe server, HGX platform or hosted deployment.