An INT8 model file does not prove that an NVIDIA B300 deployment will run the intended INT8 kernel. That distinction can surface painfully late: the model loads, memory is committed, and the first real forward pass discovers that the serving engine has no valid path for the target GPU.
A new source-level audit by Teng-Ruei Chen follows INT8 W8A8 support through the Blackwell Ultra stack. It examines NVIDIA's published figures, the PTX instruction set, CUTLASS, vLLM and SGLang. The paper's conclusion is narrow but commercially important: for the versions and code paths inspected, default INT8 deployment support on B300 breaks at several layers. The paper does not benchmark B300, prove that every INT8 method fails, or show that Blackwell Ultra is a poor inference platform.
The buyer's answer is therefore conditional. B300 can still be the right choice for large-model inference, reasoning, training and mixed-precision work. It should not be ordered around an INT8 capacity or cost model until the exact model, quantisation method, container, CUDA build, serving engine and kernel path have completed a cold-start acceptance test on B300. A hardware feature and a production-ready software route are different deliverables.
The purchasing answer
If your production plan depends on W8A8 INT8, treat the software path as a procurement gate rather than a post-delivery optimisation task. The acceptance test should use the image you expect to deploy, not a vendor demonstration with a different model format or backend. It should prove model load, first forward pass, sustained generation, numerical quality, concurrency behaviour and restart behaviour.
That advice is stricter than checking whether a framework lists INT8 among its supported quantisation methods. A method can appear in a top-level compatibility table while the selected architecture, shape, model class or backend reaches a different implementation. The paper illustrates this problem by tracing a specific path down to instructions and generated kernels.
Buyers without a fixed INT8 requirement should avoid turning the finding into a general objection to B300. Blackwell Ultra places considerable emphasis on floating-point formats used by modern AI workloads. An FP8, FP4, BF16 or mixed-format deployment may have a well-supported path even when a particular W8A8 implementation does not. The correct comparison starts with the model and service objective, then chooses precision and hardware together.
Current B300 platforms can be compared in the GPUMachines HGX server range. For a multi-node project, the GPU cluster configurator helps establish node count, scale-out fabric and rack requirements. Neither step replaces the runtime acceptance test, but both make its hardware boundary explicit.
What the new audit found
The paper, submitted on 12 August 2026, describes an ISA- and source-level review rather than a performance study. Its author pins the examined sources to commits or digests and records access dates. That is useful because framework support changes quickly, but it also means the conclusions belong to a particular software snapshot.
The audit starts from the published dense-compute figures. It calculates a roughly 30:1 ratio between FP8 and INT8 tensor-core throughput for B300, compared with a 1:1 relationship for the H200 and B200 figures it examined. That ratio is not a measured application speedup or slowdown. It is evidence that the two data types do not occupy the same position in the published Blackwell Ultra compute profile.
The author then inspects four implementation layers:
- The PTX ISA does not expose the fifth-generation integer tensor-core route examined by the paper on the B300 architecture target, leaving the legacy integer matrix-multiply path as the architecturally legal route identified in the audit.
- The inspected CUTLASS generator logic skips the audited INT8 UMMA generation path when building for the relevant B300 target while generating FP8 support.
- The inspected vLLM route has no B300 INT8 GEMM for the path tested by the author and raises an error at the first forward pass after model loading.
- The inspected SGLang ahead-of-time INT8 implementation stops at an earlier architecture target, while its floating-point Blackwell support has moved further.
These are claims from the paper's source audit, not findings reproduced by GPUMachines. They also do not mean that every operation involving eight-bit integers is absent from B300. INT8 weights, INT8 activations, quantised communication, dequantisation into another compute format and an actual W8A8 tensor-core GEMM are different things. A framework saying that it accepts an INT8 model does not tell the buyer which one occurs on the target accelerator.
The paper documents a possible vLLM escape route through a just-in-time Triton backend. That is technically relevant, but it should not be mistaken for evidence of production readiness. A fallback may compile, run and return correct outputs while still having different performance, warm-up, caching, observability or support characteristics from the expected kernel. The paper explicitly contains no performance measurements.
Why the whole stack decides whether a format exists
Quantisation is often discussed as a model attribute: W8A8, weight-only INT8, FP8 or NVFP4. Deployment turns it into a chain of contracts.
The model has to encode weights and scaling information in a form the loader understands. The framework must select a compatible quantisation method for the architecture. The operation dispatcher must find a kernel for the model's shapes and layer types. That kernel must be legal for the GPU target and compiled with a suitable CUDA toolchain. The serving engine must then preserve correctness through batching, tensor parallelism, CUDA graphs and any fused operations used in production.
A gap at any link changes the result. The runtime may reject the model immediately, fail only when the first GEMM is dispatched, fall back to a slower implementation, dequantise into another format, or silently select a different backend. Those outcomes have very different capacity and support consequences.
NVIDIA's current CUTLASS repository lists B300 as compute capability 10.3 with CUDA 13.0 as the minimum toolkit for that target. It also warns that architecture-accelerated features can require the exact architecture target rather than relying on ordinary forward compatibility. That gives procurement teams another reason to pin the toolchain: a container built for B200 or a generic Blackwell target is not automatically evidence for B300.
The framework layer is equally fluid. vLLM and SGLang are active projects with frequent releases, new kernels and backend changes. By the time a server arrives, a limitation described in a paper may have been fixed, moved or replaced by a different path. Conversely, a broad compatibility statement can conceal a shape-specific or model-specific failure. The only durable control is a reproducible acceptance test tied to exact versions.
INT8 is not one deployment path
Before evaluating hardware, write down what INT8 means for the workload. At least five different requests can hide behind that label:
- weights stored as INT8 but converted before multiplication;
- W8A8 GEMM with integer weights and activations;
- weight-only quantisation paired with floating-point activations;
- INT8 communication between ranks while compute uses another format;
- an INT8 checkpoint accepted by the loader but executed through a fallback backend.
Those variants do not have the same memory saving, tensor-core path, accuracy risk or throughput. They also do not create the same reason to buy B300. A team focused on reducing model memory might accept a conversion step that would be unsuitable for a latency-sensitive serving operator. A team migrating an existing W8A8 fleet may care more about behavioural continuity than the lowest theoretical precision available on newer hardware.
Model architecture matters as well. Dense transformer layers, mixture-of-experts blocks, attention projections and embedding operations can reach different kernels. Tensor-parallel and expert-parallel layouts add collective communication and shape changes. A small single-GPU smoke test may therefore pass while the production topology reaches an untested route.
This is why GPUMachines would ask for the model identifier or model family, quantisation recipe, expected context length, concurrency target, tensor-parallel degree and serving engine before treating B300 plus INT8 as a complete requirement.
A practical B300 acceptance test
The test needs to fail early enough to influence procurement. Running it after a full cluster is installed converts a compatibility question into an incident.
1. Freeze the deployment candidate
Record the GPU target, firmware, driver, CUDA toolkit, container digest, PyTorch build, serving-engine commit or release, quantisation library and model revision. Store the launch command and environment variables. A test against latest cannot be repeated when a dependency changes.
The current CUTLASS compatibility information makes the CUDA target especially important for B300. Confirm that the build identifies the accelerator as compute capability 10.3 and that architecture-specific code was compiled for the intended target.
2. Test the real model format
Use the same checkpoint conversion and scaling metadata planned for production. Do not substitute an FP8 model merely because it is easier to launch, then infer that W8A8 will behave similarly. Verify hashes for converted artefacts so that a later test is not using a subtly different model.
Run at least one representative dense model and, where relevant, the actual mixture-of-experts model. Kernel selection can change with layer type and matrix shape.
3. Start from a cold environment
Clear any compiled-kernel cache according to the framework's supported procedure, start a fresh container and capture the complete log. This exposes just-in-time compilation, missing binaries and first-use failures that a warm development node can hide.
Success means more than reaching a healthy HTTP endpoint. Send a real request and wait for the first forward pass. The paper's vLLM finding is important precisely because model loading can complete before the failing operation is reached.
4. Prove which kernel ran
Collect framework dispatch logs and a GPU trace suitable for identifying the selected GEMM path. Confirm whether the operation is native to the intended target, a Triton JIT kernel, a legacy instruction route or a conversion into another format. Avoid naming a trace result native INT8 unless the instruction and data path justify that description.
The paper also warns about a profiler false negative in the obvious method it considered. That is a reminder to validate the tracing method itself against a known kernel rather than assuming absence from one counter proves absence from the hardware.
5. Check correctness before throughput
Compare outputs against an agreed higher-precision baseline using a task-relevant evaluation set. Quantisation quality should be measured on the buyer's workload, not accepted from a generic perplexity result. Include long contexts, tool calls, structured output and any domain-specific evaluation that can influence production risk.
Only after correctness passes should the team measure tokens per second, time to first token, inter-token latency, request completion time, memory usage and power under representative concurrency. A fallback that produces text is not automatically an acceptable capacity result.
6. Exercise production topology and recovery
Repeat the test at the intended tensor-parallel or expert-parallel width. Fill concurrency to the planned operating range. Restart replicas, rebuild caches and roll the exact container onto a clean node. Capture whether behaviour changes after warm-up or under CUDA graphs.
For a cluster purchase, make the passing software bill of materials part of the acceptance record. A future upgrade can then be tested against a known baseline instead of rediscovering the original path.
How this changes B300, B200 and H200 selection
The paper does not establish that H200 or B200 will outperform B300 for a given INT8 service. It does give a buyer reason to test an existing W8A8 stack on each candidate rather than assuming generational progression preserves the same route.
H200 can remain attractive when a mature Hopper software path, large HBM capacity and deployment continuity matter more than adopting the newest accelerator. B200 may suit teams that want Blackwell while their chosen kernels and frameworks are already proven there. B300 becomes compelling when its memory, Blackwell Ultra platform capabilities and supported low-precision paths match the workload. None of those judgements can be made from an INT8 label alone.
The comparison also needs to include migration cost. Re-quantising a model, changing a serving backend, building custom kernels and repeating quality evaluation consume engineering time. If B300 requires a different precision route, its hardware advantage should be assessed together with that work. If the service can move cleanly to FP8 or FP4 and retain required quality, the reported INT8 gap may be largely irrelevant. If regulatory, model or compatibility constraints fix the deployment to W8A8, it may decide the purchase.
Buyers considering rack-scale Blackwell Ultra can read the GPUMachines GB300 NVL72 buying guide. It explains the wider facility and operating-model decision. The software warning in this article still applies: a larger scale-up domain does not repair an unverified model path.
When B300 is still the right system
B300 remains a strong candidate when representative testing confirms the chosen precision and framework, or when the workload is not dependent on the audited INT8 route. Large-model inference, reasoning, training and post-training can use several formats and kernel families. Buyers should select among them through measured quality, latency, throughput and memory behaviour.
It can also be the right strategic platform when the team owns enough of the software stack to qualify new kernels responsibly. An organisation with GPU performance engineers may be comfortable adopting a JIT backend or contributing support upstream. A smaller serving team seeking a stable appliance-like deployment may reasonably prefer a more mature route.
The strongest B300 purchase case has three pieces of evidence: the model fits the memory and scale-up design; the actual software image passes on the target accelerator; and the facility or hosting plan can operate the platform at useful utilisation. Missing any one of those weakens the business case.
Who should not buy around this assumption
Do not order B300 capacity solely because a model card says INT8. The model card describes an artefact, not necessarily the target kernel.
Do not use theoretical INT8 throughput to build a fleet-size or return-on-investment calculation until the runtime has demonstrated the intended path and measured application performance. The paper supplies no benchmark data, so it cannot support such arithmetic.
Do not choose B300 as a way to avoid software migration work. New accelerator generations often require newer drivers, toolkits, containers and kernels. If the organisation cannot pin, test and maintain that chain, a mature H200 or proven B200 deployment may create less operational risk.
Do not buy an on-premise cluster merely to answer a compatibility question. A short evaluation on hosted target hardware can be cheaper and faster. GPUMachines Buy & Host is also relevant when dedicated infrastructure is required but rack power, cooling and operations should sit with a hosting provider.
What GPUMachines can establish before configuration
GPUMachines can frame the hardware and deployment decision around a buyer-supplied model and runtime requirement. The useful inputs are not limited to GPU quantity. They include precision, model revision, serving engine, parallelism, context length, concurrency, latency objective, expected utilisation, storage path, networking and deployment location.
From there, the project can compare HGX B300, B200, H200 and hosted validation routes without pretending that a data sheet resolves the software question. Where the workload depends on a changing open-source stack, the configuration record should name the validated build and its evidence boundary.
This is also the point to separate sourcing from software assurance. GPUMachines can source and configure suitable GPU infrastructure, but final compatibility depends on the complete customer workload and software image. A useful configuration review exposes that dependency and turns it into an acceptance item rather than a vague guarantee.
FAQ
Does NVIDIA B300 support INT8?
NVIDIA's published platform information includes integer capability, but that does not guarantee that every W8A8 model has an optimised production path through every framework. The new paper reports gaps in the specific PTX, CUTLASS, vLLM and SGLang routes it audited. Treat support as a whole-stack question tied to versions and kernels.
Does the paper prove that INT8 is slow on B300?
No. The paper explicitly reports no performance measurements. It examines whether selected software and instruction paths are available. A JIT, legacy or alternative backend would need separate correctness and performance testing.
Can a vLLM workaround make INT8 deployable?
The paper describes rerouting the audited vLLM path to a JIT-compiled Triton backend through an environment setting. That may provide a functional route for the tested case. It does not establish production throughput, latency, stability or support for another model. Test it as a separate deployment candidate.
Should an existing H200 INT8 service move to B300?
Only after the exact service passes on B300 and the measured gain justifies migration. Include model conversion, quality evaluation, container changes and operational retraining in the comparison. H200 may remain the lower-risk choice when continuity matters most.
Is FP8 a better choice than INT8 on Blackwell Ultra?
It may be for some AI workloads because Blackwell software development strongly supports floating-point low-precision formats. It is not automatically better for every model. Compare output quality, memory, latency and throughput using the real service and target hardware.
What should be written into a B300 acceptance test?
Name the model revision, quantisation recipe, container digest, driver, CUDA version, framework build, launch command and topology. Require cold model load, first forward pass, kernel-path evidence, quality checks, representative concurrency, restart behaviour and recorded performance.
Can GPUMachines help validate the configuration before purchase?
GPUMachines can review the intended model, precision, runtime, server platform, networking, storage and hosting route as part of configuration planning. Workload-specific software compatibility should be demonstrated on the final candidate build and recorded as an acceptance condition.
Verdict
The B300 INT8 audit identifies a procurement problem that hardware tables cannot solve. A quantised model is deployable only when the instruction target, kernel library and serving engine meet on a working path. In the software snapshot examined by the paper, that path breaks for the audited default W8A8 routes, sometimes after the expensive step of loading the model.
That is a reason to test B300 properly, not a reason to dismiss it. Buyers whose service depends on INT8 should make cold-start, first-forward, kernel and quality evidence a condition of purchase. Buyers free to use FP8, FP4 or another proven route should compare those options on their own workload.
Use the GPUMachines GPU cluster configurator to define the candidate platform, fabric and deployment boundary, then bring the exact model and software image into the configuration review. The passing software stack should be part of what gets accepted, not an assumption left for installation week.
Sources and Further Reading
- Spec Sheets Are Not Kernels: An ISA- and Source-Level Audit of INT8 Availability on NVIDIA Blackwell Ultra, Teng-Ruei Chen, submitted 12 August 2026. This is the primary research source. Its findings are source-level observations and it reports no performance measurements.
- NVIDIA CUTLASS repository, including current architecture and CUDA-toolkit compatibility information for B300.
- NVIDIA PTX ISA documentation, the official instruction-set reference used to understand architecture targets and legal instruction paths.
- vLLM repository, the upstream serving-engine source. Support changes frequently, so deployments should pin and record the tested revision.
- SGLang repository, the upstream serving-framework source. Its current quantisation options should be checked against the exact architecture and backend.