A diffusion service can run comfortably for most of a request, then fall over during one brief allocation spike. Anyone who has watched a high-resolution image job reach the decoder, or a video request stretch its latent sequence across dozens of frames, will recognise the pattern: average memory use looks acceptable, yet the job still ends with an out-of-memory error.
That awkward gap between average use and peak demand is the problem behind Xema, a research system described in a paper submitted to arXiv on 13 July 2026. The authors examine production diffusion serving with Flux.2, CogVideoX-5B and LTX-2. Their proposal uses offline memory traces, fine-grained mitigation and precomputed execution plans to intervene only where a request would exceed the available GPU memory.
The reported results belong to the paper's authors. GPUMachines hasn't reproduced the experiments, and Xema isn't presented here as a product that buyers can order. The research still carries a useful message for infrastructure planning: a model's peak allocation, request shape and serving policy can matter as much as the nominal size of its weights.
Executive Summary
- The Xema paper treats diffusion serving as a template-aware systems problem rather than a single VRAM figure.
- Its authors report that memory pressure can be short-lived and predictable for a fixed model, resolution, frame count and denoising schedule.
- The proposed system combines targeted offloading, temporary serialisation, chunking, fusion and a static memory layout.
- For buyers, the practical question is whether workloads need a larger-memory GPU, more GPUs, lower concurrency or better serving software.
- The findings are research results from specific pipelines and hardware; they shouldn't be read as a universal performance claim.
Teams planning commercial image or video generation can compare PCIe GPU servers, HGX systems, GPU Cloud and Buy & Host with GPUMachines.
What the Paper Actually Studies
Xema targets services that support a defined menu of requests. A request template combines the model, output resolution and frame count. A text-to-image endpoint might offer a handful of supported resolutions; a video service may add several duration or frame-count options. Once those choices are fixed, the broad execution path and tensor shapes become predictable even though the prompt content changes.
That predictability lets Xema build a memory trace before real traffic arrives. The system performs a shape-only run, maps tensor lifetimes and identifies intervals where the predicted footprint crosses the GPU budget. It then plans mitigation for those intervals instead of enabling a heavy memory-saving method for every stage of every request.
The distinction matters because diffusion inference doesn't consume memory in one uniform block. The text encoder runs near the start. A Diffusion Transformer denoiser repeatedly refines latent data, and the VAE decoder converts the final latent representation into pixels. Different request templates put pressure on different stages.
According to the paper, Flux.2 carries a large weight footprint, while long CogVideoX requests can become activation-heavy. The authors report examples where a Flux.2 image request needs more than 64 GB and a CogVideoX video request exceeds 100 GB at batch size one. Those figures describe the tested setups, not a sizing table for every implementation or precision.
Four Places GPU Memory Goes
| Memory source | What creates it | Why buyers should care | | --- | --- | --- | | Model weights | Text encoder, denoiser and decoder parameters | Precision and sharding choices can decide whether the model loads at all. | | DiT activations | Intermediate tensors created during repeated denoising | Longer sequences and video frames can push peaks far beyond the weight footprint. | | VAE activations | High-resolution feature maps near pixel output | A job may fail late, after most of the expensive denoising work has finished. | | Allocator reservation | Cached blocks and fragmentation inside the runtime | Active tensors may appear to fit while reserved memory still crosses the device limit. |
The last row is easy to miss. GPU monitoring often shows used or reserved memory without explaining whether an allocation can be reused by the next tensor. A 48 GB card doesn't behave like a perfectly packed 48 GB box. Shapes arrive and disappear at different times; gaps form, and the allocator may keep blocks that don't match the next request.
The paper reports PyTorch caching-allocator utilisation between 58 and 89 per cent in its diffusion workloads. That range is wide enough to change a procurement decision. Yet it would be a mistake to apply it blindly to another stack, because allocator behaviour depends on the pipeline, tensor shapes, framework version and execution policy.
Why Coarse Memory Controls Can Be Expensive
Weight offloading moves parameters between host memory and GPU memory. Sharding divides work or weights across devices. VAE slicing reduces the amount of decoder work held at once. Each method can make an impossible request fit, but each also asks the system to do something extra: cross PCIe, communicate between GPUs, run smaller kernels or serialise work that would otherwise execute together.
The paper gives a pointed Flux.2 example. Partial block offloading was enough to fit the tested request on a 48 GB GPU, while full offloading raised per-step latency from 3.1 seconds to 8.4 seconds. The buyer lesson isn't that offloading is bad. It is that an all-or-nothing switch may recover far more memory than the request needs, then charge a latency penalty for the surplus.
Video serving makes the mismatch sharper. If activation pressure appears during a narrow DiT or VAE interval, moving model weights throughout the run may attack the wrong source. Adding more host RAM won't fix slow transfers, and adding another GPU won't guarantee good scaling if the chosen parallel method creates too much communication.
Xema's Main Idea: Treat the Peak, Not the Whole Request
The authors report that only a small fraction of allocation events sit near the phase peak in their CogVideoX measurements. Xema uses that observation in two ways.
First, it applies only enough mitigation to bring the predicted trace below the memory budget. A subset of blocks can move to host memory while the rest stay resident. Chunking can become active around a specific operator rather than across an entire phase. Temporary serialisation can reduce the batch at a peak, then restore normal concurrency afterwards.
Second, Xema limits those actions to the troublesome interval. Suppose four requests can share the GPU through most of denoising, but one VAE upsampling operation won't fit with all four active. A coarse policy might run the entire lifecycle at lower concurrency. Xema's design aims to serialise near the peak and preserve parallel work elsewhere.
That is a more useful mental model for operators than "the model needs 80 GB". The real requirement could be 45 GB for most of the job and 62 GB for a short interval, with the difference created by an activation that can be chunked. Or the weights may dominate throughout, making a larger-memory device or sharding plan the cleaner answer.
Static Memory Layout Is More Than Tidying Up
Xema also assigns predictable tensors to precomputed address ranges. Tensors whose lifetimes don't overlap can reuse the same region. The paper frames the problem as packing rectangles across address and time: tensor size determines height, lifetime determines width, and overlapping rectangles can't occupy the same address range.
Why bother? Because an offline trace isn't much use if runtime allocation produces a different reserved-memory peak. Static placement gives the planner a closer link between the trace it analysed and the memory the live request will reserve. The system keeps a margin for dynamic allocations such as CUDA contexts, NCCL buffers and runtime internals.
This design depends on regularity. It suits supported request templates with data-independent execution. A service that accepts arbitrary graphs, plug-ins or control flow would be harder to map. Operators would also need to rebuild plans when a model version, precision, framework, driver or hardware target changes.
The Planner Connects Memory to Service Levels
Memory fit is only one constraint. A production endpoint also has latency targets and throughput expectations. Xema's offline planner considers parallelism, concurrency and memory controls together, rejects configurations that exceed the memory budget or service-level objective, then chooses among the remaining candidates using profiled performance.
This avoids a familiar trap. Increasing batch size may improve arithmetic efficiency but raise activation demand. Sequence parallelism can lower per-GPU activations while adding communication. Weight sharding reduces resident parameters, yet every gather can lengthen the critical path. None of those controls has a fixed value outside a request shape.
The paper reports up to 3.7 times better SLO attainment than compared serving configurations and a reduction in planning time from 6.3 hours to 197 seconds versus grid search. Those are author-reported maxima, not GPUMachines benchmark figures. The workloads, baselines and implementation details decide how far the result travels.
Our Technical View
Xema is interesting because it points at the part of diffusion infrastructure buyers tend to learn late. VRAM capacity is a hard boundary, but the route to staying below it isn't always "buy the next GPU up".
For a stable service with known models and a controlled set of resolutions, template-specific planning makes sense. Traffic can be routed by request type, and each template can carry its own concurrency, parallelism and memory policy. In that setting, fine-grained controls could turn unusable headroom into productive capacity.
A research studio with constantly changing pipelines has a different problem. Engineers may swap checkpoints, VAEs, schedulers and extensions several times a week. Rebuilding and validating offline plans can become operational work of its own. More VRAM may cost less than the staff time needed to maintain a tight serving plan.
Hardware still sets the outer limit. Software can shave peaks, reduce fragmentation and move data, but it can't make PCIe transfers free or remove every activation. Large-memory GPUs remain the direct answer when high-resolution and long-video requests must run with predictable latency. Multi-GPU systems become sensible when model or activation footprints exceed one device and the application can carry the communication cost.
What This Means for GPU Selection
Start with the request catalogue, not the model name. Record output width and height, frame count, precision, denoising steps, expected batch, concurrency and response-time target. Run the real pipeline, because a model card's parameter count won't reveal VAE peaks or allocator reservation.
For image generation with a large weight footprint and moderate activations, a single GPU with enough VRAM may offer the simplest operating model. Offloading can make smaller cards usable for occasional work, although sustained services need careful latency testing.
Video generation often pushes activation memory harder. Longer sequences can favour larger-memory accelerators, sequence parallelism or a multi-GPU node. That node also needs CPU lanes, host RAM and a sensible device topology; otherwise transfers and collectives will take back the capacity gained by splitting the job.
Local NVMe matters for checkpoint loading and model turnover, but it doesn't replace GPU memory. System RAM helps offloading only when capacity and bandwidth are both adequate. A server with eight GPUs and a starved CPU or poor NUMA placement can still behave badly.
Configuration Paths Worth Testing
Controlled image-generation service: begin with one or more independent large-memory GPUs, local NVMe for models and enough CPU capacity for request preparation. Measure the late VAE peak before deciding that every request needs multi-GPU execution.
Long-video generation: test the full target frame count and resolution on a multi-GPU PCIe or HGX-class node. Watch DiT activation growth, communication time, host memory traffic and failed allocations. A short demo clip proves very little about a production duration.
Studio or research queue: favour flexibility over a tightly fixed plan. Several independent GPUs can let different users run different models, while a scheduler assigns jobs by VRAM requirement. GPU Cloud can absorb experiments that don't justify a permanent hardware tier.
Dedicated production platform: use Buy & Host or an on-premise server when the request mix and utilisation have settled. Preserve a test environment for new model versions so a seemingly harmless update doesn't move the memory peak past the deployed limit.
Where Xema's Approach May Be a Poor Fit
An endpoint with arbitrary user-supplied graphs, plug-ins or changing execution paths won't offer the same template stability. Nor will a small internal service necessarily earn back the engineering effort. If jobs run a few times each day, accepting lower concurrency or renting a larger GPU can be the cheaper choice.
Teams should also avoid treating lossless memory control as a substitute for output-quality review. Quantisation, lower resolution, VAE tiling and reduced frame count may save more memory, but they can change the result. Xema focuses on system-level controls; buyers still need a separate quality budget when lossy methods enter the stack.
Questions to Answer Before Buying Hardware
- Which exact model, precision, resolution and frame counts will enter production?
- Does peak memory come from weights, DiT activations, VAE activations or allocator reservation?
- How much latency does offloading, slicing or sharding add on the intended server?
- Can requests be grouped into stable templates with separate execution plans?
- Is failure recovery acceptable after a long video job has already consumed minutes of GPU time?
- Would several independent GPUs serve the queue better than one tightly coupled model instance?
FAQ
Does Xema mean I can buy a GPU with less VRAM?
Possibly for a controlled workload, but the paper doesn't support a blanket reduction. Fine-grained mitigation trades data movement or extra work for lower peaks. Test the exact request catalogue and latency target before reducing the hardware specification.
Is offloading enough for high-resolution diffusion?
Offloading primarily reduces resident weights. It may help a weight-heavy image model, while an activation-heavy video request can still fail. Host RAM capacity, PCIe bandwidth and overlap with computation all affect the result.
Would HGX always beat a PCIe GPU server for video generation?
No. HGX becomes attractive when requests need tightly coupled multi-GPU execution and can benefit from its scale-up fabric. Independent image jobs or mixed research queues may use PCIe GPUs more efficiently and at lower platform cost.
How should a team measure diffusion memory?
Capture active and reserved GPU memory across the full pipeline, including text encoding, every denoising step and VAE decode. Repeat at production resolution, duration, precision and concurrency. A single end-of-run figure hides the peak.
Can GPUMachines help plan a diffusion-serving system?
GPUMachines can review GPU memory, CPU platform, host RAM, NVMe, device topology, networking, rack power, cooling and deployment route. Software-specific acceptance testing should use the buyer's real model and request set.
Sources and Further Reading
- Xema: Efficient Diffusion Serving through Fine-Grained Memory Management and Auto-Configuration
- Xema experimental HTML paper
- NVIDIA CUDA C++ Best Practices Guide
Verdict
Xema makes a persuasive case for measuring diffusion workloads over time instead of reducing them to one VRAM number. Its strongest idea is simple: if a request crosses the memory limit for only a short interval, don't slow every other interval to solve it.
Production teams with fixed models and predictable output templates should pay attention. Research groups with rapidly changing pipelines may still prefer larger memory margins and a simpler queue. GPUMachines can help compare those routes across GPU servers, HGX platforms and hosted capacity before hardware is committed.