GPU and VRAM Requirements for Open AI Music Models

Every open music model, what it actually needs to run, and which figures are documented by the project versus estimated. Plus what fits on 8, 16 and 24 GB, and how to measure your own workload.

Last updated 2026-09-12

"Can I run this on my GPU?" is the first question about any open music model, and the hardest to get a straight answer to. Project READMEs quote different things — some give training minimums, some inference, some assume fp16, some don't say. This page collects the figures in one table, and is explicit about which are documented by the project and which are estimates.

How to read VRAM numbers

Three things consume GPU memory, and only one of them is fixed:

• Weights. Predictable: roughly 4 bytes per parameter at fp32, 2 at fp16/bf16. A 1.5B model is about 6 GB at fp32, 3 GB at fp16.
• Activations. Scales with how much audio you generate at once — clip duration for diffusion models, sequence length for autoregressive ones. This is usually the part that blows up.
• Batch size. Multiplies the activations.

So a single quoted number is always shorthand for "this model, this precision, a reasonable duration, batch of one". Doubling your clip length or batch size can push a model that fit comfortably into an out-of-memory error, which is why the same model gets reported as needing anywhere from 8 to 24 GB depending on who is asking.

The matrix

Verified 12 September 2026. "Documented" means the project states the figure; "estimated" means it is derived from parameter count and typical overhead, and you should treat it as a starting point rather than a guarantee.

Two things jump out of that table. Separation is cheap and generation is expensive — Demucs on 3 GB versus YuE2 on 24 GB is an eightfold spread for tasks people mentally file together. And Apple Silicon is no longer an afterthought: ACE-Step is the only full-song model that runs properly on a Mac, while Magenta RealTime inverts the usual picture entirely — Apple Silicon is its primary target for real-time streaming, and NVIDIA GPUs are limited to offline inference.

RAVE training, by configuration

RAVE is the one model here where most people train rather than just infer, so its configs come with their own documented minimums:

What to run on what you have

CPU only, or a laptop with no discrete GPU

More than you would expect. Demucs is genuinely fine on CPU at around 1.5× track length. Bark with SUNO_OFFLOAD_CPU and small models works. Stable Audio Open Small was built for exactly this. RAVE inference runs real time. MusicGen small technically runs and will test your patience.

8 GB (RTX 3060, 4060, many laptops)

The sweet spot is better than it sounds. ACE-Step fits with its offload flags and gives you full songs with vocals. Stable Audio Open fits at fp16. Demucs is comfortable. MusicGen small is fine; medium is out.

# ACE-Step on 8 GB
acestep --torch_compile true --cpu_offload true --overlapped_decode true

# Stable Audio Open on 8 GB
pipe = StableAudioPipeline.from_pretrained(
    'stabilityai/stable-audio-open-1.0', torch_dtype=torch.float16)
pipe.enable_model_cpu_offload()

# Bark on 8 GB
import os
os.environ['SUNO_USE_SMALL_MODELS'] = 'True'

12–16 GB (RTX 4070 Ti, 3080, A4000)

MusicGen medium and melody open up, which is the biggest single quality jump in the instrumental models. Stable Audio Open runs without contortions, at longer durations and with several candidates per prompt. Bark full fits. RAVE v2 training becomes possible.

24 GB (RTX 3090, 4090, A5000)

Everything in the table except RAVE v3 training. YuE2 becomes available, and it's the main reason to want 24 GB — it has no quantised path, so this is a hard floor rather than a comfort level. MusicGen large fits. ACE-Step runs without offloading, at roughly 34× real time on a 4090.

Apple Silicon

Unified memory means the numbers above don't map directly, but support is the real constraint, not capacity:

The pattern: projects that kept their dependencies current run on MPS; projects pinned to an old torch don't. Magenta RealTime goes further and treats Apple Silicon as the target platform, using MLX rather than PyTorch.

The five levers when you run out of memory

In the order worth trying:

1. Halve the precision. fp16 or bf16 instead of fp32 roughly halves weight memory for a quality difference most people can't hear. Usually a one-line change.
2. Offload to CPU. Keep only the active stage on the GPU. Costs speed, not quality. enable_model_cpu_offload() in diffusers, --cpu_offload true in ACE-Step, SUNO_OFFLOAD_CPU in Bark.
3. Shorten the output. Activations scale with duration. Generating 4× 10 s instead of 1× 40 s often fits when the long version won't.
4. Drop the batch size to one. Obvious, and worth checking you haven't left it higher.
5. Quantise. 8-bit via bitsandbytes is the last resort — real quality cost, and not every model supports it.

Model-specific equivalents worth knowing: Demucs has --segment (window length, the direct memory dial) and the PYTORCH_NO_CUDA_MEMORY_CACHING=1 environment variable; MusicGen has two_step_cfg=True, which halves throughput to save memory.

Measure it yourself

Every number on this page is someone else's. Yours is one function call away:

import torch

torch.cuda.reset_peak_memory_stats()

# ... load the model and generate exactly what you actually plan to generate ...

peak = torch.cuda.max_memory_allocated() / 1024**3
reserved = torch.cuda.max_memory_reserved() / 1024**3
print(f'peak allocated: {peak:.2f} GB')
print(f'peak reserved:  {reserved:.2f} GB')   # what the OOM error cares about

Run that around your actual workload — your duration, your batch size, your precision — and you have a figure that applies to you rather than to a benchmark. max_memory_allocated reports the peak, which is what determines whether you OOM, not the steady state.

If you want to see where it goes:

# weights alone, before any generation
torch.cuda.reset_peak_memory_stats()
model = load_your_model()
print(f'weights: {torch.cuda.max_memory_allocated() / 1024**3:.2f} GB')

# then the cost of one generation on top
torch.cuda.reset_peak_memory_stats()
output = model.generate(...)
print(f'generation: {torch.cuda.max_memory_allocated() / 1024**3:.2f} GB')

Renting instead

For the 24 GB models, an hour of rented GPU is cheaper than a graphics card, and the workload suits it — generation is bursty, and you are not going to keep a 4090 busy.

• Match the model, not the budget. YuE2 needs 24 GB and BF16 support; a cheap 16 GB card cannot run it at any price.
• Weights are the slow part. Several GB per model on first run. Use persistent storage or you re-download every session.
• Generation is one-at-a-time on most of these. YuE2 documents one request at a time. A bigger GPU is worth more than more GPUs.
• Watch the licence. Running a non-commercial model on rented hardware does not make it commercial — see the licensing matrix.

A note on these figures

Model requirements move. Projects add offloading, quantisation lands, a new small variant appears. The documented figures here were checked against project documentation on the date at the top of this page; the estimates are exactly that. Measure before you buy hardware, and treat any single number — including these — as a starting point.

Next: the licensing matrix covers the other question that decides which of these you can actually use.

← AI music generation hub