Train Your Own RAVE Model
RAVE learns the timbre of a sound corpus and lets you play through it in real time. Dataset preparation, every training config, the export flag everyone forgets, and what to do with the latent once you have it.
Last updated 2026-09-12
RAVE is not a text-to-music model, and that is the point. It is a variational autoencoder from IRCAM that learns the timbre of a sound corpus — a violin, a voice, a room full of percussion, your own modular rig — and then lets you play through it in real time, inside Max/MSP or Pure Data, at low enough latency to perform with.
You feed it audio, it re-synthesises that audio in the voice of whatever it was trained on. Sing into a model trained on darbouka and you get drums that follow your phrasing. That is a fundamentally different creative proposition from prompting for a finished track, and it is the reason RAVE has quietly become a fixture in live electronic performance.
How RAVE works, and why latency is the whole story
RAVE is an autoencoder: an encoder compresses audio into a small latent representation, a decoder reconstructs audio from it. Training makes that round trip faithful for one particular corpus.
Two things follow:
- The compression ratio is large, so the latent throws away everything that isn't characteristic of the training corpus. Reconstructing someone else's audio through it is therefore timbre transfer, for free — that is not a separate feature, it's a side effect of the architecture.
- The latent is a handful of continuous signals you can see and manipulate. Inside Max or Pure Data you can scale them, offset them, delay them, cross-fade between two sources, drive them from an LFO. This is where the interesting work happens — not in the model, but in what you do to the latent before it reaches the decoder.
Everything about the workflow below exists to keep that round trip fast enough to play.
Two ways in
| Pretrained models | Train your own | |
|---|---|---|
| Effort | Download and load | Days of GPU time |
| Audio needed | None | Hours of a coherent corpus |
| Hardware | A laptop | 8–32 GB VRAM depending on config |
| Result | Someone else's timbre | Your own instrument |
Start with the pretrained streaming models from the ACIDS collection to learn the patching side. Train your own when you know what you want the latent to sound like.
There is also a RAVE VST in beta from Forum IRCAM, for Windows, Mac and Linux, if you want this in a DAW without patching anything.
Install
python -m venv .venv && source .venv/bin/activate
# torch FIRST, matched to your CUDA version — see pytorch.org
pip install torch torchaudio
pip install acids-rave
# ffmpeg is required
conda install ffmpeg # or brew install ffmpeg / apt-get install ffmpegUsing a pretrained model offline
Before touching Max, hear what the model does to your audio. The batch script processes files or whole directories:
rave generate /path/to/model.ts input_folder/ another_file.wav --out out_path/This is also the fastest way to audition several models against the same source material and decide which one is worth building a patch around.
Training your own: dataset preparation
Three steps — preprocess, train, export. The first one decides most of your result.
rave preprocess --input_path /audio/folder \
--output_path /dataset/path \
--channels 1
# large corpus of compressed files you cannot afford to expand
rave preprocess --input_path /audio/folder \
--output_path /dataset/path \
--channels 1 --lazy| Flag | Default | What it does |
|---|---|---|
| --input_path | required | Directory of audio. Can be given more than once |
| --output_path | required | Where the prepared dataset goes |
| --channels | 1 | 1 for mono, 2 for stereo. Must match what you pass to train |
| --sampling_rate | 44100 | Training sample rate |
| --num_signal | 131072 | Samples per training example — about 3 s at 44.1 kHz. Lower it for short source files |
| --lazy | off | Train directly on mp3/ogg without converting. Saves enormous disk, costs a lot of CPU during training — badly on Windows |
| --max_db_size | 100 | Dataset cap in GB |
| --ext | aif, aiff, wav, opus, mp3, aac, flac, ogg | Extensions to search for |
What to feed it matters more than any flag. RAVE learns one coherent timbre; it does not learn variety. A few hours of solo cello gives a usable cello model. A few hours of "assorted music" gives mush. The rules of thumb that hold up:
- Coherence over quantity. One instrument, one voice, one recording setup. Two hours of consistent material beats ten hours of everything.
- A few hours is a realistic minimum. Less than an hour tends to overfit audibly.
- Clean and dry. Heavy reverb gets baked into the model and you can never remove it. If the source is a full mix, separate it with Demucs first and train on a single stem.
- Keep the dynamics. Don't feed it limited masters; the model learns loudness behaviour too.
Training
rave train --config v2 \
--db_path /dataset/path \
--out_path /model/out \
--name my_instrument \
--channels 1| Flag | Default | What it does |
|---|---|---|
| --config | v2 | Architecture. Repeatable — configs combine |
| --db_path | required | The preprocessed dataset |
| --out_path | required | Where checkpoints and logs go |
| --name | required | Run name |
| --channels | 0 | Must match preprocessing |
| --batch | 8 | Lower it if you run out of VRAM |
| --max_steps | 6000000 | Effectively "until you stop it" — see below |
| --val_every | 10000 | Checkpoint interval |
| --save_every | 500000 | Keep a numbered checkpoint this often |
| --n_signal | 131072 | Samples per example. Match preprocessing |
| --ckpt | none | Resume from a checkpoint |
| --gpu | auto | Which GPU(s) |
| --augment | none | Data augmentation, repeatable |
Choosing a configuration
| Config | Min GPU | What it's for |
|---|---|---|
| v2 | 16 GB | The default. Faster and higher quality than v1. Start here |
| v2_small | 8 GB | Smaller receptive field, noise generator — tuned for timbre transfer on stationary signals, and the practical choice on consumer GPUs |
| v1 | 8 GB | The original continuous model |
| v3 | 32 GB | Snake activation, Descript discriminator, Adaptive Instance Normalization for real style transfer |
| discrete | 18 GB | Token-based, like SoundStream or EnCodec. Needed if you want a prior |
| v2_nopqmf | 16 GB | Experimental; more efficient for network bending |
| onnx | 6 GB | Noiseless v1 for ONNX export |
| raspberry | 5 GB | Lightweight enough for real-time Raspberry Pi 4 inference |
Configs combine, which is how you get a causal model for lower latency:
# lower latency for live playing
rave train --config v2_small --config causal \
--db_path /dataset/path --out_path /model/out \
--name my_instrument_live --channels 1
# discrete, causal
rave train --config discrete --config causal ...| Modifier | Effect |
|---|---|
| causal | Causal convolutions — lower latency, slightly lower quality. Worth it for live playing |
| noise | Enables the v2 noise synthesiser. Helps on breathy or noisy material |
| hybrid | Mel-spectrogram input |
| wasserstein | Wasserstein objective (MMD) instead of the default ELBO — v2 only |
| spherical | Spherical autoencoder objective — v2 only |
| spectral_discriminator | EnCodec's MultiScale discriminator |
Augmentation
Added in 2.3, and genuinely useful when your corpus is small:
rave train --config v2 --augment mute --augment compress --augment gain \
--db_path /dataset/path --out_path /model/out \
--name my_instrument --channels 1mute randomly silences batches so the model learns silence — which matters more than it sounds, because a model that has never heard silence will hallucinate sound into your rests. compress applies light non-linear amplification, and gain a random gain in roughly [-6, 3] dB.
How long training takes
The honest answer is days. The default max_steps of six million is not a target you are expected to reach — you watch the TensorBoard logs and the audio samples, and you stop when reconstructions stop improving.
Rough expectations on a single modern GPU: something recognisable within a day, something usable in two to three, diminishing returns after that. Training is resumable with --ckpt, so stopping to listen costs you nothing.
Export — and the mistake everyone makes
rave export --run /path/to/your/run --streaming| Flag | Default | What it does |
|---|---|---|
| --run | required | The training run to export |
| --streaming | off | Cached convolutions for real-time use. You want this |
| --fidelity | 0.95 | Latent fidelity for inference, 0.1–0.999 (variational mode). Lower keeps fewer latent dimensions — blurrier, more "the model's own voice" |
| --ema_weights | off | Export exponential-moving-average weights |
| --prior | none | Script a trained prior alongside the model |
| --channels / --sr | from run | Override channel count or sample rate |
| --output / --name | run dir | Where the .ts file goes |
The output is a TorchScript .ts file. That single file is your instrument.
Playing it: nn~ in Max/MSP and Pure Data
Load the exported model in nn~. The default method is forward — encode then decode in one object, which is the fastest path and the right default:
nn~ my_instrument forwardSplitting encode and decode into separate objects does the same thing slightly slower, but hands you the latent in between. That is the whole reason to bother:
- Scale and offset latent dimensions to exaggerate or suppress characteristics of the model.
- Delay individual dimensions against each other to smear the timbre in time.
- Freeze a latent and let the decoder drone on it.
- Cross-fade between two sources' latents before a single decoder.
- Drive dimensions from LFOs or envelopes instead of from audio at all.
For style transfer, recent versions add Adaptive Instance Normalization, so source and target styles can be set through nn~'s attribute system directly in the patch. Other attributes — enable, gpu — toggle computation or move it to the GPU.
Priors: let it play itself
A prior is a second model trained over the latent space, so RAVE can generate without any input audio — unconditional synthesis in the voice of your corpus.
rave train_prior --model /path/to/your/run \
--db_path /dataset/path \
--out_path /prior/outThen script the prior alongside the model at export time:
rave train_prior --model /path/to/your/run \
--db_path /dataset/path \
--out_path /prior/outFor discrete models the project points at the separate msprior library, though the v1 prior has been re-integrated in 2.3.
Troubleshooting
| Symptom | Cause and fix |
|---|---|
| Clicking artefacts in Max | Exported without --streaming. Re-export |
| Preprocessing stuck at 0it [00:00, ?it/s] | Source files are shorter than the training window. Lower --num_signal in preprocess, and pass the matching --n_signal to train |
| ValueError: n_components=128 must be between 0 and min(n_samples, n_features)=64 | Not enough data for the internal latent PCA, which needs at least 128 batches. Add more audio |
| Out of memory while training | Lower --batch, or move to v2_small |
| Output is mush | Corpus isn't coherent enough. One instrument, one setup |
| Reverb you can't remove | Baked in from the training audio. Retrain on dry material |
| Latency too high to play | Add the causal config and retrain; raise your audio buffer only as a last resort |
| Channel-count errors at training | --channels must match between preprocess and train |
Licensing
A model you train on your own audio is still produced with CC BY-NC tooling, so if money is involved, read the licence rather than assuming your training data settles the question.
Where it fits
RAVE is the odd one out in this section and the most rewarding if your interest is performance rather than production. It does not write music. It gives you an instrument with a voice nothing else has, that responds to playing in real time, and that you built from a corpus you chose.
The nearest relative is DDSP, which is more explicitly physical in its modelling and better on clean monophonic instruments; RAVE handles messier material and is built for live use. If you want generation instead of transformation, go to MusicGen or ACE-Step.
For the patching side, see Max/MSP and Pure Data; for preparing a corpus, Demucs and pedalboard.
How this model’s licence compares with every other open model → · GPU & VRAM requirements →