RAVE vs DDSP
Both learn an instrument’s timbre and let you play through it, and they are the only two tools here that do. They disagree about how, and the difference shows up in what material each handles well.
Side by side
| RAVE | DDSP | |
|---|---|---|
| Approach | Variational autoencoder over raw audio | Differentiable DSP inside a neural net |
| Licence | CC-BY-NC 4.0 — code included | Apache-2.0 |
| Commercial use | No | Yes |
| Real-time | Yes — built for it | Not the focus |
| DAW / patcher | Max/MSP and Pure Data via nn~, plus a VST beta | Mostly notebooks |
| Control surface | Latent dimensions you can manipulate | Explicit pitch, loudness, harmonics |
| Best source material | Anything coherent, including polyphonic | Clean monophonic instruments |
| Training GPU | 8–32 GB by config | Modest |
Where they actually differ
Interpretable parameters versus a learned latent
DDSP models audio with actual DSP components — oscillators, filters, harmonic amplitudes — that remain differentiable. The parameters therefore mean something: pitch is pitch, loudness is loudness, and you can reach in and change them predictably. RAVE compresses audio into an abstract latent whose dimensions have no assigned meaning. You can manipulate them, and the results are often musically interesting, but you are exploring rather than dialling in a known value.
RAVE is built to be played
RAVE exports a TorchScript file that loads into nn~ for Max/MSP and Pure Data, with a streaming mode using cached convolutions specifically so it works on a live audio stream. There is also a VST beta from Forum IRCAM. DDSP has no comparable performance path — it lives in Python and notebooks.
DDSP is fussier about its input
Because it models harmonics plus filtered noise, DDSP is at its best on clean monophonic material — solo violin, a single voice, a flute. Feed it a polyphonic mix and its assumptions break down. RAVE is far more tolerant: it will learn whatever corpus you give it, provided the corpus is stylistically coherent.
The licence gap is decisive for commercial work
RAVE is CC-BY-NC 4.0, and unusually that covers the implementation itself, not only released weights — so even a model you train on your own audio is produced with non-commercial tooling. DDSP is Apache-2.0 throughout. For anything you intend to sell, this is the row that matters.
Which should you choose?
Choose RAVE when…
- You want to perform with it, live
- Your source material is polyphonic or messy
- You work in Max/MSP or Pure Data
- The work is non-commercial
Choose DDSP when…
- You want interpretable, predictable control
- Your source is a clean monophonic instrument
- You need a permissive licence for commercial use
- You are building your own model rather than performing
Full RAVE tutorial — dataset prep, configs, export and nn~ →