Generate Songs with YuE2

YuE2 writes an editable melody-and-chord score, then renders it as a full 48 kHz song with vocals — the only open music model where you can read and change the composition before it becomes audio.

Last updated 2026-09-12

YuE is the open lyrics-to-song model from M·A·P and HKUST. It is also the most-changed project in open music generation: the original YuE has been superseded by YuE2, which works in a fundamentally different way and posts benchmark numbers competitive with Suno.

Almost everything written about YuE online describes version 1. This page covers YuE2 — what actually installs today.

What makes YuE2 different: the score is visible

Nearly every music model is a black box — prompt in, audio out, and if it's 80% right your only option is to re-roll and hope. YuE2 splits generation into two stages you can get between.

One AR–NAR Mixture-of-Transformers backbone predicts a symbolic plan — melody and chords, written as ABC notation — and semantic tokens autoregressively. Flow matching then generates acoustic latents from that plan, and a VAE decodes them to 48 kHz stereo audio.

The consequence is the whole point of the model: you can read, play and edit the composition before it becomes audio. Don't like the chord in bar 12? Change that chord and re-render, with the melody, tempo, meter and lyrics held exactly. The project calls this white-box music generation, and no other open music model offers it.

It also explains why YuE2 handles covers well. Transcribe an existing recording to a score, hand the score back with a different style prompt, and you get the same tune performed differently.

Hardware and platform: read this before installing

That 24 GB on Linux is the steepest requirement of any model covered on this site. There is no small variant, no CPU fallback and no Apple Silicon support. A 4090, a 3090, or rented cloud time.

Install

git clone https://github.com/multimodal-art-projection/YuE.git
cd YuE
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install .

python examples/generate.py --output outputs/first-song

Model files download from Hugging Face on first use. Open outputs/first-song/audio.flac — and note the output directory also keeps the score, the semantic tokens, the acoustic latents, the settings and the model identities. That record-keeping is deliberate, and it's what makes comparisons between runs meaningful.

Your first song

A request is a small JSON object. This is the project's own example:

{
  "id": "city_lights",
  "style": "English, warm piano pop, expressive female voice, acoustic piano, rounded bass and light drums, lyrical memorable melody, unhurried phrasing, 88 BPM",
  "lyrics": "[Verse]\nNeon fades along the lane\nFootsteps keep the time of rain\nFold the night and leave it here\nMorning has a sky to clear\n\n[Chorus]\nLet the day come into view\nEvery road begins with you\nHold a little room for light\nWe will sing beyond the night",
  "cot": "full",
  "seed": 831001
}

The two text fields do different jobs. style carries genre, instruments, vocal character, language and tempo. lyrics carries the words, with [Verse] and [Chorus] section tags. Note the style field leads with the language — that's how you steer it.

import json
from pathlib import Path
from yue2 import YuE2Pipeline

request = json.loads(Path('examples/song.json').read_text(encoding='utf-8'))

with YuE2Pipeline.from_pretrained('m-a-p/YuE2-3B', device='cuda') as pipe:
    song = pipe(**request)
    song.save_artifacts('outputs/my-song')
    print(song.truncated)

Or from the command line:

yue2 generate --request examples/song.json --output outputs/song-cli

One call produces one candidate. The headline benchmark figure uses best-of-8 selection, which is a separate evaluation step — so if you compare a single local generation against the published numbers, you are not comparing like with like.

Planning modes

The distinction between full and melody is about how much freedom the arrangement gets. A full score fixes the harmony; a melody-only score lets the model reharmonise to suit the style you asked for. That is exactly why melody-only is the recommendation for covers — you want the tune, not the original chords.

cfg_scale controls text guidance. The defaults are ready to use, and the project's own guidance is that changing sampling or guidance settings may change quality — not necessarily improve it.

The staged API: stop between plan and audio

This is the part worth learning. Rather than one call, run the stages yourself and inspect the composition in between:

import json
from pathlib import Path
import soundfile as sf
from yue2 import YuE2Pipeline, SymbolicPlan

request = json.loads(Path('examples/song.json').read_text(encoding='utf-8'))

with YuE2Pipeline.from_pretrained('m-a-p/YuE2-3B', device='cuda') as pipe:
    plan = pipe.plan(**request)       # 1. the composition
    plan.save('outputs/plan')         #    inspect outputs/plan/score.abc here

    restored = SymbolicPlan.load('outputs/plan')
    semantic = pipe.generate_semantic(restored)   # 2. semantic tokens
    latents  = pipe.synthesize(semantic)          # 3. acoustic latents
    audio    = pipe.decode(latents)               # 4. waveform

sf.write('outputs/plan/audio.flac', audio, 48000)

SymbolicPlan.load restores exact token IDs and verifies the saved files, so re-rendering an unchanged plan is genuinely reproducible.

Editing the composition

The workflow that makes YuE2 worth the 24 GB. Generate a plan, edit the score, re-render:

python skills/yue2-music/scripts/run_yue2.py plan \
  --request examples/song.json --output outputs/plan

cp outputs/plan/score.abc edited.abc

Now edit edited.abc — change chord symbols, adjust a melody note, alter the section order — and render it:

python examples/generate.py --request examples/song.json \
  --abc-file edited.abc --cot full --output outputs/edited

The repository ships abc_tools.py for checking that an edit changed only what you intended:

python skills/yue2-music/scripts/abc_tools.py inspect edited.abc
python skills/yue2-music/scripts/abc_tools.py compare outputs/plan/score.abc edited.abc

A match: true result means notes, timing, meter and tempo are unchanged — the comparison deliberately allows harmony to differ. That's a real regression test for a musical edit, which is an unusual thing to be able to say about a generative model.

One limit worth being clear about: editing re-renders the whole song. It does not preserve the original waveform outside the edited region. If you loved the vocal take, editing the harmony will not keep it.

Covers: transcribe, restyle, re-render

Covers use transcription of a different kind — SheetSage2, the project's own audio-to-score model. It needs its own environment, because its dependencies conflict with YuE2's:

python3.11 -m venv .venv-sheetsage2
.venv-sheetsage2/bin/python -m pip install huggingface-hub==0.36.0
.venv-sheetsage2/bin/huggingface-cli download m-a-p/SheetSage2 \
  --local-dir models/SheetSage2
.venv-sheetsage2/bin/python -m pip install \
  torch==2.8.0 torchaudio==2.8.0 \
  --index-url https://download.pytorch.org/whl/cu126
.venv-sheetsage2/bin/python -m pip install -r models/SheetSage2/requirements.txt

Transcribe the source, keeping melody only so the accompaniment can move:

.venv-sheetsage2/bin/python models/SheetSage2/infer.py source.wav \
  --output cover-score --melody-only

Then hand the score back to YuE2 with new lyrics and a target style:

.venv/bin/python examples/generate.py --request cover-request.json \
  --abc-file cover-score/score.abc --cot melody --output outputs/cover

Run the two sequentially so they can share one GPU. And review the transcription before generating — transcription errors propagate straight into the cover.

The reported cover numbers are striking: 0.647 CLEWS mAP with a full score versus 0.006 without one, on 948 works, using the general checkpoint with no cover-specific fine-tuning. The score is doing essentially all the work of preserving musical identity.

Where it actually ranks

From the project's WildSongBench evaluation — 192 prompts, automatic metrics. Abbreviated to the relevant rows; † marks models with publicly available weights.

Read these carefully, and with the usual caution that they are the authors' own evaluation of their own model:

• The jump from YuE 1 to YuE2 is enormous — 4.92 to 6.73, and word error rate from 36% to 8%. Version 1's mangled lyrics were its defining weakness; that is fixed.
• An open model is now in the same band as Suno. That was not true a year ago.
• The authors say so themselves: the small gap between the top means does not establish statistical significance. YuE2 at 6.96 versus Suno v5 at 6.87 is a tie, not a win.
• Suno still wins on lyric intelligibility (PER 5.80% for v4.5 vs 8.44%) and text alignment (MuLan 0.5428).

Reproducibility and decoders

Two decoders ship, and which you use changes the audio:

with YuE2Pipeline.from_pretrained(
        'm-a-p/YuE2-3B',
        vae='m-a-p/YuE2-Vae-legacy',
        device='cuda') as pipe:
    ...

# or decode cached latents again with a different VAE,
# instead of generating a whole new song
python skills/yue2-music/scripts/run_yue2.py decode \
  --source outputs/song --output outputs/song-benchmark \
  --vae m-a-p/YuE2-Vae-legacy

from_pretrained also accepts local model directories, revision, vae_revision, cache_dir and local_files_only=True. Pin both model and VAE revisions when comparing runs — and note that a seeded generation can still differ across GPUs, runtime versions or sampling settings. A seed is not a guarantee here.

Troubleshooting

Licensing: the catch

That puts YuE2 in the same commercial position as MusicGen: excellent, open to inspect, not licensed for you to sell the output of. Among open song models with vocals, ACE-Step remains the one whose weights are Apache-2.0.

Where it fits

YuE2 is the best-sounding open song model, and the only one where you can read and edit the composition before it becomes audio. If you are researching music generation, doing zero-shot covers, or want musical control rather than prompt roulette, it is the most interesting thing in open source right now.

It is also Linux-only, needs 24 GB, and its weights are non-commercial. For a product, use ACE-Step. For instrumental texture, use MusicGen. For sound design, use Stable Audio Open. For a song you want to shape like a composer rather than describe like a customer, this is the one.

See also YuE vs ACE-Step and the open models overview.

How this model’s licence compares with every other open model → · GPU & VRAM requirements →

← AI music generation hub