Can user feedback replace expert labels for audio AI? What the data says

Last updated: July 25, 2026

TL;DR: Google's MusicRL trained on 300,000 user preferences, yet its own ablations show preference tracks musicality — not text adherence or audio quality (Cideron et al., ICML 2024). User feedback can tune an already-good model; expert labels are what you need for fine-grained judgment, where music experts agree just ~75% of the time (CMI-RewardBench, ICML 2026).

User feedback, crowd votes, and expert labels: what's the difference?

User feedback, crowd votes, and expert labels differ on what they measure, how far they scale, and how reliable they are. User feedback captures real behaviour at massive scale but noisily; crowd votes capture broad listener preference; expert labels capture fine-grained quality and prompt adherence with high consistency. Choosing a signal means choosing which of those you can trade away.

The same three-way split shapes how AI judges are built, too — see what the data says about whether AI can judge music quality.

What did Google learn from 300,000 user preferences?

Google's MusicRL collected 300,000 pairwise user preferences and found that user preference tracks musicality, not text adherence or audio quality alone. The paper's authors write that "ablation studies shed light on the musical attributes influencing human preferences, indicating that text adherence and quality only account for a part of it" (Cideron et al., Google DeepMind, ICML 2024).

MusicRL layered a quality-and-adherence reward first and then user preference; that combined version (MusicRL-RU) was preferred over the baseline 87% of the time — more than preference tuning alone. User feedback improved the model, but only as a finishing layer on signals the crowd does not supply.

Do crowd votes and expert panels judge music the same way?

Only partly — crowd preference is dominated by musicality, so reward models scored on musicality reach roughly 73% agreement with crowd votes, but the crowd and experts diverge on prompt adherence and fine detail. Live platforms like Music Arena (Carnegie Mellon) aggregate anonymous pairwise votes — including an explicit "both are bad" option — into public rankings.

Those rankings are useful but are not a proxy for expert judgment. In an independent probe, two researchers blind-labeled 30 pairs from the Music Arena split and agreed with the public vote only 63.3% and 70.0% of the time, while agreeing with each other on 86.7% — a sample its authors call too small for a precise ceiling (TuneJury, 2026). Crowd data tells you what listeners prefer, not why a track is well made.

How much more reliable are expert annotators?

Expert music annotators agree with each other about 75% of the time on head-to-head musicality judgments — a moderate ceiling, not a perfect one. On CMI-RewardBench (ICML 2026), expert re-annotation agreement was 75.2% on musicality and 75.0% on alignment, with a moderate Krippendorff's alpha near 0.50.

Expert panels earn that reliability through structure: SongEval was rated by 16 professional annotators across 2,399 full-length songs (140+ hours) on five aesthetic dimensions, and MUSHRA (ITU-R BS.1534) calls for 15–20 screened listeners per test. Reward models trained on expert preferences reach about 78% agreement on CMI-RewardBench, above general models like Gemini 3 Pro (65.8%) and Qwen3-Omni (60.4%).

Are user preferences biased?

Yes — user ratings skew toward musicality and perceived authenticity and away from text adherence and technical quality, and identical tracks score lower once listeners know they are AI-generated. In a 2026 study, listeners preferred AI-generated music in blind tests but became less willing to replay or pay for it the moment it was labelled "AI" (ProMarket, 2026).

Preference data also inherits popularity and cultural bias from the population that produced it. None of that makes user feedback useless — it measures reception, not craft.

When is user feedback enough to train on?

User feedback is enough when you are tuning an already-capable model, optimizing for engagement, or operating at a scale expert labels cannot reach. Once a model is good, preference data is the cheapest way to push it toward what listeners enjoy, and platforms like Spotify treat likes and skips as exactly this implicit signal.

The constraint is quality, not volume: a smaller set of clean, high-signal judgments beats a larger noisy one, so millions of ambiguous skips do not substitute for a well-designed preference set.

When do you need expert labels instead?

You need expert labels for supervised training, for fine-grained or segment-level judgment, and for the "both outputs are bad" case where preference data breaks down. When two candidates are each poor, a forced pairwise choice teaches a model "which bad output is less bad" rather than how to be good — the degenerate-preference problem that clean expert annotation avoids by scoring against an absolute standard.

Expert labels are also the only signal that reliably captures prompt adherence, structure, and mix-level detail — the dimensions the crowd underweights. Once you need them, the next question is who supplies them: crowd platforms, data vendors, and expert audio teams compared.

How to choose: match the signal to the decision

The signals are complements, not rivals, so the useful question is which one fits the decision in front of you. Ask three things:

(1) Are you tuning an existing model or teaching it a new capability? Tuning tolerates preference data; new capability needs labelled ground truth. (2) Do you need a holistic "which is better," or a fine-grained, segment-level judgment? Holistic suits the crowd; fine-grained needs experts. (3) Can you tolerate noise, or do you need a defensible standard? Engagement metrics are fine when noise averages out; evaluation and safety-critical work are not. Match the signal to the answer rather than defaulting to whichever data is cheapest to collect.

FAQ

Can you train a music model on likes and skips?

Yes, for preference tuning on an already-capable model. Google's MusicRL used 300,000 pairwise preferences this way, but ablations show likes track musicality, not prompt adherence, so likes and skips refine a model rather than teach it a new skill.

Do arena leaderboards reflect expert judgment?

Only partly. Crowd and arena preference correlates with musicality, but in an independent 30-pair probe (TuneJury, 2026), two researchers agreed with released Music Arena votes just 63–70% of the time while agreeing with each other 87% of the time — crowd rankings and expert judgment are related but distinct measures.

What agreement rate do music experts reach?

About 75% on head-to-head musicality judgments. CMI-RewardBench (ICML 2026) measured 75.2% re-annotation agreement on musicality and 75.0% on alignment, with a moderate Krippendorff's alpha near 0.50.

Why does preference data need a good model first?

Preference tuning refines a model rather than building capability. In MusicRL, the version that applied a quality reward first and then user preference (MusicRL-RU) beat the baseline 87% of the time, outperforming preference-only tuning.

Are user preferences biased when rating music?

Yes. Preferences skew toward musicality and perceived authenticity and away from text adherence and technical quality, and listeners rate identical tracks lower once told they are AI-generated (ProMarket, 2026).

Wave is a team of audio-industry veterans with decades in studios and control rooms — producing expert-annotated, studio-quality audio datasets for AI teams. The ~75% expert ceiling in this article is the product: custom datasets built to your spec — tasks, traces, captions, attributes, corrections, segment-level labels — by ears that can hear the difference. Talk to us.