Color Never Hears the Music | Exploring AI, Perception & Sensory Experience | Tanmay Kulkarni

The video opens with two clips of the same flat landscape. In one, the music is quick and bright and the sky is yellow; in the other, the music is slow and low and the sky is blue-gray. Tanmay Kulkarni asks the viewer to pair each sky with a tune. Most people put the yellow sky with the quick tune, which raises a puzzle: nothing about a yellow sky sounds quick, so what is being matched?

The Berkeley experiment

The answer comes from a 2013 study by Stephen Palmer and colleagues at Berkeley. Listeners heard 18 orchestral pieces by Bach, Mozart, and Brahms, each 50 seconds long, spread across slow, medium, and fast tempos and across major and minor keys. About 97 people in the United States and Mexico took part. For each piece they picked the five colors from a set of 37 that fit best and the five that fit worst. In both countries, quick major music drew lighter, more saturated, yellower colors, while slow minor music drew darker, grayer, bluer ones.

Feelings as the middle step

The researchers also had people rate every piece and every color on feelings: happy to sad, strong to weak, and more. When the ratings of the music and the ratings of the colors it drew were compared, they lined up closely. The correlation was 0.97 for happy to sad and 0.96 for strong to weak, and across all four word pairs it ranged from 0.89 to 0.99. The idea that follows is that color does not hear the music. Both carry the same feeling, and the step in the middle is the rating, so the sound never has to touch the color. The video is careful about one limit: these are correlations, and the authors say a direct link is not ruled out.

Two worked examples from the paper

The video tests the idea on the paper's own examples. Bach's fast dance in F major is rated happy and strong, and the paper places saturated yellow and light yellow in that corner. Brahms' slow adagio in C minor is rated sad and weak, and it drew black, dark gray, and dark cyan.

How far the match travels

The video gives two reasons color might be ready to carry feeling. An essay the video draws on says babies sort colors into categories by about six months and name them by about eighteen, so sorting comes before words. As for reach, correlations between the United States and Mexico ran from 0.8 to 0.97 across the color measures. But those are two countries in the Americas, and the music was Western classical. The authors write that it is unclear for music built on other scales, and the essay notes that color symbols differ: white means purity in much of the West and mourning in many Eastern cultures. The video's advice is to take the finding as strong, not as a law.

Putting it to work in film

The same essay describes how editors set a rhythm of arousal with scene length: long, calm scenes against short, busy ones. Its rule of thumb is a calm scene of 7 to 12 seconds set against a busy one of 1 to 3 seconds, with cool colors paired to the calm strip and warm colors to the busy one. A separate test of pacing, by Angeline Lillard and Jennifer Peterson in 2011, had four-year-olds watch a fast cartoon, watch a slower one, or draw for 9 minutes. Right afterwards, the fast-cartoon group did worse on four tasks of self-control and planning. The video notes that this measured the immediate effect on one group and was about pacing, not color.

A prediction exercise

Fast music pulls toward yellow and minor pulls toward blue, so where does fast minor music land? The viewer is asked whether its color is lighter or darker than slow minor, and whether it is more or less saturated than fast major. The paper's chart for United States listeners shows fast minor drew colors lighter, more saturated, and yellower than slow minor, and less so than fast major. Tempo and mode each moved the colors on their own, and fast minor landed between.

Key takeaways

• People in two countries matched quick major music to light, saturated, yellower colors and slow minor music to darker, grayer, bluer ones.
• Ratings of music and colors on happy-sad and strong-weak lined up with correlations of 0.97 and 0.96.
• The link is explained as shared feeling, not sound touching color.
• The results are correlations, and a direct link is not ruled out.
• The study used two Americas countries and Western classical music, so it may not hold for music built on other scales.
• Color symbolism differs by culture, so treat the finding as strong but not a law.
• Tempo and mode each shifted color choices independently, and fast minor landed between the poles.

Try it yourself

Pick a short piece of music, rate it yourself on happy to sad and strong to weak, and choose the colors you would expect to fit. Then pause and predict, as the video does, where a piece that is fast but minor would land, and check whether you guess between the poles or at one end.