Hearing sounds and seeing colors --Measuring the synesthesia

(JAL Sky Seminar, JAL SKYWARD vol.49, October, 2006.)

I think many of you know the Story of Bouba and Kiki. It is a brain science experiment introduced in Phantoms in the Brain, a Brief Tour of Human Consciousness: From Impostor Poodles to Purple Numbers written by V. S. Ramachandran. The story focuses on two figures, one a roundish figure and the other a pointed figure. Which of these is a 'bouba,' and which is a 'kiki'? To this question, it is said that 98% of people select the pointed figure as a kiki. This story indicates that the attribute "sharp", which is common between the figure (visual sensation) and the word (auditory sensation) are recognized in the brain across multiple sensations.

Now, how many of you have feelings of multiple sensations mixed together, such as "feeling the color of black wood in the letter a", or "feeling a sharp shape in the taste of sauce". This is called "synesthesia" and it is said that one in every 2000 person has this feeling. Synesthesia has been known for more than 100 years. However, a scientific breakthrough has taken place recently by progress in brain functional measurement technology. Measuring the brains of babies 3 months old and under, we can see that the visual field in the occipital lobe is activated when auditory stimulation is given. Also, though this is just a hypothesis, at the time of birth a path between the auditory area and the visual area exists, but through the growth process, it gets cut off (spontaneous death of cells referred to as apoptosis). In other words, it is assumed that originally everybody may have experienced synesthesia and that bouba and kiki are mechanisms similar to that.

Our theme is the "colored-hearing" phenomenon that gives feeling of color in tonality of music such as C major and F major. Ms. N.T., a pianist, is said to have remarked when she was 6 years old that "Minuet (G major) is music with a blue color". Even now she says she does not wear dresses with colors that do not match the music she plays. Ms. S.O., a cellist, says that a color that seems to be "wriggling" looks like a gradation. When we measure the brain activity when these musicians are listening to music with their eyes closed, an activity is seen to be taking place in the vicinity of the color perception areas called V4 and V8 in the visual cortex. What is interesting is that when they listen to music with their eyes open, the activity in the color perception area becomes strong. This is considered to be evidence that a person who detects colored-hearing can, while receiving the colors of the real world, dually represent the color evoked by the music at the same time.

As a matter of fact, I also see color. To me, C major music has a rough white texture. I memorize historical eras and phone numbers by lines of colors. It has been natural for me to do this since I was little. When I found out that I'm different, I felt that it would be very inconvenient if I could not see colors. In the world of art, including the Disney picture Fantasia and pictures of Kandinsky, attempts to use interactions of the five senses have been made. If synesthesia is a function that everyone has, it must have been useful in creating arts and multi-media. If you are one of the people who feel color in the tonality of music, I would like to ask for your kind participation in our research.