Aviva Rutkin| Newscientist | Feb 4th 2014
“He moistened his lips uneasily.” It sounds like a cheap romance novel, but this line is actually lifted from quite a different type of prose: a neuroscience study.
Along with other sentences, including “Have you got enough blankets?” and “And what eyes they were”, it was used to build the first map of how the brain processes the building blocks of speech – distinct units of sound known as phonemes.
The map reveals that the brain devotes distinct areas to processing different types of phonemes. It might one day help efforts to read off what someone is hearing from a brain scan.
“If you could see the brain of someone who is listening to speech, there is a rapid activation of different areas, each responding specifically to a particular feature the speaker is producing,” says Nima Mesgarani, an electrical engineer at Columbia University in New York City.
Snakes on a brain
To build the map, Mesgarani's team turned to a group of volunteers who already had electrodes implanted in their brains as part of an unrelatedtreatment for epilepsy. The invasive electrodes sit directly on the surface of the brain, providing a unique and detailed view of neural activity.
The researchers got the volunteers to listen to hundreds of snippets of speech taken from a database designed to provide an efficient way to cycle through a wide variety of phonemes, while monitoring the signals from the electrodes. As well as those already mentioned, sentences ran the gamut from “It had gone like clockwork” to “Junior, what on Earth's the matter with you?” to “Nobody likes snakes”.
This showed that there are distinct areas in a brain region called the superior temporal gyrus that are dedicated to different types of sounds – the STG isalready known to be involved in filtering incoming sound.
One cluster of neurons responded only to consonants, while another responded only to vowels. These two areas then appeared to divide into even smaller groups. For example, some of the consonant neurons responded only to fricatives – sounds that force air through a narrow channel, like the s sound at the beginning of “scientist”. Others responded only to plosives – sounds that block airflow, like the b in “brain”.
Other scientists have used neural activity in attempts to recreate what people are looking at or capture people's thoughts in the form of their inner voice.
Mystery of meaning
Mesgarani thinks that the phoneme map may make it easier to figure out what someone is hearing from nothing but brain signals. He has tried to do this before by analysing neural responses to sound in ferrets and in people. The map should make relating the brain activity to specific sounds much easier.
