Just Think It, and the Computer Will Say It

December 3, 2012

This super-advanced technology would articulate letters or words by decoding brain activity triggered by the patients’ thinking of the sounds they wish to make. “Our experiments began a long time before meeting the patients,” Tankus says. “First we designed what we would ask patients to do and decided to focus on vowels in order to build speech from vowels to simple syllables and hopefully to combine them later into words and maybe sentences.”

But how to find volunteers for such a study? At [America’s] UCLA Medical Center [where his partners work], patients with epilepsy unresponsive to medication were having electrodes surgically implanted in their heads, so that doctors could record brain activity on the cell level during seizures. Many of the volunteers in this study were happy to participate in Tankus’ research.

Ariel Tankus First he instructed volunteers to say a vowel sound—“ah,” for example—every time they heard a beep. By synchronizing the software on his laptop with the neural recording equipment already hooked up to the patient, Tankus recorded brain activity accompanying each “beep-ah-beep-ah” sequence. He also recorded activity when the patient said a consonant-and-vowel syllable.

Next, he coded the data so the software could predict—based only on brain activity and not a sound recording—what sound the person spoke. Sure enough, when a patient said “ah,” the computer played back “ah.” The researchers mapped out the mathematical arrangement of how the various vowel sounds are represented in areas of the brain, connecting the brain representation with the anatomy and physiology of vowel articulation.

In the next stage, the brain-machine interface predicted and played back a vowel sound that the patient thought about after hearing the beep but did not utter out loud. Tankus reports that the patients were “surprised that they managed to make the computer speak instead of them.” For more information: [email protected]

Source: Excerpts of an article by Abigail Klein Leichman, www.israel21c.org