STEMJazz Talklet with Stephanie Haro
Abstract
Stephanie Haro gave a STEMJazz talk on Wednesday. It was eye-opening and amazing not only in the importance of what she’s doing, but in the difficulty of the problem.
Most of us are familiar with prosthetic and orthotic devices that replace or support people who have lost limbs. We’re also familiar with the promise of “bionic” versions whose movements are directed by residual muscle and, in the near future, by signals from residual peripheral nerve fibers—making such replacements as good as or better than what was lost.
Those problems, however, pale in comparison to restoring speech to people who have lost control of their vocal apparatus through diseases like ALS and other neurodegenerative conditions. For limb prostheses, the “intent” and sensory signals reside in identifiable peripheral nerve fibers near the injury. For neurodegenerative diseases, these nerve connections no longer exist because the nerves themselves have stopped carrying signals to and from the central nervous system.
For such cases, there is no choice but to insert electrodes into the brain—sometimes relatively superficially and often into deeper structures—to tap either the motor signal sources or the regions where the intent to speak is formed.
This is where Stephanie works. She seeks to give the gift of speech back to profoundly impaired individuals by studying signals from implanted brain electrodes in real time.
The basic setup digitizes signals from multi-electrode implanted arrays (8×8 at 30 kHz sampling for each channel). These electrodes are fine enough that individual action potentials can be resolved rather than just mean field potentials. For signal processing and machine learning, that’s where analysis begins. The data-processing theorem reminds us that information can only be lost through processing, and the high-speed raw data from each channel are distilled into 50 Hz segments (20 ms intervals) before being fed into a large language model that outputs phoneme estimates within the structure of English. The high data rates and limited storage make these engineering tradeoffs necessary.
Stephanie works in this post-processed world, interpreting what the data reveal about phonemes and how those estimates interact with the language model that produces speech. She is finding ways to improve separation of phonemes—she showed vowel and consonant data—by organizing and characterizing a very large signal space.
As happens with many STEMJazz talks, she did not reach all her slides, but the most exciting one was the last: a look “under the hood” at the signals she studies, showing signal-space trajectories.
The technology is not yet perfect, but it performs well above random and, in some cases, remarkably well. Even now, the results show that meaningful communication can be restored where none seemed possible before.
Stephanie finishes this postdoctoral program on November 1 and continues her research at Brown. What she’s doing is inspiring.
Thank you, Stephanie, for an extraordinary STEMJazz talk.
