Semantic Predictions and the Waves of Speech

Source: BenMack/Pexels

How does the brain use the information provided by the acoustic “envelope” or “wave” of ongoing speech to predict what's coming next? Going beyond this, is it possible that the brain can recruit semantic information to facilitate this component of speech perception? A recent study led by Nicola Molinaro at the Basque Center on Cognition, Brain and Language explored these interesting questions.

It is known that auditory-related neural activity oscillates in synchrony with rhythmic regularities of external sounds. This, it has been claimed, is essential for encoding acoustic speech properties. One of the possible computational roles of this mechanism is to organize the realignment of the phase of maximum neural excitability to match temporal regularities of perceptual content (i.e., between the peak and trough of the ongoing oscillatory phase).

It is conceivable that by providing people with a rich semantic context, the perceptual representation of a predictable stimulus is enhanced. For example, it may be that by discussing the issue of space and the moon, language users might easily predict a word like “spacecraft.” Lacking such semantic context, encountering the word “spacecraft” may result in a less rich and efficient degree of perceptual enhancement.

Molinaro and colleagues exposed participants to a recording of naturalistic text reading, using scalp electroencephalography (EEG) to record cortical neural signatures. The authors evaluated speech entrainment effects by analyzing phase alignment between neural activity and the envelope of the target words that followed either a semantically constrained context or a non-constrained context.

The authors found increased speech entrainment for words in contexts that led participants to predict the target words between 400 and 450ms. These effects were located in eight central scalp electrodes (directly on the top surface of the skull).

Overall, these intriguing results are not just fascinating from a mechanistic perspective. They also contribute to our conceptual understanding of language processing in the brain. For instance, the results support a cost-minimization model: when internal, endogenous semantic inferences cannot be readily generated in discourse contexts with little semantic narrowing/canalization, the brain exploits other resources to aid comprehension. It seems that the brain invests more computational resources in lower-level envelope tracking when little semantic information has been pre-activated/primed.

Importantly, the acoustic properties of the target word stimuli in both conditions did not differ, and so these effects seem to arise purely from semantic properties of the discourse. As such, some kind of trade-off between top-down and bottom-up processes seems to be at play, although the precise granularity of this trade-off (i.e., when and how it occurs, and whether envelope tracking is the only mechanism that comes to the aid of higher-order semantic inference generation) remains unclear.

The authors wanted to avoid the use of highly controlled stimuli in an unecological setting, so included no specific experimental task for participants, who simply had to passively listen to the stimuli. Moving forward, it would be interesting to see if effects of attention and memory impact the recruitment of oscillatory phase-speech tracking, alongside the apparent involvement of pre-activated semantic representations.