Perceptual weighting of the envelope and fine structure across frequency bands during continuous and interrupted speech

Listening to speech requires the processing of complex patterns of spectral and temporal acoustic properties. The speech signal may be divided into spectral frequency bands, each band containing temporal properties of the envelope (i.e., amplitude modulations) and fine structure (i.e., frequency modulations). However, certain acoustic properties may be more important for understanding speech. Furthermore, the listening environment may influence the acoustic properties that are most informative at any given moment due to acoustic interactions between the speech source and competing sound sources. For maximal speech understanding, listeners must allocate their perceptual resources to the most informative acoustic properties. Understanding such perceptual weighting strategies is essential for the design of assistive listening devices that need to preserve these important speech acoustics. This study measured the perceptual weighting strategies of young normal-hearing listeners for the envelope and fine structure in each of three frequency bands for sentence materials. Perceptual weights for these acoustic properties were obtained under three listening contexts: (1) when each acoustic property was presented individually; (2) when multiple acoustic properties were available concurrently during a continuous speech context; and (3) when multiple acoustic properties were available concurrently during an interrupted speech context. A new processing method was designed to vary the availability of each acoustic property independently by adding noise at different levels. Perceptual weights were determined by correlating a listener's performance with the availability of each acoustic property on a trial-by-trial basis. Results demonstrated that perceptual weights were: (1) equal when acoustic properties were presented individually; (2) biased toward envelope and mid-frequency information when multiple properties were available; and (3) altered during interruption at a rate of 4 Hz, corresponding to the syllabic rate of speech. Specifically, interruption interfered with mid-frequency envelope cues, whereas greater perceptual weight was placed on high-frequency envelope information. Notably, the perceptual weight for low-frequency information in either continuous or interrupted contexts was very low. Overall, listeners weight these acoustic properties differently in different listening contexts. Furthermore, these results suggest a complex interaction between the acoustic properties and the listening context in determining how to best allocate perceptual resources.