Abstract:

The subject of auditory‐visual integration has become increasingly important with the introduction of high‐resolution playback devices for on‐screen object segregation. What effect does the visual location of an object have on a participant’s expectations of that object’s auditory stimulus? Several studies have looked at the phenomenon in recent years, but these studies rely on static photographic images to represent the visual scene and mostly use speech signals for testing. Building on these studies, our aim is to propose a testing method using monochromatic compositing (blue‐screen technique) to locate a music performance recorded in a studio in a number of virtual acoustical environments being assessed by a listener. In this study, the participant is asked to perceptually adjust two acoustical parameters, early‐to‐late reverberant energy ratio and reverberation time, of two music performances in five contrasting visual environments according to his or her expectations of how the room should sound given its visual appearance. This study reveals participant resiliency in the presence of forced auditory‐visual mismatch; participants are able to adjust the presented acoustic modeling algorithm in a statistically similar way despite randomized starting values for the monitored parameters. Subjective results of the experiment are presented along with objective measurements for verification.