Give Me Proper Cues In Multimedia Learning:The Effects Of Dual-Modality Cues On Attention Guidance And Learning Outcomes

Why do cues added in multimedia instructional materials effectively guide visual attention at a perceptual level,but not bring about a consistent improvement of learning outcomes across previous studies as well as an exact relationship between parameters of eye tracking and learning performance?One possible explanation is that gaining the advantage of eye movements is just a first step,rather than a key step to learning.It's the text-picture integration that is considered to be the most crucial processing for meaningful learning.However,the vast majority of studies have just used visual cues in the visual channel or just auditory cues in the auditory channel(i.e.,single-modality cues),which do not necessarily guarantee better text-picture integration.In the present study,we created dual-modality cues through combining visual and auditory cues.Three experiments were conducted to examine the educational effects of dual-modality cues on visual attention guidance and immediate learning outcomes,as well as their relationships.The two major questions we focused on were(a)whether or not dual-modality cues served as more scientific aids than single-modality cues(Experiment 1),and(b)why dual-modality cues were proper on text-picture integration(Experiments 2&3).In Experiment 1 students were asked to learn the transmission process of nervous impulse in chemical synapses and were randomly assigned to each of the four conditions:dual-modality cues,visual-only cues,auditory-only cues,and no cues as a control group.Eye movements were recorded during learning.Once finishing the multimedia instruction,participants were required to undertake a retention test and a transfer test.The results revealed that learners in the condition with dual-modality cues attended faster,longer,and more frequently to the text-relevant elements,and scored higher on both retention and transfer tests;the improvement of retention performance was associated with faster visual search.Although the visual attention of learners in the condition with visual-only cues was also well guided,no better learning outcomes were found.Besides,learners in the group with auditory-only cues didn't show any superiority over learners in the no-cueing group on visual attention and learning performance(except for the retention test).Accordingly,dual-modality cues were more scientific than single-modality ones.In Experiment 2 participants were randomly assigned to each of the three conditions:matched cues(i.e.,dual-modality cues in Experiment 1),mismatched cues,and no cues.The results showed that students in the group with matched cues not only fixated more quickly,longer,and more frequently at the text-relevant elements(in the right regions of the screen),rather than the text-irrelevant elements,but also performed better on learning tests than their counterparts in the control group.Conversely,the visual attention of leaners in the condition with mismatched cues was directed to the text-irrelevant element,rather than the corresponding elements,and their learning outcomes were not improved;moreover,the reduction of retention score was related to slower visual search for the text-relevant elements.Accordingly,we speculated that dual-modality cues had a potential spatial-matched processing advantage in terms of text-picture integration.In Experiment 3 participants were randomly assigned to each of the four conditions:synchronized cues(i.e.,dual-modality cues in Experiment 1),advanced cues,delayed cues,and no cues.The results indicated that learners attended faster,longer,and more frequently to the text-relevant elements,and scored higher on both retention and transfer tests when visual and auditory cues were presented synchronously;the improvement of retention performance was associated with shorter visual search latency.When visual cues were presented before or after auditory cues,however,learners didn't obtain higher post-test scores than those in the control group and their visual attention was not guided as well as learners in the synchronized-cueing group.These findings suggested that dual-modality cues had a potential temporal-synchronized processing advantage in terms of text-picture integration.An overall analysis across experiments confirmed the robust effects of proper cues(i.e.,dual-modality cues)on attention guidance and learning outcomes,whereas cues that were improper(i.e.,mismatched cues,unsynchronized cue)didn't make any sense.Our study indicates that(a)dual-modality cues not only guide attention to the text-relevant elements faster,longer,and more frequently,but also(b)facilitate the text-picture integration on account of spatial-matched and temporal-synchronized processing,thus promoting learner,retention and transfer performance.All in all,dual-modality cues can serve as proper cues.