A Study On The Modality-switching Effect In Chinese Perception Verbs Processing

Perception verbs can reflect the conceptualization of perceptual experience,which is of great significance for exploring the specific link between perception and cognition.According to the conceptual symbol theory and the results of relevant empirical research,perceptual information can facilitate conceptual processing when verifying properties in the same modality,since conceptual processing is based on the activation of different sensorimotor systems and modality-specific perceptual simulation;otherwise,the corresponding switching costs will be generated,that is,the modality-switching effect will take effect.At present,most studies on Chinese perception verbs mainly focus on the classification and interpretation of the meanings of Chinese perception verbs from the theoretical level,while few studies investigate the conceptual processing of Chinese perception verbs through empirical means.Based on previous studies,this study selected perception verbs as the research object to further explore the modality switching effect in the processing of perception verbs by conducting a behavioral experiment from the perspective of modality switching.The specific research questions are as follows:(1)Is there a modality-switching effect in the processing of perception verbs?(2)Is there a priming modality effect in the processing of perception verbs?(3)Is there a modality salience matching effect in the processing of perception verbs?(4)Is there an interaction of modality-switching condition,priming modality and modality salience matching condition in the processing of perception verbs?This study adopted a 2(modality-switching condition: modality-matched condition,modality-switched condition)× 2(priming modality: visual priming,auditory priming)× 2(modality salience matching condition: modality salience matched condition,modality salience switched condition)three-factor within-subject experimental design.Thirty pairs of Chinese two-character visual verbs and auditory verbs were selected as experimental materials,and thirty-six students from Sichuan International Studies University were recruited as participants.The experiment adopted the lexical decision task,in which the subjects were asked to make a judgment on the part of speech of the target.Then,the recorded behavioral data were used to examine whether the modality-switching effect exists in the processing of perception verbs.The experiment was programmed with E-Prime 2.0 software to collect and record the behavioral data generated in the experiment,then analysis of the behavioral data was conducted offline,and the results were input into SPSS 26.0statistical software for statistical analysis.The experimental findings are as follows:(1)The main effect of modality-switching condition was significant,F(1,29)= 10.088,p =.004 <.01,η~2 =.258.These results indicate that there is a modality-switching effect in the processing of perception verbs,and modality-switching condition has a significant effect on the processing of perception verbs.(2)The main effect of priming modality was not significant,F(1,29)= 2.803,p =.105 >.05,η~2 =.088.These results indicate that there is no priming modality effect in the processing of perception verbs,and priming modality has no significant effect on the processing of perception verbs.(3)The main effect of modality salience matching condition was not significant,F(1,29)= 0.228,p =.637 >.05,η~2 =.008.These results indicate that there is no modality salience matching effect in the processing of perception verbs,and modality salience matching condition has no significant effect on the processing of perception verbs.(4)The interaction of modality switching condition,priming modality and modality salience condition was not significant,F(1,29)= 1.026,p =.319 >.05,η~2=.034.These results indicate that there is no interaction of modality-switching condition,priming modality and modality salience matching condition in the processing of perception verbs.However,there was an interaction between the modality-switching condition and the priming modality,F(1,29)= 12.449,p =.001<.01,η~2 =.300;The results show that modality-switching condition and priming modality interrelate in the perception verbs processing.The simple effect analyses of modality-switching condition and priming modality shows that there was no significant difference in the reaction time under modality-matched condition and modality-switched condition(MD =-16.233,p =.419 >.05)when the priming modality condition was visual priming.However,under the auditory priming condition,there was a significant difference in the reaction time under modality-matched condition and modality-switched condition(MD =-75.000,p=.000 <.001).The results of simple effect analyses also show that when the modalities of the prime and the target were consistent,there was a significant difference in the reaction time under different conditions of priming modality(MD =19.450,p =.031 <.05).When the modalities of the prime and the target were inconsistent,the reaction time also showed significant differences under different conditions of priming modality(MD =-39.317,p =.002 <.05).These results indicate that it would require higher cognitive processing costs when the target is preceded by a visual verb than by an auditory verb under the modality-matched condition.However,under the modality-switched condition,the modality-switching costs from auditory modality to visual modality were higher.Given all that,modality-switching effect exists in the processing of Chinese perception verbs.The switching of modality in semantic processing involves the activation of different sensory modalities and simulation of different perceptual experiences,thus generating corresponding modality switching costs.