Dynamic Multimodal Discourse Analysis On Informative TED Speech

With the continuous development of multimedia and the wide use of the Internet,multimodality has gradually become an important mean of daily communication.The way of communicating information transfers from single text to multimodal discourse which contains multiple patterns.The theory of multimodal discourse analysis has been developed to analyze different symbolic modes in discourse or communicative events.Its purpose is to explore the interaction of different symbolic modes in the process of constructing representation meaning,interactive meanings and compositional meaning.At present,the predecessors at home and abroad mainly focus on the perfection of theory and the study of static multimodal discourse.For dynamic images,videos and other multimodal discourse,the study of the interaction of different modalities in meaning construction is relatively small.Existing researches on dynamic multimodal discourse mainly focus on film,advertisement and so on,but seldom involve speech video discourse.TED,short for Technology,entertainment,and Design,is a series of speeches that have spread rapidly over the Internet in recent years.TED talks are designed to spread newest insights from experts in various fields around the world in the hope that people will be exposed to the latest and cutting-edge ideas.The time of TED speech video is limited within 18 minutes,and the speakers have to express their new discoveries,new ideas,new insights,to the audience within in such a short period of time.The producers of TED videos also need to streamline the production of the videos so that they can communicate the message of the videos in a short time.This paper chooses the informative speech in TED as the research object to explore how different modes in the video cooperate with each other to construct the overall meaning of the text.From the perspective of systemic functional linguistics and visual grammar,this paper studies the selected videos on the basis of Zhang Delu's comprehensive theoreticalframework of multimodal discourse analysis and multimodal video transcription criteria.Firstly,from the perspective of context,this paper explores whether cultural context and situational context have an impact on modal selection and determines the genre of video.Secondly,the distribution characteristics of modalities in different stages are analyzed at the expression level.Finally,the video is explored in detail at the content level,including how visual modality constructs representation meaning,interactive meaning and compositional meaning in the text from the meaning level,and how different modalities construct the whole meaning of the discourse through different relations at the relational level.The results show that cultural context and situational context have certain effects on modal selection.Cultural context is the key to make communicative modality possible.Cultural context determines that people adopt different communication stages and steps,namely genre,according to their specific situational context and communicative purposes.The genre of the object of this study is speech video,which can be divided into nine stages.At different stages,the distribution of different modes have little difference.The voice mode formed by the speaker's language is the main modality,which runs through the whole speech part of the video.Secondly,the visual mode formed by the text on PPT takes the second place as the auxiliary mode in the speech process.In the first and ninth stages,the visual mode is the dominant mode and the auditory mode formed by background music is the auxiliary mode.In the other stages,the voice mode formed by the speaker's language is the main mode,and the content presented on the PPT is the auxiliary mode,which helps the audience understand the content better,while the visual mode formed by the speaker itself plays a relatively small role in the construction of the overall meaning.In addition,in the analysis of modal relations,it is found that the relationship between visual modality and auditory modality is mainly complementary,and the coordination of complementary relationship is the main one,accounting for the largest proportion;in the construction of meaning,the non-complementary relationshipbetween modals mostly appears in the form of inclusion and overlapping.Through the study of TED speech video,this paper proves that Zhang Delu's dynamic multimodal discourse analysis framework is suitable for speech video in a certain extent.At the same time,it expands the research field of dynamic multimodal discourse analysis to a certain extent,and has reference value for the future analysis and research on multimodal speech discourse.