A Contrastive Study Of Visual-Verbal Relations Via Rhetorical Structure Theory

With the development of multimedia technology, the construction of meaning depends more and more on the integration of various linguistic and non-linguistic signs, consequently more and more scholars have been attaching great importance to multimodal discourse analysis. In recent years, a lot of research has focused on the forms and meanings of different modes. Less frequently discussed are the relations between different modes. Therefore this thesis tries to explore the visual-verbal relations via Mann&Thompson’s Rhetorical Structure Theory.Rhetorical Structure Theory is a theory of natural text organization. It is widely applied to the analysis of inner relations in texts. This thesis employs it to analyze the visual-verbal relations in multimodal homepages of shopping websites. The research broadens the application scope of Rhetorical Structure Theory.By both qualitative and quantitative methods, this thesis goes into the visual-verbal relations in the homepages of two most representative shopping websites respectively in China and America:Taobao and Amazon. Through detailed observation and comparison, the author summarizes the types of images in both homepages:in Taobao’s homepage, there are icons and pictures; while in Amazon’s homepage, there are icons, pictures and star charts.Through exploring the relations between the images and their related texts, the author finds that in both homepages, there exist both single visual-verbal relation and multiple visual-verbal relation. Single visual-verbal relation refers to the relation appearing alone in multimodal discourses, while multiple visual-verbal relation means two or more than two relations appearing simultaneously in a multimodal discourse. In Taobao’s homepage, four kinds of single visual-verbal relations are discovered, namely Elaboration relation, Enablement relation, Preparation relation and Restatement relation. However, in Amazon’s homepage, only three kinds of single visual-verbal relations are detected, that is, Elaboration relation, Preparation relation and Restatement relation. In Taobao’s homepage, three kinds of multiple visual-verbal relations are sorted as well:(1) Elaboration and Restatement relation;(2) Elaboration and Preparation relation;(3) Elaboration, Preparation and Restatement relation. It follows that among the four kinds of single visual-verbal relations in Taobao’s homepage, only Enablement relation does not appear together with other visual-verbal relations. Besides, Elaboration relation, Preparation relation and Restatement relation are presented not only in configuration of single visual-verbal relation, but also in the shape of multiple visual-verbal relation. In Amazon’s homepage, two kinds of multiple visual-verbal relations are found:(1) Elaboration and Evaluation relation, and (2) Preparation and Enablement relation. What’s more, Elaboration relation and Preparation relation are constructed in both single visual-verbal relation and multiple visual-verbal relation. Restatement relation can’t shape multiple visual-verbal relation, while Enablement relation and Evaluation relation can only form multiple visual-verbal relation.Besides, this thesis probes into the status of images in multimodal discourse from the perspectives of nucleus and satellite. The findings show that in both homepages, most images are satellites. This further implies though images can express meanings by images themselves in shopping websites, most images are ancillary to their related texts.Based on the contrastive study of visual-verbal relations in both homepages, this thesis surveys two communicative functions of visual-verbal relations:"to tell" function and "to do" function."To tell" function means images and texts are used to narrate products or services."To do" function means images and texts are used to communicate for the retailer and potential customers as well. In both homepages, the "to tell" function is mainly to realize the description of price, features or names of products and so on; and the "to do" function is mainly to manifest the dual perspectives:to evaluate the purchased products and to offer channels for potential customers to buy certain products or search for information. What’s more, the "to do" function maximizes the convenience that multimedia technology brings to e-commerce. Through analysis, this thesis demonstrates the "to tell" function is mainly embedded in Restatement relation, Elaboration relation and Preparation relation, while the "to do" function is mainly implanted in Enablement relation and Evaluation relation.According to the investigation results, this thesis gets some implications:Firstly, the study shows that in the aspect of web design, Chinese e-commerce is obviously distinctive from that of the US in the followings. Amazon’s homepage emphasizes more on verbal description, therefore it has a simple design, presenting less images than Taobao’s homepage. In order to reach the goal of the promotion of products or services, Taobao takes the greatest advantage of multimedia technology and its web space by offering colorful images for potential customers to experience.Secondly, the visual-verbal relations results ultimately from the interaction between images and texts, thus these relations definitely convey business communicative functions of images and texts. These functions induce potential customers to take the purchasing actions to a certain extent. Therefore, the retailers reach their ends by e-commerce.In short, this thesis makes a contrastive study of the visual-verbal relations in Taobao’s and Amazon’s homepages, by which the author wishes to open up a new approach to multimodal discourse analysis and broaden the scope of applied research of Rhetorical Structure Theory.