| In recent years,neural machine translation technology driven by the deep learning theory has developed rapidly.Consequently,the quality of translation has been significantly improved.However,the end-to-end neural machine translation meets various challenges in fusing external effective information.In view of this,this paper analyzes two key issues about the integration of external information.First,in order to explore how to effectively fuse the external modality,the paper carries out studies on the multimodal neural machine translation.Second,in order to further explore the efficient organization and access of external information,this paper studies the semi-parametric neural machine translation methods.The contributions of this paper are two-fold:(1)Aiming at the cross-modality alignment issue faced by the frontier research of image-based multimodal neural machine translation,this paper solves the phenomenon that the auxiliary visual features are regarded as redundant information by the translation model.This paper proposes a novel object-level visual context modeling framework,called OVC,for multimodal neural machine translation based on visual entity masking strategy.OVC aims to enhance the ability of visual alignment and understanding in multimodal neural machine translation.In this OVC framework,two novel auxiliary functions are proposed,namely,vision-masking loss function and vision-weighted translation objective,to highlight the attention of multimodal neural machine translation model to visual information,so as to optimize the visual text alignment ability of multimodal neural machine translation model.The experimental results and analysis on multiple multimodal machine translation benchmarks that the proposed method effectively promotes the integration of vision and text,improves the attention distribution of visual entities and source language text,and enhances the translation performance of multimodal machine translation model.(2)Retrieval-based semi-parametric neural machine translation method has played a significant role for enhancing domain adaptation.However,its efficiency is greatly limited by the semantic representation datastore designed for retrieval.The existing retrieval-based semi-parametric neural machine translation lacks in-depth analysis of semantic representation datastore.To solve this problem,this paper takes improving retrieval efficiency and translation quality as the starting point,and optimizes the efficiency of semi-parametric neural machine translation model from the two aspects of feature compression and data pruning.Specifically,this paper proposes a lightweight Compact Network learned in a contrastive-learning manner,which not only compresses semantic representations to a large extent,but also improves the separability between semantic clusters.Meanwhile,combined with the clustering signals of translation cost,this paper proposes a new data-pruning strategy to reduce the size of the large datastore.Based on the proposed methods,this paper surpasses the optimal semi-parametric neural machine translation method while reducing the semantic dimension by 90%.And it further removes redundant key pairs of 10% ～ 40% in the datastore while maintaining the translation quality.Finally,the proposed method achieves better or comparable translation performance than the state-of-the-art semi-parametric benchmark models while reducing the translation latency by up to 57%. |
