Research On Joint Source-Channel Coding Technology In 3D Models' Streaming Transmission

With the development of Virtual Reality technology and diverse smart terminals,3D models have been a prevalent data type transmitted in networks.To provide more realistic experience to users,the accuracy and data amount of various virtual scenes have risen sharply.However,such applications are interactive in nature,leading to high requirement of real-time performance.Streaming transmission technology has efficiently relieved the contradiction between huge data amount and limited bandwidth,making it possible to choose the model's resolution adaptively according to network conditions and the processing capacity of user terminals.However,in current networks,especially wireless networks,packet loss is unavoidable due to congestions and interference to links,which is a big challenge to fast and accurate reconstruction on the user side.Now VR videos and 3D virtual applications mostly are transmitted with TCP(Transmission Control Protocol),which is not realtime-friendly due to its congestion control mechanism and retransmission methods,thus it's not the best solution for this scenario.In this paper,both streaming transmission technology and joint source-channel coding applied to adaptive streaming are investigated,then a real-time friendly and reliable 3D model transmission scheme is proposed for streaming in lossy networks.Firstly,a 3D model is progressively encoded to a base mesh and several refinement layers based on PM(Progressive Meshes).A geometric data prediction algorithm is presented to enhance the decoding robustness.Secondly,the limitations for current unequal error protection methods to be used in 3D model based scenario are analyzed,then a new designed UEP-LT(Unequal Error Protection-Luby Transform)code with a specific coding graph is proposed to provide unequal error protection to connectivity and geometric data in one refinement layer.Finally,combining model's rate-distortion property with network conditions and the user's demand,an expected distortion mathematical model is built at the receiver side to guide the rate allocation for each refinement layer.So the reconstructed model at the receiver side will have the optimal quality theoretically.The proposed joint source-channel coding scheme can achieve UEP properties for:(1)connectivity and geometric data within one refinement layer;(2)different refinement layers,and is further compared with equal error protection(EEP)and no error protection(NEP)schemes.Finally,the new transmission scheme is compared to traditional All-TCP scheme under various network conditions.The results show that in the same time budget,the proposed scheme can achieve higher reconstruction quality at the user side under lossy network.