Design And Implementation Of Data Storage And Processing System For Integrated Circuit Manufacturing Data Based On Distributed Technology

With the rapid development of the integrated circuits(IC)industry,the categories of IC manufacturing equipment are constantly increasing,and the precision of these equipment is also improving.In order to more precisely control the manufacturing process and ensure the yield of chips,acquisition of a large amount of data is needed.The above-mentioned phenomena have caused the amount of data collected in the manufacturing processes to exceed the limit of conventional production system databases.This has caused degradation of the read-write efficiency and the reliability of data.In this thesis,a data transmission and processing system based on distributed technology is designed.It is designed for solving the data collection challenge faced by conventional IC manufacturing database,as well as the data processing challenge using traditional processing methods.The main contribution of this thesis is as follows:(1)This thesis analyzed the data collection modes of the equipment on the IC production line and designed a data collection and transmission module based on Kafka distributed message queue.The Kafka message queue is optimized according to the requirements for system throughput,delay performance and reliability.(2)This thesis designs a detailed table structure for Cassandra distributed database,and proposes an ordered partitioner with adjustable weights for Cassandra.The partitioner divides the nodes according to the categories of IC manufacturing data.Moreover,this thesis optimizes the Cassandra cluster according to different data types,which improves the read-write performance of the Cassandra database.(3)This thesis proposes a stream processing framework for IC manufacturing data based on Kafka Stream and using inline data processing as an example.A specific application of the stream data processing framework is demonstrated.At the same time,a batch processing framework based on Kafka stream is also proposed.On the one hand,it reduces the complexity of the entire big data cluster,on the other hand,it can take full advantage of the processing power of a Kafka cluster.Using WAT data as an example,a specific application of batch data processing framework is also demonstrated.(4)This thesis plans the deployment scheme for the entire system and applied it to deploy the system in a real production environment.At the same time,this thesis designs test plans for the system,and evaluates the overall system performance based on the tests of different modules.The test results suggested that the overall system performance meets the design requirements.