Design And Implementation Of Kernel Mode Key-value Storage System Based On Hybrid Storage

With the rapid development of cloud computing and big data processing technology and the gradual improvement of infrastructure,human society is increasingly dependent on digital and intelligent production and lifestyle.Digitization and intelligence have greatly improved social productivity,and at the same time,the speed of data generation in society has become faster and faster.This puts forward three requirements for storage systems: large capacity,low cost,and high performance.However,for any specific storage device,such as magnetic disk(HDD),solid-state disk(SSD)and non-volatile memory(NVM),these three requirements cannot be satisfied at the same time,and often only one of them.In this thesis,different storage media are combined,so that multiple storage media can learn from each other's strengths and weaknesses,and use software scheduling to present a storage system that simultaneously meets the three requirements of large capacity,low cost and high performance.The file system is an important part of the computer operating system and has always been the best choice for computers to manage user data.However,the file system's numerous functional requirements make its performance low under certain workloads.Most of the existing key-value storage systems choose to build on the specific file system provided by the operating system,which makes the path of user data from the application to the disk too long,and the file system metadata operation introduces more random reads.write,further reducing the performance of key-value storage systems.Finally,the log function of the file system itself overlaps with the log function of the key-value system itself,and a lot of useless work has been done,resulting in a waste of resources.This thesis attempts to bypass the file system and build the key-value storage system directly on the raw disk,saving a lot of overhead at the file system layer.Internet services include a variety of business scenarios,and the data characteristics generated by different business scenarios are also different.Among them,unstructured data accounts for a large proportion of the data generated by social production and life.As a storage system friendly to unstructured data,key-value storage has a wide range of application scenarios.However,existing key-value storage systems are often built in user mode,and access to data on the device requires frequent mode switching and process context switching through system calls.However,each mode switch and context switch will be accompanied by a large number of hardware contexts being pushed into and out of the stack,which seriously reduces the efficiency.In this thesis,the main body of the key-value storage system is implemented in the kernel mode,exposed as a character device,and the message queue shared by the user mode and the kernel mode achieves the purpose of reading and writing data without system calls,thereby improving the performance of the system.