Key Technologies Of A Weapon Control System Based On Multi-channel Human-computer Interaction

With the continued development of artificial intelligence,brain-computer interface(BCI)and other technologies,the way of human-computer interaction is constantly improving.The manner in which man-computer interaction affects both real and virtual worlds.To be more natural and convenient is the development direction of human-computer interaction,as well as in the military realm.Take the example of a fighter or a tank,with more and more functions of weapon integration,the mode of operation becomes more and more complex.In the case of overload and vibration,the traditional limb operation is easy to make mistakes.In this paper,a non-limb operation mode with multi-channel of eye movement,EEG and speech is designed,and the factors that affect the accuracy of instruction judgment in each channel are studied experimentally.The main research contents and results are as follows:1.Centering on the key steps of weapon control ”environmental observation,target recognition,target selection and aiming,target confirmation,attack”,the non–limb operation mode of ”aiming with eyes,confirming with EEG and instructing with voice” is put forward,based on which the workflow and hardware composition of the system are determined.2.Focusing on the issue of ”aiming with eyes”,the subsystem for selecting and aiming the target based on eye tracking and automatic target recognition and tracking is constructed using Tkinter.To solve the problem of determining whether it is an aiming behavior,we introduce a queue to save the IDs of the targets being gazed at and set a length and a threshold to get a reliable result.An experimental study was carried out on the relationship between the queue length,the threshold and the correct rate of aiming behavior judgment.3.Focusing on the problem of “confirming with EEG”,the technical implementations are compared and analyzed.Based on Open BCI,the subsystem of SSVEP(Steady State Visually Evoked Potential)BCI is constructed,and the two-classification is realized.Based on Open BCI,the Steady State Visual Evoked Potential(SSVEP)BCI subsystem is built,and the binary classification is carried out.The relationship between the sampling time of EEG signals and the accuracy of instruction recog-nition is investigated through experiments.Furthermore,the experiments in the case of 4,6 and 8 classification based on 8 channel signals are carried out and the expansibility of the system is verified.4.Concentrate on the problem of ”instruction with voice”,the speech control scheme is analyzed,a convolution neural network for Mel spectrogram classification is built,and the speech instructions of two categories of isolated words are deter-mined,and the feasibility is verified by experiments.5.With the integration of the above subsystems,a multi-channel human-computer in-teraction control system is constructed.Under the set scene,the designed non-limb operation mode is experimentally studied to verify the feasibility of the operation mode.Through experiments,the correct completion rate of the task and the av-erage time needed to complete the task are studied,and the operation results of multi-channel mode and single-channel mode are compared and analyzed.