Human Action Recognition Based On Hand-crafted And Deep Learning Features

With a wide variety of applications in computer vision,human action recognition has become one of the most appealing research fields.Ambiguities of action detection derive not only from the difficulties in identifying the motion of body parts,but also from several other real-world problems such as camera motion,a complex context,and bad weather.There is a growing need to advance human action recognition systems in real-world scenarios,which motivates others to work in this field.Even though a slew of rigorous methods has been presented in the literature,they are nevertheless inadequate to address many of the issues.As a workaround,a collection of robust action descriptors are suggested,that have completed a human action recognition challenge in this dissertation.Proposed novel features are based on hand-crafted and deep learning features,proposed descriptors are CAD,ASD-R,SNSP,R-WAA,QDC,and ARK.Descriptors are evaluated by all three domains:silhouettes frames,skeletal joints points,and RGB video frames.Initially,an action descriptor is proposed that can effectively execute human action recognition tasks for silhouette frames.Action descriptor computes information like motion,spatial-temporal,diversion concerning the centroid,critical point,and keypoint detection,whereas the existing approaches lack to address these challenges.Action descriptors are developed from signature-based optical flow,signature-based corner points,and binary robust invariant scalable keypoints.These action descriptors are applied to silhouette and silhouette's skeleton frames.These aforementioned action descriptors lead to developing the concatenated action descriptor(CAD).To conduct human action recognition using three-dimensional and two-dimensional skeletal joint points is the second research objective.In the first part,an action recognition descriptor is proposed which is using only the three-dimensional skeletal joint points to perform this unsettle task.Joint's point interrelationships and frame-frame interrelationships are presented,which is a solution backbone to achieve human action recognition.Here,many joints are related to each other,and frames depend on different frames while performing any action sequence.Joints point spatial information calculates using angle,joint's sine relation,and distance features,whereas joints point temporal information estimates from frame-frame relations Angle,Sine relation,and Distance features are extracted using interrelationships of joints and frames(ASD-R).Two-dimensional skeletal joint points to perform action recognition,as a solution,two descriptors are proposed:SNSP,and R-WAA.To performed human action recognition by using a novel SNSP descriptor that acquired complex spatial information among all skeletal joints.In particular,the proposed SNSP determines combined and unite details using the super joint.Our features are calculated using the standard normal,slope,and parameter space features.The neck is proposed as a super joint,SNSP is utilizing features and a super joint.By observing the independent angle characteristics(slope),formed R-WAA.R-WAA is composed of standard normal,angle features,and parameter space features.A model termed QDC is composed of quadrangular,diversion,and convolution features.Initially,Quadrangular features are extracted by using block formation,and in each block mean,features are extracted.Diversion features are extracted by using embed diversion features,which are obtained by corner points,and strongest corner point features.Convolutional features are formed by using the Laplacian,horizontal,and vertical Laplacian derivative kernel.QDC action descriptor is composed of Quadrangular,Diversion,and Convolutional features.Lastly,deep learning-based methods are extracted and termed Action Recognition Kernels.Three different Action Recognition Kernels are proposed,each of the sizes is 7 x 7,5 x 5,and 3 x 3.A series of convolutional neural network layers are applied across each Kernel,to extract action features.It is observed that as compared to ordinary deep learning,ARK features efficient size,yet efficient action descriptor.Our approach not only achieves competitive recognition efficiency within individual datasets relative to state-of-the-art methods but also demonstrates improved generalization capabilities across multiple datasets.The robustness of the suggested descriptors is shown by studies performed on data with missing values.Proposed descriptors:CAD,ASD-R,SNSP,R-WAA,QDC,and,ARK performed efficiently,and achieved state-of-the-art performance.