General Interactiing Object Detection Algorithms For Action Understanding

Video action understanding is one of important basic tasks in computer vision.Many recent researches have suggested that using the object features detected in video frames can really help boost the performance of action understanding models.However,current object detection algorithms used in video action understanding models can only detect certain classes of objects.These kinds of object detectors lack of generalizability and are not designed for video action understanding.Thus,given the existing problems in current research,this article proposed a novel problem called the general interacting object detection problem.This problem aims to detect the objects the target person is interacting with.To solve this problem,this article carries out two studies:The first one is a General Interacting Object Detector based on personobject interactions.The second one is a General Object Labels Generation Method which can improve the generalizability of object detection model.The General Interacting Object Detector can be trained to detect any interacting objects with only action labels and normal detection labels.No interacting object labels are needed.It has two key components.The first one is a Non-linear Interaction Block which adapts the Interaction Block to better modeling interactions under limited computation resources.The second one is Completeness Regional Proposal Network.This network replaces this original objectiveness branch in Reginal Proposal Network with a novel completeness branch so that the generalizability of our model can be ensured.Experiments on Kinetics 400 datasets have proven the effectiveness of our previous design choices.In order to further improve the generalizability of object detection model and reduce the bias produced from the datasets.This article proposed a General Object Labels Generation Method based on existed instance segmentation datasets.It has two novelties.The first one is a brand-new definition of general objects using silhouettes and textures.The second one is the general object labels generation method using previous definition.It can generate lots of general object labels for training as well as help improve the generalizability of detector models.Experiments with Kinetics400 dataset and MSCOCO dataset have proven the effectiveness of our proposed method.