Reconstructing 3D Motion Trajectory Of Particle Swarms

Phenomena such as insect swarms, bird flocks, and fish schools occur preva-lently in our environments. They have attracted significant attention by scientists in many disciplines [58,51,4]. The availability of the 3D motion trajectory for each individual in the swarm can greatly facilitate the study of animal collec-tive behavior. In order to reconstruct such 3D scenes made up of large number of particles, we first proposed an approach-Relative Epipolar Motion(REM)-towards solving the correspondence problem in stereopsis by utilizing the motion clue. Using the relative epipolar motion cues, it can reconstruct dynamic scenes of large number of indistinguishable moving objects. It offers an alternative way to project structured lights in active mode for deforming surface reconstruction. The REM method however strongly relies on the performance of tracking on 2D image planes. If tracking failed, it will cause significant reconstruction error. Noticing stereo matching and tracking are in fact coupled, we proposed a global optimiza-tion method that treats a possible corresponding feature points as a'pairing'. The trajectory reconstruction problem becomes a process of pairing selection. Through minimizing a cost function, we obtain the optimum pairing sequences. The trajec-tories are then reconstructed from these sequences by triangulation. Experiments show the significant advantage of the proposed method over existing methods. We used the proposed method to reconstruct the 3D trajectories of about hundreds of fruit flies simultaneously flying in a glass box. We were the first to our best knowledge that obtain complete 3D trajectories of a swarm of hundreds of flying fruit flies, enabling biologists to conduct thorough study of collective behavior of animals.