Three-dimensional Reconstruction And Shape Analysis Of Human Feet Based On Multiple Depth Cameras

Feet are known as the ‘second heart of the human body'.They are not only responsible for walking and weight-bearing,but also closely related with the health of the other parts of the human body.The footwear plays an important role of support and protection for foot.Poor fitting shoes are one of the major causes of foot related diseases.With the improvement of the living standards,people have paid more attention to the footwear fit.To help consumer to buy the appropriate footwear has become one of the research focuses nowadays.Currently,the footwear is designed by the statistical data.The foot difference among various customers could not be concerned by this method.Besides,as online shopping become more popular,it is even more difficlut for people to find good fiting shoes online.There are mainly two solutions to this problem,namely customization and virtual fitting technology.The first option is to provide personalized service in the design process;the second option is to provide intelligent recommendation to the consumers online.In either case,the key is to obtain the foot sizes of the customers.The more accurate and detailed of the measuring result,the better of the measuring method.The more convenient,more quickly,it has the higher application value.Although the traditional manual measurement is simple,its efficiency is low and its measuring results are closed related to the operator's experience.Hence,the manual measurement is not suitalbe for these two solutions.With the popularity of sensor technology and computer-aided design,non-contact measurements have developed rapidly.Since this method can restore the three-dimensional(3D)shape of the target object by photoelectric technology,the size of the target object can be described accurately.From the technical requirements of customization and virtual fitting,this paper developed a 3D foot scan system based on multiple depth cameras.A complete algorithm was propsoed to measure the 3D foot shape automaticly.Based on the deep analysis of the foot structure,a new method was proposed to establish the relationship between 3D foot shapes and their two-dimensional(2D)foot silhouettes,via which a complete 3D foot shape can be predicted by simply inputting its two 2D silhouettes.The main contents of this paper are summarized as follows:(1)3D foot scanning system based on multiple depth camerasSeveral commercialized foot scanners have been available by which detailed 3D foot shape and dimensions could be obtained.However,these scanners are usually expensive and are mainly used in research areas such as medical applications.With the invention of the low cost RGB-depth cameras,more and more attention has been paid on using them to build a scanner.These RGB-depth cameras are able to capture the image and the depth data at video rate without additional requirements on the illumination and/or intervention of the surface texture of the subject.In this paper,a 3D foot scanning system was designed with 6 depth cameras.This paper solved the mutual interference among different depth cameras effectively,and gave the detailed steps of point cloud preprocessing and surface reconstruction.The scanning procedure can be completed within 1 second.The surface reconstruction can be completed within 10 seconds.The mean error was within 2 millimeters.(2)Fast calibration for multiple depth camerasThe key of the scanning system is to stitch the point clouds obtained form different depth cameras.There are mainly two methods: the first one is to apply non-rigid registration to different point clouds.This method has high accuracy,and can overcome the body sways during scanning.However,this method needs a large amount of calculation,and needs a long time for reconstruction.The second method is to calibrate the scanning system in advance.Although this method requires additional time for calibration,the scanning and reconstruction is fast.In most current works,the calibrations of the mult-depth cameras systems need two separated steps which is time consuming and cumbersome in operation.In this paper,an improved calibration method based on a T-shaped checkerboard was used to calculate the extrinsic parameters of the 6 depth cameras.T-shaped virtual checkerboards are introduced to further fine-tune the accuracy of calibration.We proposed a new method to index the features of cloud points by color image,which greatly reduced the computational time for refine registration.Besides,our calibration method can correct the direction deviation caused by traditional registration.(3)Automatic measurement for the 3D foot shapeOnce the 3D shape of a given foot has been captured,it is necessary to extract various foot sizes which are the foundation of the shoe-last design.Since the automatic calculation of lengths,widths,and heights depends on the measuring axis,which can be affected by the customer's foot orientation during scanning,an automatic alignment is required to adjust the orientation of the scanned foot to ensure consistency.However,current methods are sensitive to foot deformities;even a mild foot deformity will cause significantly measurement errors.Anatomical landmarks are the keys for foot measurement.Several methods were proposed to detect characteristics including edges from point cloud,mesh or surface of objects.Unfortunately,these measurements always demand landmarks to be identified and marked on the foot manually prior to the scanning.In this paper,a complete procedure was introduced to measure the foot automatically by locating the anatomical landmarks without manual intervention.A new alignment algorithm was proposed to reduce the error caused by foot deformity.To decrease the difference between manual and 3D measurement,an automatic method is firstly introduced to detect the top point of the first metatarsal bone of foot.The mean errors between the manual and automatic measurements were within 2 millimeters.(4)Predict the 3D foot shape using 2D inputsWith the rapid development of internet,the footwear industry is being reshaped because of the increasing popularity of online-shopping.The major challenge for online shoe shopping is the inability to actually try on so that potential customers have to rely on the information provided online.To improve the situation,virtual fitting technologies have been explored by various researchers.In fitting a shoe with the foot,an essential prerequisite is to acquire the 3D foot shapes accurately.One of the most accepted methods to acquire the 3D foot shape is use of laser scanning.However,these scanners are usually expensive and are mainly used in research areas.Some other methods have been proposed to reconstruct 3D foot shape from 2D images.However,these methods need high computation cost,and the robustness of their algorithms are poor.This paper proposed a new method to reconstruct the 3D foot shapes by using only their 2D inputs.Firstly,the 3D foot models were separated into a set of vertical contours.And the contours were described by Elliptical Fourier Descriptor(EFD).Then,the side and top view contours of the 3D foot shapes were also transformed into EFD.Finally,a multi-linear regression model was established between the vertical contours and the side and view contours by using the principal component analysis.Thus,a complete 3D foot shape could be generated accurately by simply inputting its two 2D images captured orthographically from the up and the side views.In our case,each individual foot shape could be predicted with a mean error of 1.27 mm for the right foot and 1.21 mm for the left foot.The average measurement errors for 11 linear dimensions and 3 girths were all less than 3 mm.