Mobile Visual Search Based On CRBM And NetFisher

Nowadays,Mobile visual search as being the emerging field of research faces many difficulties and challenges.One challenges if that,since the mobile devices have limited calculative capacity and storage capacity,large-scale image retrieval assignments face the problem of high real-time performance and the difficulty of data storage.The other example is that,mobile devices access to the network by means of wireless network which is slow and unstable.As a result,it will cause an unbearable delay if the mobile devices send the whole query image to the remote server to retrieve similar images.And so on.Compact descriptors for visual search(CDVS)is a recently completed standard from the ISO/IEC moving pictures experts group(MPEG).The primary goal of this standard is to provide a standardized bitstream syntax to enable interoperability in the context of image retrieval applications.And this standard has made great progress in the applications of large-scale image retrieval on compacted density of image features,discriminative of image features,scalability of image features and computation complexity of image features.The CDVS retrieval pipeline has two stages.In the first stage,the visual retrieval algorithm deployed on mobile devices extraction the image features and compressed image features to aggregate the CDVS bistream.Then,mobile device send the CDVS bitstream through wireless network to the remote server.In the second stage,the server decoded the CDVS bitstream and perform the retrieval algorithm in the database.In the end,the server sends the query results to the mobile device.The retrieval algorithm performed on the server and two steps.In the first steps,it decodes the global compacted descriptors and search the nearest neighbors in the database to get the candidate set.In the second step,it uses Geometric Consistency Checks(based on RANSAC)for finding relevant database images with high precision.We bring the following two innovations in our architecture to improve the SCFV pipeline:(1)We utilize Continuous Restricted Boltzmann Machine to preform dimensionality reduction instead of PCA.(2)We introduce fisher layer in our architecture which is an end-to-end trainable system.