Real Estate Appraisal System Based On Spark Mllib

In today's information age,the explosive growth of data and the huge commercial value hidden behind these data gave birth to generation after generation of large data processing technology,in addition to the well-known Hadoop,there has developed a lot of new computing frameworks in recent years,which is represented by Spark,Storm,Flink,etc.Among them,Spark is fast,easy to develop,and take account of both batch processing and real-time data analysis,which is very suitable for machine learning algorithms.The emergence of large data processing framework break the limitations of parallel computing,so that people see the hope of solving the massive data processing problem in the information explosion era.This thesis explored a system based on Spark platform for secondhand housing appraisal.This is both a study of algorithms in the context of large data,and at the same time is a study of practical problems.Combined with the characteristics of large data and real estate appraisal,this thesis introduces the machine learning algorithm in large-scale data into the second-hand housing appraisals model,which is aimed at the existing problems of second-hand housing appraisal in the information age.First,we use the web crawler to grab the massive transaction information on the service website of second-hand housing,then extract the characteristics and establish quantitative indicators,build the second-hand housing appraisal system based on supporting vector machines,random forests and neural networks.The next step is training model through the selection of a variety of machine learning algorithms,comparing and analyzing the results of several algorithms and predict the results of integration.Finally,we use the test data to evaluate the model prediction effect.For the acquisition of massive data,the appraisal system will be extended to the Spark platform.The experimental results show that the prediction error of this model is small,the operation speed is fast,the stability is high,and our system can be applying to the housing appraisal.