Abstract: With the popularization of Internet, social network has become a vital part of people's lives. Social media promotes flow and dissemination of information, but also brings a deluge of social media data and user data. Social media analysis is the main component of public opinion monitor system. Analysing and monitoring of public opinion data is one of the new technical problems caused by media in this era. In recent years, new technology such as big data processing provides proven solution to cope with the massive data . There are many big data processing platforms, in which Hadoop platform has a mature community and its structure is stable and easy to use. To text classification problems, LDA statistical modeling brings a new approach. Therefore, this paper proposes a public opinion monitoring system based on proven open source architectures. The system bases on Hadoop platform, with Nutch as a crawler, using Solr to achieve the core index search function. The entire platform demonstrates its high efficiency in the mass data processing analysis. while providing intelligent analysis and statistical functions in response to the problems caused by massive amounts of data.

Key words: public opinion monitor, crawler, search, LDA algorithm, social media