Research and Implementation of Micro-blog Keyword Extraction Method Based on Clustering

doi:10.3969/j.issn.1671-1122.2014.12.006

Abstract

Abstract: This paper presented a Micro-blog keyword extraction based on Clustering. It achieved in three steps. At first, the experiment pre-processed and breaked word on the microblogs, then used TF-IDF and TextRank algorithm to calculate word weight, according to the characteristics of short text microblogging used a combination of the two methods calculate weighting terms and extracted candidate keyword by clustering algorithm. Secondly, taked n is 2 defines the maximum probability left neighbor and maximum probability right neighbor based on the theory of n-gram language model, accordingly extended the candidate keywords into key phrases. At last, the result filtered according to the concept of accessory variety and semantic number of units in the semantics extension model. The experimental results show this method can effectively extracted the microblogs keywords and TextRank performed better than the TF-IDF when processed short text .

Key words: Key Words: micro-blog, clustering algorithm, TF-IDF, TextRank, n-gram language model

CLC Number:

TP309

SUN Xing-dong, LI Ai-ping, LI Shu-dong. Research and Implementation of Micro-blog Keyword Extraction Method Based on Clustering[J]. 信息网络安全, 2014, 14(12): 27-31.

References

[1] TURNEY P. Learning to Extract Key phr-ases from Text[R]. National Research Cou-ncil, Institute for Information Technology, T-echnical Report NRC/ERB-1057,1999.
[2] FRANK E, PAYNTER G W, WITTEN I H, et al. Domain-Specific Keyphrase Extraction[C]//IJCAI ’99: Proceedings of the Sixteenth I-nternational Joint Conference on Artificial Intelligence. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1999:668-673.
[3] WAN X, XIAO J. CollabRank: towards a c-ollaborative approach to singledocument k-eyphrase extraction[C]//Proceedings of the22nd International Conference on Computa-tional Linguistics-Volume 1, 2008:969-976.
[4] FRANTZI K, ANANIADOU S, MIMA H. Automatic recognition of multi-word terms:theC-value/NC-value method[J]. International J-ournal on Digital Libraries, 2000, 3(2):115-130.
[5] MATSUO Y, ISHIZUKA M. Keyword extr-action from a single document using wordcooccurrence statistical information[J]. Inter-national Journal on Artificial Intelligence T-ools, 2004, 13(1):157-170.
[6] TURNEY P. Learning algorithms for keyp-hrase extraction[J].Information Retrieval,2000, 2(4):303-336.
[7] HULTH A. Improved automatic keyword extraction given more linguistic knowledge[C]//Proceedings of the 2003 conference o-n Empirical methods in natural language processing,2003,2:16-223.
[8] HULTH A, DUMAIS S, MARCU D, et al.Enhancing Linguistically Oriented AutomaticKeyword Extraction[C]//HLT-NAACL 2004: S-hort,2004:17-20.
[9] TOMOKIYO T, HURST M. A language m-odel approach to keyphrase extraction[C]//Proceedings of the ACL 2003 workshop o-n Multiword expressions.Morristown, NJ, USA: Association for Computational Linguistics, 2003:33-40.
[10] BRIN S, PAGE L. The anatomy of a lar-gescale hypertextual Web search engine[J].Computer Networks and ISDN Systems, 19-98, 30(1-7):107-117.
[11] MIHALCEA R, TARAU P, L IN D, et al.TextRank: Bringing Order into Texts[C]//Pro-ceedings of EMNLP 2004, 2004:404-411.
[12] YANG J, JI D, CAI D, et al. Keyword Extraction Multi-Document Based on Joint Weight (in Chinese)[J]. Journal of Chinese Information Processing (in Chinese),2008, 22(06):75-79.
[13] WAN X, YANG J, X IAO J. Towards an Iterative Reinforcement Approach for Simul-taneous Document Summarization and Ke-yword Extraction[C]//Proceedings of the 45-th Annual Meeting of the Association of Computational Linguistics.Prague, Czech Republic: Association for Computational Lingu-istics, 2007:552-559.
[14] 中国科学院计算技术研究所. 汉语词法分析系统ICTCLAS 2009版[EB/OL]. http://ict-clas.org/,2009-05-10.
[15] STANISLAW O, STEFANOWSKI J, WEISS D. Li-ngo: Search Results Clustering Algorithm Based on Singular Value Decomposition[C]// Proc. of International Conference on Intelligent Information Systems. Springer, 2004: 359-368.