Algorithm of Chinese Keywords Extraction based on Multi-feature

doi:10.3969/j.issn.1671-1122.2014.08.007

Abstract

Abstract: In text processing area, key words has become a critical technique for a long time. Key words extraction is aimed to extract the vital words or phrases which can summarize the literature content. Considering the influence of 6 factors (such as term frequency, term length, part of speech, position, internet-dictionary and stop word list) to the weight of keywords in text, we propose a new algorithm of Chinese keywords extraction in this paper. The proposed algorithm combines linear weighting, and compound word construction and filtering. The experimental data consist of 10 categories of literature which are downloaded from China National Knowledge Infrastructure, namely environment, information technology, transportation, education, economics, culture and history, chemistry, medicine, agriculture and politics. The results show when the value of candidate words equals 5, the approximate matching precision is 54.8%, the recall rate is 65.1%. The proposed method can not only solves the problem of low recall coursed by word-segmentation in keyword extraction, but also extract words which are not high-frequency but important for the text meaning effectively.

Key words: extraction, multi-feature, weighting factor, compound word

CLC Number:

TP309

PAN Li-min, WU Jun-hua, LIN Meng, LUO Sen-lin. Algorithm of Chinese Keywords Extraction based on Multi-feature[J]. 信息网络安全, 2014, 14(8): 40-44.

References

[1] Yang W F. Chinese Keyword Extraction based on Max-duplicated Strings of the Documents [C]Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Tampere, Finland: ACM, 2002.439-440.
[2] Yang W F, Li X. PAT-TREE Based Language Model and Automatic Keyword Extraction [J]. Computer Engineering and Applications, 2001, (15): 17-19.
[3] Wang J. Updating Thesaurus via Extracting Keywords from Metadata [J]. Journal of Chinese Information Processing, 2005, 19(6): 36-43.
[4] Suo H G, Liu Y S, Cao S Y. A Keyword Selection Method Based on Lexical Chains [J]. Journal of Chinese Information Processing, 2006, 20(06): 25-30.
[5] Turney P D. Learning Algorithms for Keyphrases Extraction [J]. Information Retrieval. 2000, 4(02): 303-336.
[6] Witten I H, Paynter G W, Frank E et al. KEA: Practical Automatic Keyphrase Extraction [C]Proceedings of the 4th ACM Conference on Digital Libraries, California, USA: ACM, 1999.245-255.
[7] Li S J, Wang H F, Yu S W et al. Research on Maximum Entropy Model for Keyword Indexing [J]. Chinese Journal of Computers, 2004, (06): 1192-1197.
[8] Ohsava Y, Benson N E, Yachida M. KeyGraph: Automatic Indexing by Co-occurrence Graph based on Building Construction Metaphor [C]Proceedings of the 1998 IEEE Forum on Research and Technology Advances in Digital Libraries (IEEE ADL’98), Santa Barbara, CA : IEEE Computer Society, 1998.12-18.
[9] Matsuo Y, Ohsawa Y, Ishizuka M. KeyWorld: Extracting Keywords from a Document as a Small World [C]Proceedings of the 4th International Conference on Discovery Science, Washington DC, USA: Springer, 2001.271-281.
[10] Zhang H P, Yu K K, Xiong D Y et al. HHMM-based Chinese Lexical Analyzer ICTCLAS[C]Proceedings of the 2nd SIGHAN Workshop on Chinese Language Processing, Sapporo, Japan: ACM, 2003.184-187.
[11] Luo J, Chen L, Xia D L et al. Research on Fast Text Classifier Based on New Keywords Extraction Method[J]. Application Research of Computers, 2006, (04):32-34.
[12] Apte C, Damerau F, Weiss S M. Automated Learning of Decision Rules for Text Categorization [J]. ACM Transactions on Information Systems, 1994, 12(03): 233-251.
[13] Sun H L. Concluding Syntax Rules form Labeled Corpus [M]. Beijing: Tsinghua University Press, 1997.37-65.

[1]	HOU Liuyang, LUO Senlin, PAN Limin, ZHANG Ji. Multi-feature Android Malware Detection Method [J]. Netinfo Security, 2020, 20(1): 67-74.
[2]	KANG Jian, WANG Jie, LI Zhengxu, ZHANG Guangda. A Model for Anomaly Intrusion Detection with Different Feature Extraction Strategies in IoT [J]. Netinfo Security, 2019, 19(9): 21-25.
[3]	DUAN Yongcheng, WANG Yuqing, LI Xin, YANG Le. RSAR-based Random Forest Network Security Situation Factor Extraction [J]. Netinfo Security, 2019, 19(7): 75-81.
[4]	LI Hui, NI Shice, XIAO Jia, ZHAO Tianzhong. Emotion Classification Technology for Online Video Comments on the Internet [J]. 信息网络安全, 2019, 19(5): 61-68.
[5]	WANG Xudong, YU Xiangzhan, ZHANG Hongli. Research on Traffic Identification Technology for Unknown Protocols [J]. Netinfo Security, 2019, 19(10): 74-83.
[6]	WEN Weiping, LI Jingwei, JIAO Yingnan, LI Hailin. A Vulnerability Detection Method Based on Random Detection Algorithm and Information Aggregation [J]. 信息网络安全, 2019, 19(1): 1-7.
[7]	LU Gang, GUO Ronghua, ZHOU Ying, WANG Jun. Review of Malicious Traffic Feature Extraction [J]. 信息网络安全, 2018, 18(9): 1-9.
[8]	MA Chenxi, CHEN Xingshu, WANG Wenxian, WANG Haizhou. Chinese Event Detection Based on Recurrent Neural Network [J]. 信息网络安全, 2018, 18(5): 75-81.
[9]	DUAN Guihua, SHEN Zhuoxiang, SHEN Dongjie, LI Zhi. Research on a Download Link Recognition Scheme Based on Feature Extraction [J]. 信息网络安全, 2018, 18(10): 31-36.
[10]	QI Ben, WANG Mengdi. A Method Using Information Gain and Naive Bayes to Extract Network Situation Information [J]. 信息网络安全, 2017, 17(9): 54-57.
[11]	XU Yan. Research on Network Link Prediction Based on Data Mining [J]. 信息网络安全, 2017, 17(6): 30-34.
[12]	SONG Shunan, YANG Zhen. Research on Quantization Method of Reducing Bit Disagreement Rate in Secret Key Extraction [J]. 信息网络安全, 2017, 17(4): 46-52.
[13]	LI Hongling, ZOU Jianxin. Research of SQL Injection Detection Based on SVM and Text Feature Extraction [J]. 信息网络安全, 2017, 17(12): 40-46.
[14]	GAO Chuan, YAN Hanbing, JIA Zixiao. Research on the Method for Network Vulnerabilities Situational Awareness Based on the Features [J]. 信息网络安全, 2016, 16(12): 28-33.
[15]	QIU Yue. Research on the Hidden Web Technology for the Network Content Security [J]. 信息网络安全, 2015, 15(9): 106-108.