信息网络安全

信息网络安全 ›› 2022, Vol. 22 ›› Issue (10): 91-97.doi: 10.3969/j.issn.1671-1122.2022.10.013

• 入选论文 • 上一篇    下一篇

基于Hadoop的异构网络协同过滤推荐算法设计

余先荣1(), 樊捷杰2   

  1. 1.江西警察学院科技与信息安全系，南昌 330100
    2.北京科技大学计算机与通信工程学院，北京 100089
  • 收稿日期:2022-08-19 出版日期:2022-10-10 发布日期:2022-11-15
  • 通讯作者: 余先荣 E-mail:yuxianrong@jxga.edu.cn
  • 作者简介:余先荣（1983—），男，江西，讲师，硕士，主要研究方向为数据挖掘、复杂网络数据分析|樊捷杰（1985—），男，江西，博士研究生，主要研究方向为机器学习、人工智能和数据安全
  • 基金资助:
    江西省教育厅科学技术研究项目(GJJ151198)

Design of Collaborative Filtering Approach Recommendation Algorithm Based on Hadoop

YU Xianrong1(), FAN Jiejie2   

  1. 1. Department of Science Technology and Information Security, Jiangxi Police Institute, Nanchang 330100, China
    2. School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100089, China
  • Received:2022-08-19 Online:2022-10-10 Published:2022-11-15
  • Contact: YU Xianrong E-mail:yuxianrong@jxga.edu.cn

RichHTML

22

PDF (PC)

73

摘要：

在处理复杂计算任务时，来自不同人群的大量异构数据会在异构网络中产生异常值和噪声，这极易导致推荐算法性能低下。针对此类问题，文章提出了一种基于物品和用户的协同过滤个性化推荐算法，并在Hadoop平台下实现算法的分布式并行化。首先，基于皮尔森相关和余弦相似度的方法，在其相似度计算中引入评分贡献权重函数；其次，构造异构网络，通过对权重函数的设计，计算物品相似性，进而实现了离群点敏感性的钝化；最后，在Hadoop平台下，分别对基于物品和用户的异构网络的协同过滤推荐算法在影视数据集上进行验证。实验结果表明，该算法能有效提高推荐算法的准确率和实时性，改善网络监测质量，延长网络生存时间。

关键词: 协同过滤, 推荐, 异构网络, Hadoop

Abstract:

While dealing with complex computing tasks, the large number of heterogeneous data from different populations will cause abnormal values and noise in heterogeneous networks, which will lead to low performance of the recommendation algorithm easily. Thus, a personalized recommendation algorithm is proposed for such issue based on the items. Firstly, based on the Pearson correlation and cosine similarity method, the weight function of the item contribution is introduced in the similarity calculation.Secondly, according to the construction of the heterogeneous network, the similarity of two items is calculated by the design of the weight function, the insensitive performance of the outliers is realized. Finally, according to the movie data, we realized the collaborative filtering of the recommendation algorithm based on the Hadoop platform. Experimental results show that the method can effectively improve the accuracy and real-time performance of recommendation algorithm, improve the quality of network monitoring and prolong the network lifetime.

Key words: collaborative filtering, recommendation, heterogeneous network, Hadoop

中图分类号: