基于Q-Learning的无线传感器网络生命周期平衡路由

doi:10.3969/j.issn.1671-1122.2015.04.013

信息网络安全 ›› 2015, Vol. 15 ›› Issue (4): 74-77.doi: 10.3969/j.issn.1671-1122.2015.04.013

基于Q-Learning的无线传感器网络生命周期平衡路由

苏彬庭^1,², 方禾^1,²(), 许力^1,²

1. 福建师范大学数学与计算机科学学院,福建福州 350007
2.福建省网络安全与密码技术重点实验室,福建福州 350007

收稿日期:2015-02-10 出版日期:2015-04-10 发布日期:2018-07-16
作者简介:
作者简介：苏彬庭（1990-）,男,福建,硕士研究生,主要研究方向：网络与信息安全;方禾（1991-）,女,福建,博士,主要研究方向：网络与信息安全;许力（1970-）,男,福建,博士,教授,主要研究方向：无线网络与移动通信、网络与信息安全、物联网与云计算、智能信息处理、复杂系统和网络的建模和仿真。
基金资助:
国家自然科学基金[U1405255];福建省自然科学基金[2013J01222];福州市科技项目[2013-G-84]

Q-Learning-based Routing Protocol for the Balance of WSN Lifetime

Bin-ting SU^1,², He FANG^1,²(), Li XU^1,²

1. School of Mathematics and Computer Science, Fujian Normal University, Fuzhou Fujian 350007, China
2.Fujian Provincial Key Laboratory of Network Security and Cryptology, Fuzhou Fujian 350007,China

Received:2015-02-10 Online:2015-04-10 Published:2018-07-16

摘要/Abstract

摘要：

无线传感器网络（wireless sensor network,WSN）由于容易部署和安装成本低等优势,受到学术界和工业界的广泛关注。然而无线传感器网络的节点在能量、计算能力、存储能力和带宽等方面都存在很大的局限性,复杂的传统网络路由协议无法直接应用到该网络中,因而简单高效的路由协议成为无线传感器网络的研究重点。为了延长传感器的工作时间,文章基于增强学习算法提出一种平衡无线传感器网络生命周期的路由协议Q-WRP。该协议综合考虑了节点的能量、到汇聚节点的跳数、传输时延等信息,为每个转发节点分配计算一个转发质量（即Q值）,最终根据各转发节点Q值的大小选择出最优的转发路径。NS2仿真结果表明,该算法延迟了网络第一个死亡节点的出现时间,可以有效平衡网络节点的生命周期。

关键词: 无线传感器网络, 路由协议, 增强学习

Abstract:

Wireless sensor network (WSN) is extensive concerned by academia and industry because of its good performances such as flexible deployment and low cost. But the nodes of wireless sensor network have the great limitation in the aspect of energy, computation, memory size and bandwidth, the complex routing protocols of traditional network can't be applied directly in wireless sensor network, thus a simple and efficient routing protocol became the research focus of wireless sensor network. In order to extend working hours, this paper proposes a routing protocol, Q-WRP, which can balance the wireless sensor network lifetime on the basis of reinforcement learning. The protocol takes account of the factors of residual energy, hop count to sink node, and propagation delay time, allocates Q-value for each node, and finds the optimal routing path according the Q-values of each node at last. Simulation result from NS2 shows that Q-WRP extends the occurrence time of the node that dies firstly, and can balance the wireless sensor network lifetime efficiently.

Key words: wireless sensor network, routing protocol, reinforcement learning

中图分类号:

TP309

苏彬庭, 方禾, 许力. 基于Q-Learning的无线传感器网络生命周期平衡路由[J]. 信息网络安全, 2015, 15(4): 74-77.

Bin-ting SU, He FANG, Li XU. Q-Learning-based Routing Protocol for the Balance of WSN Lifetime[J]. Netinfo Security, 2015, 15(4): 74-77.

图/表 6

参考文献 14

[1]	SK Singh, MP Singh, DK Singh.Routing protocols in wireless sensor networks-A survey[J]. International Journal of Computer Science & Engineering Survey (IJCSES), 2010, (1) : 63-83.
[2]	Halawani S, Khan AW.Sensors lifetime enhancement techniques in wireless sensor networks-a survey[J]. Computing Research Repository - CORR. 2010,(1005):34-47.
[3]	Chang J H, Tassiulas L.Maximum lifetime routing in wireless sensor networks[J].IEEE/ACM Transactions on Networking (TON),2004,12(4):609-619.
[4]	Estrin D.Wireless sensor networks tutorial part IV:sensor network protocols[C]// Proceeding of the ACM mobile Computing and Networking (Mobicom). 2002: 23-28.
[5]	Boyan JA, Littman ML.Packet routing in dynamically changing networks:A reinforcement learning approach[J]. Advances in neural information processing systems,1994,(6): 671-671.
[6]	Jafarzadeh SZ, Moghaddam MHY.Design of energy-aware QoS routing protocol in wireless sensor networks using reinforcement learning[C]//IEEE 27th Canadian Conference on Electrical and Computer Engineering (CCECE), 2014,:1-5.
[7]	Liang X, Balasingham I. SS. Byun.A reinforcement learning based routing protocol with QoS support for biomedical sensor networks[C]// International Symposium on Applied Sciences onBiomedicaland Communication Technologies (ISABEL),2008: 1-5.
[8]	Hu T, Fei Y.QELAR: a machine-learning-based adaptive routing protocol for energy-efficient and lifetime-extended underwater sensor networks[J]. IEEE Transactions on Mobile Computing, 2010,9(6):796-809.
[9]	周淑俐, 章韵, 陈志, 等. 基于Q学习的多Sink节点无线传感网路由机制研究[J]. 传感器技术学报, 2011, 9(6): 1479-1483.
[10]	Kaelbling LP, Littman ML, Moore AW.Reinforcement learning: A survey[J]. Journal of Artificial Intelligence Research (JAIR), 1996, (cs.AI/9605):237-285.
[11]	Kazemeyni F, Owe O, Johnsen EB, et al.Formal Modeling and Analysis of Learning-Based Routing in Mobile Wireless Sensor Networks[C]//Advances in Intelligent Systems and Computing: Springer International Publishing, 2014.
[12]	Tom M Mitchell.机器学习[M].曾华军,张银奎,等译.北京:机械工业出社, 2003.
[13]	Watins PD.Q-learning[J]. Machine learning, 1992,(3): 279-292.
[14]	Li R, Li F, Li X, et al.QGrid:Q-learning based routing protocol for vehicular ad hoc networks[C]//IEEE International Performance Computing and Communications Conference (IPCCC), 2014:1-8.

[1]	江金芳, 韩光洁. 无线传感器网络中信任管理机制研究综述[J]. 信息网络安全, 2020, 20(4): 12-20.
[2]	张三顺, 夏辉, 张公敬. 基于波动识别的可信组播路由协议[J]. 信息网络安全, 2018, 18(12): 72-81.
[3]	亢保元, 王佳强, 邵栋阳, 李春青. 一种适用于异构Ad Hoc无线传感器网络的身份认证与密钥共识协议[J]. 信息网络安全, 2018, 18(1): 23-30.
[4]	章红艳, 许力, 林丽美. 无线传感器网络中基于超立方体的对密钥建立方案研究[J]. 信息网络安全, 2017, 17(12): 1-5.
[5]	钟敦昊, 张冬梅, 张玉. 一种基于相似度计算的无线传感器网络入侵检测方法[J]. 信息网络安全, 2016, 16(2): 22-27.
[6]	刘伟, 叶清, 王成. 基于动态密钥管理的改进LEACH路由算法[J]. 信息网络安全, 2015, 15(8): 41-46.
[7]	龚婉婉, 刘士兴, 冯婷婷. 基于无线传感器网络的智能家居监控平台设计[J]. 信息网络安全, 2015, 15(3): 79-83.
[8]	仇各各, 汪学明, 张言胜. 基于HECC的WSN身份认证协议研究[J]. 信息网络安全, 2015, 15(12): 54-58.
[9]	杨骏玮, 方杰, 张仕斌, 陈建钧. WSNs中基于重复博弈的信任评价方法研究[J]. 信息网络安全, 2014, 14(8): 82-87.
[10]	. 基于NS的机会网络路由协议仿真[J]. , 2014, 14(5): 52-.
[11]	佟鑫;张利;戴明. 物联网感知层安全威胁建模研究[J]. , 2013, 13(Z): 0-0.
[12]	肖亮;马春光;王九如;高训兵. 一种基于随机线性网络编码的机会网络路由协议[J]. , 2013, 13(3): 0-0.
[13]	郎为民;陈林;王昊. 无线传感器网络云连接认证研究[J]. , 2013, 13(3): 0-0.
[14]	高立兵. 基于无线传感器网络的井下定位系统研究[J]. , 2013, 13(2): 0-0.
[15]	张婷婷;夏戈明;吴伟彬. 安全与应急响应的监测型移动传感器网络系统[J]. , 2013, 13(11): 0-0.

基于Q-Learning的无线传感器网络生命周期平衡路由

Q-Learning-based Routing Protocol for the Balance of WSN Lifetime

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 6

参考文献 14

相关文章 15

编辑推荐

Metrics

本文评价