基于强化学习的物联网安全资源分配方法

doi:10.3969/j.issn.1671-1122.2022.06.005

信息网络安全 ›› 2022, Vol. 22 ›› Issue (6): 44-52.doi: 10.3969/j.issn.1671-1122.2022.06.005

基于强化学习的物联网安全资源分配方法

赵洪¹^,², 李姗², 左珮良²(), 魏占祯²

1.中国科学技术大学,合肥 230026
2.北京电子科技学院,北京 100070

收稿日期:2022-02-21 出版日期:2022-06-10 发布日期:2022-06-30
通讯作者: 左珮良 E-mail:zplzpl88@bupt.cn
作者简介:赵洪（1978—）,男,四川,博士研究生,主要研究方向为密码协议设计与分析、网络信息安全|李姗（1996—）,女,河北,硕士研究生,主要研究方向为物联网、网络信息安全|左珮良（1991—）,男,山东,讲师,博士,主要研究方向为物联网、软件定义网络、网络信息安全|魏占祯（1971—）,男,青海,正高级工程师,硕士,主要研究方向为网络信息安全
基金资助:
国家自然科学基金(62001251);国家自然科学基金(62001252);国家重点研发计划(2018YFE0200600);北京高校高精尖学科建设项目(202100130401);西安电子科技大学综合业务网理论及关键技术国家重点实验室开放课题(ISN22-13)

Security Resource Allocation Method for Internet of Things Based on Reinforcement Learning

ZHAO Hong¹^,², LI Shan², ZUO Peiliang²(), WEI Zhanzhen²

1. University of Science and Technology of China, Hefei 230026, China
2. Beijing Electronic Science and Technology Institute, Beijing 100070, China

Received:2022-02-21 Online:2022-06-10 Published:2022-06-30
Contact: ZUO Peiliang E-mail:zplzpl88@bupt.cn

摘要/Abstract

摘要：

雾计算作为一种分散式计算结构应用于物联网中,其固有的广播特性导致网络通信系统面临严重的安全威胁。同时,在动态变化的物联网环境中,无线资源的合理分配对减少通信服务时延至关重要。文章在存在不可信节点的前提下,对物联网安全资源分配问题进行研究,基于雾节点接收机具备同时同频全双工自干扰消除技术的假定,提出一种具有物理层安全特性的针对雾层无线资源的智能分配方法。该方法通过构建深度强化学习神经网络,设计合理的状态、动作和奖励等参数,实现物联网感知数据在安全保密防护条件下的快速上传。实验结果表明,该方法收敛速度较快,且在性能上明显优于对比方法。

关键词: 物联网, 强化学习, 资源分配, 物理层安全, 保密通信

Abstract:

As a decentralized computing structure, fog computing is applied in the Internet of things, and its inherent broadcast characteristics will make the network communication system to confront serious security threats. At the same time, in the dynamically changing fog Internet of things environment, the rational allocation of wireless resources is crucial to reduce the delay of communication services. Under the premise of the existence of untrusted nodes, this paper studies the security resource allocation problem of fog Internet of things. Based on the assumption that fog node receivers have simultaneous co-frequency full-duplex self-interference cancellation technology, a new method with physical layer security characteristics is proposed. It is an intelligent allocation method for radio resources in the fog layer. By constructing a deep reinforcement learning neural network and designing reasonable parameters such as state, action and reward, the method realizes the rapid upload of fog IoT perception data under the condition of security and confidentiality protection. The experimental results show that the method has a faster convergence speed and is significantly better than the comparison methods in performance.

Key words: Internet of things, reinforcement learning, resource allocation, physical layer security, secure communication

中图分类号:

TP309

赵洪, 李姗, 左珮良, 魏占祯. 基于强化学习的物联网安全资源分配方法[J]. 信息网络安全, 2022, 22(6): 44-52.

ZHAO Hong, LI Shan, ZUO Peiliang, WEI Zhanzhen. Security Resource Allocation Method for Internet of Things Based on Reinforcement Learning[J]. Netinfo Security, 2022, 22(6): 44-52.

图/表 7

图1

图2

表1

表2

图3

图4

图5

参考文献 29

[1]	OMONI B, HUSSAIN R, JAVED M A, et al. Fog/Edge Computing-Based IoT(FECIoT): Architecture, Applications, and Research Issues[J]. IEEE Internet of Things Journal, 2019, 6(3): 4118-4149. doi: 10.1109/JIOT.2018.2875544 URL
[2]	GUPTA S, GARG R, GUPTA N, et al. Energy-Effcient Dynamic Homomorphic Security Scheme for Fog Computing in IoT Networks[J]. Journal of Information Security and Applications, 2021, 58: 1-8.
[3]	TARIQ N, ASIM M, AL-OBEIDAT F, et al. The Security of Big Data in Fog-Enabled IoT Applications Including Blockchain: A Survey[J]. Sensors, 2019, 19(8): 1788-1821. doi: 10.3390/s19081788 URL
[4]	SADIQUE K M, RAHMANI R, JOHANNESSON P. Fog Computing for Trust in the Internet of Things(IoT): A Systematic Literature Review[C]// IEEE. 2020 International Conference on Computer Science, Engineering and Applications(ICCSEA). New York:IEEE, 2020: 1-6.
[5]	FERRETTI L, MARCHETTI M, COLAJANNI M. Fog-Based Secure Communications for Low-Power IoT Devices[J]. ACM Transactions on Internet Technology, 2019, 19(2): 1-21.
[6]	AMOR A B, ABID M, MEDDEB A. A Secure Fog-Based Communication Scheme[C]// IEEE. 2017 International Conference on Internet of Things, Embedded Systems and Communications(IINTEC). New York:IEEE, 2017: 146-151.
[7]	WU Yongpeng, KHISTI A, XIAO Chengshan, et al. A Survey of Physical Layer Security Techniques for 5G Wireless Networks and Challenges Ahead[J]. IEEE Journal on Selected Areas in Communications, 2018, 36(4): 679-695. doi: 10.1109/JSAC.2018.2825560 URL
[8]	ZOU Yulong, ZHU Jia, WANG Xianbin, et al. A Survey on Wireless Security: Technical Challenges, Recent Advances, and Future Trends[J]. Proceedings of the IEEE, 2016, 104(9): 1727-1765. doi: 10.1109/JPROC.2016.2558521 URL
[9]	HADDAD S, OZGUR A, TELATAR E. Can Full-Duplex More Than Double the Capacity of Wireless Networks[C]// IEEE. 2017 IEEE International Symposium on Information Theory(ISIT). New York:IEEE, 2017: 963-967.
[10]	LEI Hongjiang, YANG Zixuan, PARK K H, et al. Secrecy Outage Analysis for Cooperative NOMA Systems with Relay Selection Schemes[J]. IEEE Transactions on Communications, 2019, 67(9): 6282-6298. doi: 10.1109/TCOMM.2019.2916070 URL
[11]	WANG Zhenling, PENG Zhangyou. Secrecy Performance Analysis of Relay Selection in Cooperative NOMA Systems[J]. IEEE Access, 2019, 7: 86274-86287. doi: 10.1109/ACCESS.2019.2925380
[12]	GOMEZ G, MARTIN-VEGA F J, LOPEZ-MARTINEZ F J, et al. Physical Layer Security in Uplink NOMA Multi-Antenna Systems with Randomly Distributed Eavesdroppers[EB/OL]. (2017-09-14) [2022-01-13]. https://arxiv.org/abs/1709.04693.
[13]	FENG Youhong, YAN Shihao, YANG Zhen, et al. User and Relay Selection with Artificial Noise to Enhance Physical Layer Security[EB/OL]. (2017-12-06) [2021-01-25]. https://arxiv.org/abs/1712.02104v2.
[14]	LV Lu, DING Zhiguo, NI Qiang, et al. Secure MISO-NOMA Transmission with Artificial Noise[J]. IEEE Transactions on Vehicular Technology, 2018, 67(7): 6700-6705. doi: 10.1109/TVT.2018.2811733 URL
[15]	YAN Shihao, ZHOU Xiangyun, YANG Nan, et al. Secret Channel Training to Enhance Physical Layer Security with a Full-Duplex Receiver[J]. IEEE Transactions on Information Forensics and Security, 2018, 13(11): 2788-2800. doi: 10.1109/TIFS.2018.2834301 URL
[16]	SOHRABI R, ZHU Qiping, HUA Yingbo. Secrecy Analyses of a Full-Duplex MIMOME Network[J]. IEEE Transactions on Signal Processing, 2019, 67(23): 5968-5982. doi: 10.1109/TSP.2019.2949501 URL
[17]	MOBASHERI M, KIM Y, KIM W. Toward Developing Fog Decision Making on the Transmission Rate of Various IoT Devices Based on Reinforcement Learning[J]. IEEE Internet of Things Magazine, 2020, 3: 38-42.
[18]	FAN Qiang, ANSARI N. Towards Workload Balancing in Fog Computing Empowered IoT[J]. IEEE Transactions on Network Science& Engineering, 2020, 7: 253-262.
[19]	LIU Mengyu, LIU Yuan. Price-Based Distributed Offloading for Mobile-Edge Computing with Computation Capacity Constraints[J]. IEEE Wireless Communications Letters, 2018, 7(3): 420-423. doi: 10.1109/LWC.2017.2780128 URL
[20]	XI Linhan, WANG Ying, WANG Yang, et al. Deep Reinforcement Learning-Based Service-Oriented Resource Allocation in Smart Grids[J]. IEEE Access, 2021, 9: 77637-77648. doi: 10.1109/ACCESS.2021.3082259 URL
[21]	WEI Yifei, YU F R, SONG Mei, et al. Joint Optimization of Caching, Computing, and Radio Resources for Fog-Enabled IoT Using Natural Actor-Critic Deep Reinforcement Learning[J]. IEEE Internet of Things Journal, 2019, 6(2): 2061-2073. doi: 10.1109/JIOT.2018.2878435
[22]	WYNER A D. The Wire-Tap Channel[J]. Bell System Technical Journal, 1975, 54: 1355-1387. doi: 10.1002/j.1538-7305.1975.tb02040.x URL
[23]	LI Wei, GHOGHO M, CHEN Bin, et al. Secure Communication via Sending Artificial Noise by the Receiver: Outage Secrecy Capacity/Region Analysis[J]. IEEE Communications Letters, 2012, 16(10): 1628-1631. doi: 10.1109/LCOMM.2012.081612.121344 URL
[24]	CHEN Xing, LIU Guizhong. Energy-Efficient Task Offloading and Resource Allocation via Deep Reinforcement Learning for Augmented Reality in Mobile Edge Networks[J]. IEEE Internet of Things Journal, 2021, 8(13): 10843-10856. doi: 10.1109/JIOT.2021.3050804 URL
[25]	GOUDARZI M, WU Hanming, PALANISWAMI M S, et al. An Application Placement Technique for Concurrent IoT Applications in Edge and Fog Computing Environments[J]. IEEE Transactions on Mobile Computing, 2021, 20(4): 1298-1311. doi: 10.1109/TMC.2020.2967041 URL
[26]	ZUO Peiliang, WANG Chen, YAO Ze, et al. An Intelligent Routing Algorithm for LEO Satellites Based on Deep Reinforcement Learning[C]// IEEE. IEEE 94nd Vehicular Technology Conference(VTC2021-Fall). New York:IEEE, 2021: 1-5.
[27]	YU Yiding, WANG Taotao, LIEW S C. Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks[J]. IEEE Journal on Selected Areas in Communications, 2019, 37: 1277-1290. doi: 10.1109/JSAC.2019.2904329
[28]	GU Bo, ZHANG Xu, LIN Ziqi, et al. Deep Multi-Agent Reinforcement Learning-Based Resource Allocation for Internet of Controllable Things[J]. IEEE Internet of Things Journal, 2021, 8: 3066-3074. doi: 10.1109/JIOT.2020.3023111 URL
[29]	LANGE S, RIEDMILLER M. Deep Auto-Encoder Neural Networks in Reinforcement Learning[C]// IEEE. International Joint Conference on Neural Networks(IJCNN). New York:IEEE, 2010: 1-8.

参数名称	值
雾节点$k$数量	2个
用户$u$数量	5个
潜在的通信链路数量	10条
信道带宽B	1 MHz
信噪比${{\gamma }_{u,k}}$	5~50 dB
信道增益${{g}_{u,e}}$	-80~ -100 dB
信道增益${{g}_{k,e}}$	-80~ -100 dB
用户发送功率${{p}_{u}}$	10 mw
雾节点发送功率${{p}_{k}}$	15~35 mw

参数名称	值
探索概率$\varepsilon $	0.9~0.005
$\varepsilon $衰减速率	0.995/0.01
学习率$\alpha $	0.001
折扣因子$\gamma $	0.9
经验池大小	2000
小批次容量	32

基于强化学习的物联网安全资源分配方法

Security Resource Allocation Method for Internet of Things Based on Reinforcement Learning

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献 29

相关文章 15

编辑推荐

Metrics

本文评价

[1]	吴克河, 程瑞, 姜啸晨, 张继宇. 基于SDP的电力物联网安全防护方案[J]. 信息网络安全, 2022, 22(2): 32-38.
[2]	李桐, 任帅, 王刚, 孟庆宇. 基于变色龙认证树的云边端协同流式数据完整性验证模型[J]. 信息网络安全, 2022, 22(1): 37-45.
[3]	陈庆港, 杜彦辉, 韩奕, 刘翔宇. 基于深度可分离卷积的物联网设备识别模型[J]. 信息网络安全, 2021, 21(9): 67-73.
[4]	吴克河, 程瑞, 郑碧煌, 崔文超. 电力物联网安全通信协议研究[J]. 信息网络安全, 2021, 21(9): 8-15.
[5]	李群, 董佳涵, 关志涛, 王超. 一种基于聚类分类的物联网恶意攻击检测方法[J]. 信息网络安全, 2021, 21(8): 82-90.
[6]	刘忻, 杨浩睿, 郭振斌, 王家寅. 一种实现在线注册与权限分离的工业物联网身份认证协议[J]. 信息网络安全, 2021, 21(7): 1-9.
[7]	刘忻, 郭振斌, 宋宇宸. 一种基于SGX的工业物联网身份认证协议[J]. 信息网络安全, 2021, 21(6): 1-10.
[8]	刘璟, 张玉臣, 张红旗. 基于Q-Learning的自动入侵响应决策方法[J]. 信息网络安全, 2021, 21(6): 26-35.
[9]	王开轩, 滕亚均, 王琼霄, 王伟. 隐式证书的国密算法应用研究[J]. 信息网络安全, 2021, 21(5): 74-81.
[10]	李桐, 周小明, 任帅, 徐剑. 轻量化移动边缘计算双向认证协议[J]. 信息网络安全, 2021, 21(11): 58-64.
[11]	徐绘凯, 刘跃, 马振邦, 段海新. MQTT安全大规模测量研究[J]. 信息网络安全, 2020, 20(9): 37-41.
[12]	石润华, 石泽. 基于区块链技术的物联网密钥管理方案[J]. 信息网络安全, 2020, 20(8): 1-8.
[13]	丁举鹏, 易芝玲, 王劲涛, 阳辉. 可见光通信物理链路的基准保密容量比较研究[J]. 信息网络安全, 2020, 20(5): 65-71.
[14]	刘建伟, 韩祎然, 刘斌, 余北缘. 5G网络切片安全模型研究[J]. 信息网络安全, 2020, 20(4): 1-11.
[15]	陈璐, 孙亚杰, 张立强, 陈云. 物联网环境下基于DICE的设备度量方案[J]. 信息网络安全, 2020, 20(4): 21-30.