基于随机博弈和DQN算法的云原生移动目标防御决策方法

doi:10.3969/j.issn.1671-1122.2025.06.011

摘要/Abstract

摘要：

随着云原生系统中集成应用组件的复杂性不断提高，且大部分组件为开源代码，系统组件的漏洞利用已成为影响云原生安全的主要威胁之一。移动目标防御作为一种先进的动态防御机制，被广泛认为是应对该问题的有效手段。然而，在实际应用中，频繁且无序的配置转换可能会使系统运行效率和服务质量降低，进而对资源有限系统的安全性造成不利影响。为解决云原生环境中随机攻防场景下的移动目标防御决策问题，文章结合博弈理论的建模能力与深度强化学习的求解优势，提出一种基于随机博弈和DQN算法的云原生移动目标防御决策方法，实现在大规模策略空间中进行高效最优移动目标防御策略的决策，并通过仿真实验验证了文章所提方法的有效性和实用性。

关键词: 云原生, 漏洞利用, 移动目标防御, 随机博弈, DQN算法

Abstract:

With the increasing complexity of application components in cloud-native systems, and the majority of them being open-source code, vulnerabilities exploitation in these components has become one of the primary threats to cloud-native security. Moving target defense as an advanced dynamic defense mechanism is widely recognized as an effective solution to this issue. However, the frequent and disorderly configuration transitions in the practical application of moving target defense could reduce system efficiency and service quality, potentially negatively impacting the security of resource-constrained systems. To address the strategy problem of moving target defense in cloud-native stochastic attack-defense environments, this paper combined the modeling advantages of game theory and the solution capabilities of deep reinforcement learning, and proposed a cloud-native moving target defense decision-making method based on stochastic games and the DQN algorithm. The aim was to quickly make optimal moving target defense decision in a large-scale strategy space. The effectiveness and practicality of the proposed model and algorithm are verified through simulation experiments.

Key words: cloud-native, vulnerability exploitation, moving target defense, stochastic game, DQN algorithm

中图分类号:

TP309

耿致远, 许泽轩, 张恒巍. 基于随机博弈和DQN算法的云原生移动目标防御决策方法[J]. 信息网络安全, 2025, 25(6): 967-976.

GENG Zhiyuan, XU Zexuan, ZHANG Hengwei. A Decision-Making Method for Cloud-Native Moving Target Defense Based on Stochastic Games and DQN Algorithm[J]. Netinfo Security, 2025, 25(6): 967-976.

图/表 14

图1

图2

图3

图4

表1

表2

图5

图6

图7

图8

图9

图10

图11

图12

参考文献 20

[1]	China Academy of Information and Communications Technology. Cloud Computing White Paper[R]. Beijing: China Academy of Information and Communications Technology, No. 202404, 2024.
	中国信息通信研究院. 云计算白皮书[R]. 北京: 中国信息通信研究院, No. 202404,2024.
[2]	LIANG Jinrong, CHEN Lijun, LI Zhihui, et al. Container Network Performance Anomaly Detection Based on Extended Berkeley Packet Filter and Machine Learning[C]// Springer. The International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery. Heidelberg: Springer. 2022: 1403-1415.
[3]	LI Dong, YU Junqing, WEN Ruibin, et al. Endogenous Security Methods for Container Cloud Based on IPv6[J]. Netinfo Security, 2023, 23(12): 21-28.
	李冬, 于俊清, 文瑞彬, 等. 基于IPv6的容器云内生安全机制[J]. 信息网络安全, 2023, 23(12): 21-28.
[4]	AKSAKALLI I K, CELIK T, CAN A B, et al. Systematic Approach for Generation of Feasible Deployment Alternatives for Microservices[J]. IEEE Access, 2021(9): 29505-29529.
[5]	ZHOU Shucheng, LI Yang, LI Chuanrong, et al. Context-Based Abnormal Root Cause Algorithm[J]. Netinfo Security, 2024, 24(7): 1062-1075.
	周书丞, 李杨, 李传荣, 等. 基于上下文的异常根因算法[J]. 信息网络安全, 2024, 24(7): 1062-1075.
[6]	REHMAN Z, GONDAL I, GE Mengmeng, et al. Proactive Defense Mechanism: Enhancing IoT Security through Diversity-Based Moving Target Defense and Cyber Deception[EB/OL]. (2024-04-01)[2024-12-30]. https://doi.org/10.1016/j.cose.2023.103685.
[7]	XU Hanyi, CHENG Guozhen, YANG Xiaohan, et al. Multi-Dimensional Moving Target Defense Method Based on Adaptive Simulated Annealing Genetic Algorithm[J]. Electronics, 2024, 13(3): 487-503.
[8]	AWARKEH A, EI-MALKI R, REBECCHI F. Moving Target Defense for Cloud-Native Applications[C]// IEEE. 2024 27th Conference on Innovation in Clouds, Internet and Networks (ICIN). New York: IEEE, 2024: 130-137.
[9]	ZHANG Shuai, GUO Yunfei, SUN Penghao, et al. Moving Target Defense Strategy Optimization Scheme for Cloud Native Environment Based on Deep Reinforcement Learning[J]. Journal of Electronics & Information Technology, 2023, 45(2): 608-616.
	张帅, 郭云飞, 孙鹏浩, 等. 云原生下基于深度强化学习的移动目标防御策略优化方案[J]. 电子与信息学报, 2023, 45(2): 608-616.
[10]	TAN Jinglei, JIN Hui, ZHANG Hongqi, et al. A Survey: When Moving Target Defense Meets Game Theory[EB/OL]. (2023-05-01)[2024-12-30]. https://doi.org/10.1016/j.cosrev.2023.100544.
[11]	HU Yifan, XUN Peng, ZHU Peidong, et al. Network-Based Multidimensional Moving Target Defense against False Data Injection Attack in Power System[EB/OL]. (2021-08-11)[2024-12-30]. https://doi.org/10.1016/j.cose.2021.102283.
[12]	QIAN Yaguan, GUO Yankai, SHAO Qiqi, et al. EI-MTD: Moving Target Defense for Edge Intelligence against Adversarial Attacks[J]. ACM Transactions on Privacy and Security, 2022, 25(3): 1-24.
[13]	AYRAULT M, BORDE E, KUHNE U, et al. Moving Target Defense Strategy in Critical Embedded Systems: A Game-Theoretic Approach[C]// IEEE. 2021 IEEE 26th Pacific Rim International Symposium on Dependable Computing (PRDC). New York: IEEE, 2021: 27-36.
[14]	SINGLA R, REDDY N, BETTATI R, et al. Toward a Multidimensional Analysis of the National Vulnerability Database[J]. IEEE Access, 2023(11): 93354-93367.
[15]	COSTA J C, ROXO T, SEQUEIROS J B F, et al. Predicting CVSS Metric via Description Interpretation[J]. IEEE Access, 2022(10): 59125-59134.
[16]	LIN J, ADAMS B, HASSAN A E. On the Coordination of Vulnerability Fixes[J]. Empirical Software Engineering, 2023, 28(6): 151-162.
[17]	FIGUEROA-LORENZO S, AÑORGA J, ARRIZABALAGA S. A Survey of IIoT Protocols: A Measure of Vulnerability Risk Analysis Based on CVSS[J]. ACM Computing Surveys, 2021, 53(2): 1-53.
[18]	UR-REHMAN A, GONDAL I, KAMRUZZAMAN J, et al. Vulnerability Modelling for Hybrid Industrial Control System Networks[J]. Journal of Grid Computing, 2020, 18(4): 863-878.
[19]	ANAND P, SINGH Y, SELWAL A, et al. IVQFIoT: An Intelligent Vulnerability Quantification Framework for Scoring Internet of Things Vulnerabilities[EB/OL]. (2021-09-23)[2024-12-30]. https://doi.org/10.1111/exsy.12829.
[20]	LIU Xiaohu, ZHANG Hengwei, DONG Shuqin, et al. Network Defense Decision-Making Based on a Stochastic Game System and a Deep Recurrent Q-Network[EB/OL]. (2021-12-10)[2024-12-30]. https://doi.org/10.1016/j.cose.2021.102480.

应用组件	攻击者
应用组件	High	Medium	Low
Containerd	4.0	3.0	1.8
Cri-o	3.2	2.4	1.6
Docker	2.4(4.0)	0(3.0)	0(2.0)
Cumulus	4.0	3.0	1.6
Open vSwitch	3.2(4.0)	2.6(3.0)	1.8(2.0)
Cilium	2.6	1.5	0
Redis	4.0	2.8	1.2
MySQL	4.0	2.5	2.0
PostgreSQL	3.4(4.0)	1.8(3.0)	0(2.0)