无人机端侧安全多模态大模型

doi:10.3969/j.issn.1671-1122.2026.09.001

信息网络安全 ›› 2026, Vol. 26 ›› Issue (9): 1331-1340.doi: 10.3969/j.issn.1671-1122.2026.09.001

无人机端侧安全多模态大模型

边宇阳¹, 张明², 张勖¹^,³()

¹ 北京邮电大学网络空间安全学院，北京 102200
² 中国卫星发射测控系统部，北京 100120
³ 可信分布式计算与服务教育部重点实验室，北京 100876

收稿日期:2026-06-01 出版日期:2026-09-10 发布日期:2026-09-30
通讯作者: 张勖 E-mail:selina_zhangx@bupt.edu.cn
作者简介:边宇阳（2002—），男，河北，硕士研究生，主要研究方向为无人机安全|张明（1976—），男，北京，高级工程师，硕士，主要研究方向为无线网络安全、信息安全|张勖（1973—），女，上海，副教授，博士，主要研究方向为移动自组织网络安全
基金资助:
国家重点研发计划(2023YFB3107604)

Secure on-device multimodal large model for UAV

Bian Yuyang¹, Zhang Ming², Zhang Xu¹^,³()

¹ School of Cyberspace Security, Beijing University of Posts and Telecommunications, Beijing 102200, China
² China Satellite Launch and Tracking Control General, Beijing 100120, China
³ Key Laboratory of Trustworthy Distributed Computing and Service, Ministry of Education, Beijing 100876, China

Received:2026-06-01 Online:2026-09-10 Published:2026-09-30
Contact: Zhang Xu E-mail:selina_zhangx@bupt.edu.cn

摘要/Abstract

摘要：

为提升无人机安全检测的智能化程度，并解决云端大模型的隐私泄露与离线失效问题，文章设计了轻量化无人机端侧安全多模态大模型UAV-SMLLM。首先，构建了包含20000条对话实例、覆盖10种攻击类型的多模态安全检测数据集；然后，提出了基于块级角距离冗余度分析的层剪枝策略，将LLaVA-1.5-7B-HF解码器由32层压缩至24层；最后，采用LoRA微调与AWQ 4-bit量化，使模型峰值内存占用降至3.75 GB，推理速度达5.08 tokens/s。实验结果表明，该模型对欺骗攻击、暴力破解的检测准确率分别为90.75%和84.87%，意图识别F1值由0.475提升至0.7945，通用视觉能力与基线大模型持平。

关键词: 多模态大模型, 无人机安全, 模型压缩

Abstract:

To improve the intelligence level of UAV security detection and address the privacy leakage and offline failure problems of cloud-based large models, this paper designed and implemented a lightweight UAV-side security multimodal large model, UAV-SMLLM. First, a multimodal security detection dataset containing 20000 dialogue instances and covering 10 attack types was constructed. Second, a layer pruning strategy based on block-level angular distance redundancy analysis was proposed, which compressed the LLaVA-1.5-7B-HF decoder from 32 layers to 24 layers. Finally, LoRA fine-tuning and AWQ 4-bit quantization were adopted, reducing the model’s peak memory footprint to 3.75 GB and achieving an inference speed of 5.08 tokens/s. Experimental results show that the model achieves detection accuracies of 90.75% and 84.87% for spoofing attacks and brute-force attacks, respectively; the intent recognition F1 score increased from 0.475 to 0.7945; and its general visual capability is on par with that of the baseline large model.

Key words: multimodal large model, UAV security, model compression

中图分类号:

TP309

边宇阳, 张明, 张勖. 无人机端侧安全多模态大模型[J]. 信息网络安全, 2026, 26(9): 1331-1340.

Bian Yuyang, Zhang Ming, Zhang Xu. Secure on-device multimodal large model for UAV[J]. Netinfo Security, 2026, 26(9): 1331-1340.

图/表 13

图1

表1

表2

表3

图2

表4

图3

图4

表5

表6

表7

表8

表9

参考文献 24

[1]	覃睿. 再论低空经济:概念定义与构成解析[J]. 中国民航大学学报, 2023, 41(6): 59-64.
[2]	Finn A, Jia M, Li Y, et al. Detecting Stealthy GPS spoofing attack against UAVs using onboard sensors[C]// INFOCOM 2024-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS). New York: IEEE, 2024: 1-6.
[3]	Verkerken M, D’hooge L, Wauters T, et al. Towards model generalization for intrusion detection: unsupervised machine learning techniques[J]. Journal of Network and Systems Management, 2022, 30(1): 12-20. doi: 10.1007/s10922-021-09615-7
[4]	Song C, Ma L, Zheng J, et al. Audit-LLM: Multi-agent collaboration for log-based insider threat detection[EB/OL]. (2024-08-12)[2026-04-27]. https://doi.org/10.48550/arXiv.2408.08902.
[5]	Otoum Y, Asad A, Nayak A. LLM-based threat detection and prevention framework for IoT ecosystems[EB/OL]. (2025-05-13)[2026-04-27]. https://doi.org/10.48550/arXiv.2505.00240.
[6]	Chen L, Zhao H, Liu T, et al. An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models[C]// European Conference on Computer Vision. Heidelberg: Springer, 2024: 19-35.
[7]	Lin Z, Lin M, Lin L, et al. Boosting multimodal large language models with visual tokens withdrawal for rapid inference[C]// The AAAI Conference on Artificial Intelligence. Palo Alto: AAAI, 2025: 5334-5342.
[8]	Shang Y, Cai M, Xu B, et al. LLaVA-PruMerge: Adaptive token reduction for efficient large multimodal models[C]// The IEEE/CVF International Conference on Computer Vision. New York: IEEE, 2025: 22857-22867.
[9]	Lin H, Bai H, Liu Z, et al. MoPE-CLIP: Structured pruning for efficient vision-language models with module-wise pruning error metric[C]// The IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2024: 27370-27380.
[10]	Zhang Z, Pham P, Zhao W, et al. Treat visual tokens as text? But your MLLM only needs fewer efforts to see[EB/OL]. (2024-11-30)[2026-04-27]. https://doi.org/10.48550/arXiv.2410.06169.
[11]	马云舰, 鲁华伟, 成晨, 等. 基于大语言模型的小样本网络入侵检测方法[J]. 信息通信技术, 2024, 18(3): 50-56.
[12]	Blefari F, Cosentino C, Pironti F A, et al. CyberRAG: an agentic RAG cyber attack classification and reporting tool[J]. Future Generation Computer Systems, 2026(176): 86-98.
[13]	Gromov A, Tirumala K, Shapourian H, et al. The unreasonable ineffectiveness of the deeper layers[C]// International Conference on Learning Representations. Singapore: OpenReview, 2025: 81906-81920.
[14]	Lin J, Tang J, Tang H, et al. AWQ: activation-aware weight quantization for on-device LLM compression and acceleration[C]// The 7th Conference on Machine Learning and Systems. Santa Clara: MLSys, 2024: 87-100.
[15]	Hassler S, Mughal U, Ismail M. Cyber-physical dataset for UAVs under normal operations and cyber-attacks[EB/OL]. (2023-05-25)[2026-04-27]. https://doi.org/10.21227/6f22-py65.
[16]	Hassler S C, Mughal U A, Ismail M. Cyber-physical intrusion detection system for unmanned aerial vehicles[J]. IEEE Transactions on Intelligent Transportation Systems, 2024, 25(6): 6106-6117. doi: 10.1109/TITS.2023.3339728 URL
[17]	Degirmenci E, Kirca Y S, Ozcelik I, et al. ROSIDS23: network intrusion detection dataset for robot operating system[J]. Data in Brief, 2023(51): 39-48.
[18]	Whelan J, Sangarapillai T, Minawi O, et al. UAV attack dataset[EB/OL]. (2020-02-26)[2026-04-27]. https://doi.org/10.21227/00dg-0d12.
[19]	Hadi H J, Cao Y, Khan M K, et al. UAV-NIDD: a dynamic dataset for cybersecurity and intrusion detection in UAV networks[J]. IEEE Transactions on Network Science and Engineering, 2025, 12(4): 2739-2757. doi: 10.1109/TNSE.2025.3553442 URL
[20]	Stiawan D, Idris M Y B, Bamhdi A M, et al. CICIDS-2017 dataset feature analysis with information gain for anomaly detection[J]. IEEE Access, 2020(8): 132911-132921.
[21]	Wang Y, Zhao J, Fan Z, et al. JTD-UAV: MLLM-enhanced joint tracking and description framework for anti-UAV systems[C]// The Computer Vision and Pattern Recognition Conference. New York: IEEE, 2025: 1633-1644.
[22]	Biswas T K, Zannat A, Ishtiaq W, et al. A novel unified lightweight temporal-spatial transformer approach for intrusion detection in drone networks[J]. Scientific Reports, 2026, 16(1): 14473. doi: 10.1038/s41598-026-45063-6
[23]	Khan I A, Razzak I, Pi D, et al. Fed-inforce-fusion: a federated reinforcement-based fusion model for security and privacy protection of IoMT networks against cyber-attacks[J]. Information Fusion, 2024(101): 102-112.
[24]	Ullah S, Ahmad J, Khan M A, et al. A new intrusion detection system for the internet of things via deep convolutional neural network and feature engineering[J]. Sensors, 2022, 22(10): 3607-3616. doi: 10.3390/s22103607 URL

字段	单位
海拔高度	m
对地航向角	rad
经纬度	°
经度	°
速度（x/y/z）	m/s
IP标识字段	—
TCP序列号	—
捕获帧长度	B
IP地址	—
MAC地址	—
位置（x/y/z）	m
TCP标志位	—
帧序列号	—
距上一数据包时间	s
接收窗口大小	B

类型	数量/条	占比
VQA	2000	10.000%
正常流量	7200	36.000%
DoS	2700	13.500%
Spoof	3600	18.000%
ROS未授权发布	900	4.500%
邪恶孪生攻击	33	0.165%
数据假着陆攻击	133	0.665%
重放攻击	1283	6.415%
暴力破解	716	3.580%
端口探测	1282	6.410%
中间人攻击	153	0.765%

数据集	评价指标	Qwen2.5-VL-3B-Instruct	Qwen2.5-VL-7B-Instruct	llava-1.5-7b-hf
GQA	Overall	60.18%	60.50%	60.42%
	choose	83.53%	83.26%	80.80%
	compare	67.23%	69.61%	62.31%
	logical	80.03%	78.59%	75.60%
	query	43.34%	44.10%	46.13%
	verify	81.66%	81.79%	80.73%
	推理速度	1.25 tokens/s	1.19 tokens/s	2.58 tokens/s
POPE	Overall	85.99%	86.58%	83.81%
	Acc	87.28%	87.80%	85.42%
	Precision	95.70%	96.25%	94.23%
	Recall	78.07%	78.67%	75.74%
	推理速度	2.04 tokens/s	0.90 tokens/s	5.38 tokens/s

参数	取值
秩 r	16
缩放因子 α	32（α/r = 2）
Dropout	0.05
目标模块	全部线性层（Q、K、V、O、FFN）
最大序列长度	4096 tokens
学习率	5 ? 10?⁵
训练轮数/轮	2

名称	配置
GPU	搭载32个Tensor Core的1024 NVIDIA Ampere架构 GPU
CPU	8 Arm Cortex-A78AE v8.2 64 位 CPU 2MB L2 + 4 MB L3
内存	8 GB
CPU最大频率	2 GHz
GPU最大频率	918 MHz

无人机端侧安全多模态大模型

Secure on-device multimodal large model for UAV

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 13

参考文献 24

相关文章 1

编辑推荐

Metrics

本文评价

名称	配置
CPU	Intel Xeon Platinum 8255C
GPU	NVIDIA Tesla V100 NVLink 32 GB
显存	32 GB
内容	40 GB
CPU主频	2.5 GHz
CPU核数	10

数据集	评价指标	llava-1.5-7b-hf	UAV-SMLLM	差值
VQA测试集	Acc	66.30%	60.90%	-5.40%
MME	Perception	1435.17	1395.33	-1.99%
MME	Reasoning	269.29	330.36	+15.27%
POPE	Overall	83.81%	82.56%	-1.25%
	Acc	85.42%	84.61%	-0.81%
	Precision	94.23%	95.24%	+1.01%
	Recall	75.74%	72.87%	-2.87%

数据集	评价指标	llava-1.5-7b-hf	Qwen2.5-VL-3B-Instruct	UAV-SMLLM
意图识别测试集	Precision	47.38%	48.42%	80.81%
	Recall	48.23%	48.96%	78.47%
	F1	47.52%	48.59%	79.45%
安全检测测试集	Acc	40.06%	38.83%	48.22%

类型	llava-1.5-7b-hf	Qwen2.5-VL-3B-Instruct	TSLT-Net	DCNN	FCFFN	UAV-SMLLM
暴力破解	0	0	88.18%	88.18%	88.18%	84.87%
DoS攻击	0	33.00%	82.80%	90.29%	93.43%	46.78%
evil攻击	0	0	51.15%	86.39%	48.43%	47.06%
数据假着陆	0	0	0	24.14%	0	20.00%
欺骗攻击	19.74%	0	90.07%	52.36%	100%	90.75%
中间人攻击	0	0	72.31%	0.00%	72.31%	15.29%
重放攻击	0	0	46.52%	46.73%	46.17%	72.70%
端口探测	0	0	96.00%	55.73%	97.30%	0.15%
ROS未授权发布	0	0	66.23%	66.23%	66.23%	69.87%
正常流量	99.68%	99.35%	60.23%	71.83%	59.51%	25.80%