Abstract:

To improve the intelligence level of UAV security detection and address the privacy leakage and offline failure problems of cloud-based large models, this paper designed and implemented a lightweight UAV-side security multimodal large model, UAV-SMLLM. First, a multimodal security detection dataset containing 20000 dialogue instances and covering 10 attack types was constructed. Second, a layer pruning strategy based on block-level angular distance redundancy analysis was proposed, which compressed the LLaVA-1.5-7B-HF decoder from 32 layers to 24 layers. Finally, LoRA fine-tuning and AWQ 4-bit quantization were adopted, reducing the model’s peak memory footprint to 3.75 GB and achieving an inference speed of 5.08 tokens/s. Experimental results show that the model achieves detection accuracies of 90.75% and 84.87% for spoofing attacks and brute-force attacks, respectively; the intent recognition F1 score increased from 0.475 to 0.7945; and its general visual capability is on par with that of the baseline large model.

Key words: multimodal large model, UAV security, model compression