基于稠密连接的深度修复定位网络

doi:10.3969/j.issn.1671-1122.2022.07.010

摘要/Abstract

摘要：

图像修复是计算机视觉中的一个经典应用。基于深度学习的修复算法可以用较低的成本生成逼真的修复图像。然而,这种强大的算法有潜在的非法或不道德用途,如删除图像中的特定对象以欺骗公众。尽管目前出现许多图像修复的取证方法,但在复杂的修复图像中,这些方法的检测能力仍然有限。基于此,文章提出使用稠密连接的网络有效定位逼真的深度修复图像中的篡改区域。该网络是一种基于稠密连接的编码器和解码器架构,其中引入的稠密连接模块可以更好地捕获在修复图像中细微的篡改痕迹。此外,在稠密连接模块中嵌入Ghost模块、空洞卷积和通道注意力机制可以实现更好的定位性能。实验结果表明,该方法能够在逼真复杂的深度修复图像中有效地识别出篡改区域,并且能够满足对JPEG压缩和旋转的鲁棒性需求。

关键词: 深度修复, 篡改检测, 稠密连接, Ghost模块

Abstract:

Reconstructing the missing regions of an image is a typical requirement in computer vision. With deep inpainting algorithms, one can generate realistic inpainted images at a very low cost. However, such a powerful tool has potentially illegal or unethical uses, such as removing specific objects from images to deceive the public. Although many forensic methods for image inpainting have been proposed, their detection capabilities are still limited in complex inpainted images. Motivated by that, this paper proposed an efficient network based on dense connectivity to locate tampered regions in a realistic deep inpainting image. The network was an encoder-decoder architecture based on dense connectivity, where the introduced dense connected module can better capture subtle manipulation traces in realistic inpainted images. Furthermore, embedding the Ghost modules, dilated convolutions, and the channel attention mechanism in dense connected blocks could achieve better localization performance.Experiments demonstrate that the proposed method can effectively locate the inpainted regions in sophisticated deep inpainting images, and also show that the method fulfilling the robustness requirements of JPEG compression and rotation.

Key words: deep inpainting, forgery detection, dense connectivity, Ghost module

中图分类号:

TP309

傅志彬, 祁树仁, 张玉书, 薛明富. 基于稠密连接的深度修复定位网络[J]. 信息网络安全, 2022, 22(7): 84-93.

FU Zhibin, QI Shuren, ZHANG Yushu, XUE Mingfu. Localization Network of Deep Inpainting Based on Dense Connectivity[J]. Netinfo Security, 2022, 22(7): 84-93.

图/表 15

图1

图2

图3

图4

图5

表1

表2

图6

图7

表3

图8

图9

图10

表4

图11

参考文献 29

[1]	KORUS P, HUANG Jiwu. Multi-Scale Analysis Strategies in PRNU-Based Tampering Localization[J]. IEEE, 2017, 12(4): 809-824.
[2]	FERRARA P, BIANCHI T, ROSA A D, et al. Image Forgery Localization via Fine-Grained Analysis of CFA Artifacts[J]. IEEE, 2012, 7(5): 1566-1577.
[3]	ZHAO Jie, GUO Jichang, ZHANG Yan, et al. Automatic Detection and Localization of Image Forgery Regions Based on Offset Estimation of Double JPEG Compression[J]. Journal of Image and Graphics, 2015, 20(10): 1304-1312.
	赵洁, 郭继昌, 张艳, 等. JPEG图像双重压缩偏移量估计的篡改区域自动检测定位[J]. 中国图象图形学报, 2015, 20(5):1304-1312.
[4]	WU Xueming, FANG Zhen. Image Splicing Detection Using Illuminant Color Inconsistency[C]// IEEE. 2011 Third International Conference on Multimedia Information Networking and Security. New York: IEEE, 2011: 600-603.
[5]	JIANG Xiaoyu, LIU Chunxiao. Edge and Region Inconsistency-Guided Image Splicing Tamper Detection Network[J]. Journal of Image and Graphics, 2021, 26(10): 2411-2420.
	蒋小玉, 刘春晓. 边缘与区域不一致性引导下的图像拼接篡改检测网络[J]. 中国图象图形学报, 2021, 26(10):2411-2420.
[6]	LI Peng, LI Shuaijie, YAO Zhengan, et al. Two Anisotropic Fourth-Order Partial Differential Equations for Image Inpainting[J]. IET Image Processing, 2013, 7(3): 260-269. doi: 10.1049/iet-ipr.2012.0592 URL
[7]	LIU Yunqiang, CASELLES V. Exemplar-Based Image Inpainting Using Multiscale Graph Cuts[J]. IEEE, 2013, 22(5): 1699-1711.
[8]	WU Yue, ABDALMAGEED W, NATARAJAN P. ManTra-Net: Manipulation Tracing Network for Detection and Localization of Image Forgeries with Anomalous Features[C]// IEEE. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New York: IEEE, 2019: 9535-9544.
[9]	LI Haodong, HUANG Jiwu. Localization of Deep Inpainting Using High-Pass Fully Convolutional Network[C]// IEEE. 2019 IEEE/CVFInternational Conference on Computer Vision (ICCV). New York: IEEE, 2019: 8300-8309.
[10]	WU Haiwei, ZHOU Jiantao. IID-Net: Image Inpainting Detection Network via Neural Architecture Search and Attention[J]. IEEE, 2022, 32(3): 1172-1185.
[11]	HUANG Gao, LIU Zhang, MAATEN L V D, et al. Densely Connected Convolutional Networks[C]// IEEE. 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2017: 4700-4708.
[12]	HAN Kai, WANG Yunhe, TIAN Qi, et al. GhostNet: More Features From Cheap Operations[C]// IEEE. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2020: 1577-1586.
[13]	YU F, KOLTUN V. Multi-Scale Context Aggregation by Dilated Convolutions[EB/OL]. (2016-04-30)[2022-04-04]. https://arxiv.org/abs/1511.07122.
[14]	HU Jie, SHEN Li, SUN Gang. Squeeze-and-Excitation Networks[C]// IEEE.2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2018: 7132-7141.
[15]	PATHAK D, KRAHENBUHL P, DONAHUE J, et al. Context Encoders: Feature Learning by Inpainting[C]// IEEE. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2016: 2536-2544.
[16]	IIZUKA S, SIMO-SERRA E, ISHIKAWA H. Globally and Locally Consistent Image Completion[J]. ACM, 2017, 36(4): 1-14.
[17]	YAN Zhaoyi, LI Xiaoming, LI Mu, et al. Shift-Net: Image Inpainting via Deep Feature Rearrangement[C]// Springer. Proceedings of the European Conference on Computer Vision(ECCV). Heidelberg:Springer, 2018: 1-17.
[18]	YU Jiahui, LIN Zhe, YANG Jimei, et al. Free-Form Image Inpainting with Gated Convolution[C]// IEEE.2019 IEEE/CVF International Conference on Computer Vision (ICCV). New York: IEEE, 2019: 4470-4479.
[19]	WU Haiwei, ZHOU Jiantao, LI Yuanman. Deep Generative Model for Image Inpainting with Local Binary Pattern Learning and Spatial Attention[EB/OL]. (2020-09-02)[2022-04-05]. https://arxiv.org/abs/2009.01031.
[20]	YU Tao, GUO Zongyu, JIN Xin, et al. Region Normalization for Image Inpainting[C]// ECCV.Proceedings of the AAAI Conference on Artificial Intelligence. New York: ECCV, 2020: 12733-12740.
[21]	YU Yingchen, ZHAN Fangneng, WU Rongliang, et al. Diverse Image Inpainting with Bidirectional and Autoregressive Transformers[C]// ACM. Proceedings of the 29th ACM International Conference on Multimedia. New York: ACM, 2021: 69-78.
[22]	WU Qiong, SUN Shaojie, ZHU Wei, et al. Detection of Digital Doctoring in Exemplar-Based InpaintedImages[C]// IEEE. 2008 International Conference on Machine Learning and Cybernetics. New York: IEEE, 2008: 1222-1226.
[23]	CHANGC I, YU C J, CHANG C C. A Forgery Detection Algorithm for Exemplar-Based Inpainting Images Using Multi-Region Relation[J]. Image and Vision Computing, 2013, 31(1): 57-71. doi: 10.1016/j.imavis.2012.09.002 URL
[24]	ZHAN Yuqian, LIAO Miao, SHIH F Y, et al. Tampered Region Detection of Inpainting JPEG Images[J]. Optik, 2013, 124(16): 2487-2492. doi: 10.1016/j.ijleo.2012.08.018 URL
[25]	LIANG Zaoshan, YANG Gaobo, DING Xiangling, et al. An Efficient Forgery Detection Algorithm for Object Removal by Rxemplar-Based Image Inpainting[J]. Journal of Visual Communication and Image Representation, 2015, 30: 75-85. doi: 10.1016/j.jvcir.2015.03.004 URL
[26]	ZHU Xinshan, SUN Biao. A Deep Learning Approach to Patch-Based Image Inpainting Forensics[J]. Signal Processing: Image Communication, 2018, 67: 90-99. doi: 10.1016/j.image.2018.05.015 URL
[27]	LI Haodong, LUO Weiqi, HUANG Jiwu. Localization of Diffusion-Based Inpainting in Digital Images[J]. IEEE Transactions on Information Forensics and Security, 2017, 12(12): 3050-3064. doi: 10.1109/TIFS.2017.2730822 URL
[28]	ODENA A, DUMOULIN V, OLAH C. Deconvolution and Checkerboard Artifacts[EB/OL]. (2016-10-17)[2022-04-05]. https://distill.pub/2016/deconv-checkerboard/.
[29]	ZHOU Bolei, LAPEDRIZA A, KHOSLA A, et al. Places: A 10 Million Image Database for Scene Recognition[J]. IEEE, 2017, 40(6): 1452-1464.

模块名称	核大小	步长	输出维度
稠密连接模块	3	1	16
平均池化层	2	2	16
Ghost-dense模块	3	1	32
平均池化层	2	2	32
Ghost-dense模块	3	1	64
平均池化层	2	2	64
具备通道注意力机制的Dilated-dense模块	3	96	96

模块名称	核大小	步长	输出维度
转置卷积层	4	2	64
具备通道注意力机制的Dilated-dense模块	3	1	64
转置卷积层	4	2	48
具备通道注意力机制的Dilated-dense模块	3	1	48
转置卷积层	4	2	32
卷积层	5	1	2

		LDI	Patch-CNN	HP-FCN	IID-Net	本文方法
Handcrafted dataset	F1	15.85%	35.41%	57.14%	81.40%	84.80%
Handcrafted dataset	AUC	48.73%	66.73%	68.88%	95.40%	95.92%
Random dataset	F1	10.49%	31.28%	41.52%	77.69%	79.40%
Random dataset	AUC	49.32%	73.75%	66.38%	95.45%	96.66%

	LDI/s	Patch-CNN/s	HP-FCN/s	IID-Net/s	本文方法/s
300张图像运行时间	111.8357	10.1168	19.0127	75.2662	18.0635
1张图像的平均运行时间	0.3728	0.0337	0.0634	0.2509	0.0602