Detection Algorithm of Tamper and Deepfake Image Based on Feature Fusion

doi:10.3969/j.issn.1671-1122.2021.08.009

Abstract

Abstract:

Nowadays, malicious tampering and forgery images show an explosive growth trend. The existing image tampering detection methods generally have the problems of single application scope and low detection accuracy. To solve these problems, this paper proposes a tampering and forgery image classification detection network based on image texture features. For the first time, it combines the first step edge texture image of Cb and Cr channel through Scharr operator with the second step edge texture image of G channel through Laplacian operator. Gray Level Co-occurrence Matrix (GLCM) is used to extract the features of texture image. Finally, the tampering and forgery are monitored by EfficientNet. Experiments on various image tampering and deep forgery datasets show that the model has wide applicability and high detection accuracy in both types of detection, and the classification detection accuracy of images generated by various Deepfake algorithms can reach 99.9%.

Key words: image tamper detection, Deepfake, deep learning, EfficientNet

CLC Number:

TP309

ZHU Xintong, TANG Yunqi, GENG Pengzhi. Detection Algorithm of Tamper and Deepfake Image Based on Feature Fusion[J]. Netinfo Security, 2021, 21(8): 70-81.

Figures/Tables 24

References 44

[1]	KORSHUNOVA I, SHI W, DAMBRE J, et al. Fast Face-swap Using Convolutional Neural Networks[C]//IEEE. Proceedings of the IEEE International Conference on Computer Vision, October 13-16, 2017, Nice, France. New York: IEEE, 2017: 3677-3685.
[2]	FAN Shenghao, DENG Jinxiang, SUN Jianjun. Preliminary Study on Authenticity Identification of Photos[J]. Guangdong Public Security Science and technology, 2004, 20(2):70-71.
	范胜浩, 邓金祥, 孙建军. 照片真伪鉴定初探[J]. 广东公安科技, 2004, 20(2):70-71.
[3]	BAYAR B, STAMM M C. Constrained Convolutional Neural Networks: A New Approach Towards General Purpose Image Manipulation Detection[J]. IEEE Transactions on Information Forensics and Security, 2018, 13(11):2691-2706. doi: 10.1109/TIFS.2018.2825953 URL
[4]	RAO Yuan, NI Jiangqun. A Deep Learning Approach to Detection of Splicing and Copy-move Forgeries in Images[C]//IEEE. 2016 IEEE International Workshop on Information Forensics and Security (WIFS), December 4-7, 2016, Abu Dhabi, United Arab Emirates. New York: IEEE, 2016: 1-6.
[5]	FRIDRICH J, KODOVSKY J. 1 Rich Models for Steganalysis of Digital Images[J]. IEEE Transactions on Information Forensics and Security, 2012, 7(3):868-882. doi: 10.1109/TIFS.2012.2190402 URL
[6]	ZHOU Peng, HAN Xinting, MORARIU V I, et al. Learning Rich Features for Image Manipulation Detection[C]//IEEE. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 18-23, 2018, Salt Lake City, UT, USA. New York: IEEE, 2018: 1053-1061.
[7]	HAN Yuchen, HUA Guang, ZHANG Haijian. Eye and Mouth Region Cooperative Video Face Changing Forgery Detection Based on Inception3D Network[J]. Signal Processing, 2021, 37(4):567-577
	韩语晨, 华光, 张海剑. 基于Inception3D网络的眼部与口部区域协同视频换脸伪造检测[J]. 信号处理, 2021, 37(4):567-577.
[8]	NATARAJ L, MOHAMMED T M, MANJUNATH B S, et al. Detecting GAN Generated Fake Images Using Co-occurrence Matrices[J]. Electronic Imaging, 2019, 2019(5):361-367.
[9]	DONG Jing, WANG Wei, TAN Tieniu. Casia Image Tampering Detection Evaluation Database[C]//IEEE. 2013 IEEE China Summit and International Conference on Signal and Information Processing, July 6-10, 2013, Beijing, China. New York: IEEE, 2013: 422-426.
[10]	KORUS P, HUANG Jiwu. Evaluation of Random Field Models in Multi-modal Unsupervised Tampering Localization[C]//IEEE. 2016 IEEE International Workshop on Information Forensics and Security (WIFS), December 4-7, 2016, Abu Dhabi, United Arab Emirates. New York: IEEE, 2016: 1-6.
[11]	KARRAS T, AILA T, LAINE S, et al. Progressive Growing of Gans for Improved Quality, Stability, and Variation[EB/OL]. https://arxiv.org/abs/1710.10196v3 , 2018-02-26.
[12]	KARRAS T, LAINE S, AILA T. A Style-based Generator Architecture for Generative Adversarial Networks[C]//IEEE. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 15-20, 2019, Long Beach, CA, USA. New York: IEEE, 2019: 4401-4410.
[13]	VISHWANATH D, GIRSHICK A R, BANKS M S. Why Pictures Look Right when Viewed from the Wrong Place[J]. Nature Neuroscience, 2005, 8(10):1401-1410. doi: 10.1038/nn1553 URL
[14]	JOHNSON M K, FARID H. Exposing Digital Forgeries by Detecting Inconsistencies in Lighting[C]//ACM. Proceedings of the 7th Workshop on Multimedia and Security, August 1-2, 2005, New York, NY, USA. New York: Association for Computing Machinery, 2005: 1-10.
[15]	KEE E, O'BRIEN J F, FARID H. Exposing Photo Manipulation from Shading and Shadows[J]. ACM Trans actions on. Graph ics, 2014, 33(5):1-21.
[16]	FRIEDMAN G L. The Trustworthy Digital Camera: Restoring Credibility to the Photographic Image[J]. IEEE Transactions on Consumer Electronics, 1993, 39(4):905-910. doi: 10.1109/30.267415 URL
[17]	LIU Cai. Using Exif Information Content to Identify Digital Photos[J]. China Forensic Expertise, 2010(5):49-51.
	刘猜. 利用Exif信息内容鉴定数码照片[J]. 中国司法鉴定, 2010(5):49-51.
[18]	HARMSEN J J. Steganalysis of Additive Noise Modelable Information[C]//SPIE. The International Society for Optical Engineering 2003, June 20-24, 2003, Santa Clara, CA, United States. California: SPIE, 2003: 1-12.
[19]	BAYAR B, STAMM M C. A Deep Learning Approach to Universal Image Manipulation Detection Using a New Convolutional Layer[C]//ACM. Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security, June 20-22, 2016, Vigo Galicia, Spain. New York: Association for Computing Machinery, 2016: 5-10.
[20]	TIAN Xiuxia, LI Huaqiang, ZHANG Qin, et al. Image Tampering Detection Model Based on Dual-channel R-FCN[J]. Chinese Journal of Computers, 2021, 44(2):370-383.
	田秀霞, 李华强, 张琴, 等. 基于双通道R-FCN的图像篡改检测模型[J]. 计算机学报, 2021, 44(2):370-383.
[21]	ZHANG Kejun, LIANG Yu, ZHANG Jianyi, et al. No One Can Escape: A General Approach to Detect Tampered and Generated Image[J]. IEEE Access, 2019, 2019(7):129494-129503.
[22]	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative Adversarial Networks[J]. Communications of the ACM, 2020, 63(11):139-144. doi: 10.1145/3422622 URL
[23]	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet Classification with Deep Convolutional Neural Networks[J]. Communications of the ACM, 2017, 60(6):84-90. doi: 10.1145/3065386 URL
[24]	PEARLMUTTER B A. Gradient Calculations for Dynamic Recurrent Neural Networks: A Survey[J]. IEEE Transactions on Neural Networks, 1995, 6(5):1212-1228. doi: 10.1109/72.410363 URL
[25]	ZHOU Peng, HAN Xintong, MORARIU V I, et al. Two-stream Neural Networks for Tampered Face Detection[C]//IEEE. 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). July 21-26, 2017, Honolulu, HI, USA, New York: IEEE, 2017: 1831-1839.
[26]	LI Yuezun, LYU Siwei. Exposing DeepFake Videos By Detecting Face Warping Artifacts[EB/OL]. https://arxiv.org/abs/1811.00656v3 , 2019-05-22.
[27]	BARNI M, KALLAS K, NOWROOZI E, et al. CNN Detection of GAN-generated Face Images Based on Cross-band Co-occurrences Analysis[C]//IEEE. 2020 IEEE International Workshop on Information Forensics and Security (WIFS), December 6-11, 2020, New York, NY, USA. New York: IEEE, 2020: 1-6.
[28]	LI Haodong, LI Bin, TAN Shunguan, et al. Identification of Deep Network Generated Images Using Disparities in Color Components[J]. Signal Processing, 2020, 174(4):107616. doi: 10.1016/j.sigpro.2020.107616 URL
[29]	MARRA F, GRAGNANIELLO D, VERDOLIVA L, et al. Do GANs Leave Artificial Fingerprints?[C]//IEEE. 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), March 28-30, 2019, San Jose, CA, USA. New York: IEEE, 2019: 506-511.
[30]	YU Ning, DAVIS L S, FRITZ M. Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints[C]//IEEE. Proceedings of the IEEE/CVF International Conference on Computer Vision, October 27-November 2, 2019, Seoul, Korea (South). New York: IEEE, 2019: 7556-7566.
[31]	AMERINI I, GALTERI L, CALDELLI R, et al. Deepfake Video Detection through Optical Flow Based CNN[C]//IEEE. Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, October 27-28, 2019, Seoul, Korea (South). New York: IEEE, 2019: 1205-1207.
[32]	GÜERA D, DELP E J. Deepfake Video Detection Using Recurrent Neural Networks[C]//IEEE. 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), November 27-30, 2018, Auckland, New Zealand. New York: IEEE, 2018: 1-6.
[33]	HAN Yuchen, HUA Guang, ZHANG Haijian. Collaborative Video Face-changing Forgery Detection Based on the Inception3D Network in the Eye and Mouth Regions[J]. Signal Processing, 2021, 37(4):567-577.
	韩语晨, 华光, 张海剑. 基于Inception3D网络的眼部与口部区域协同视频换脸伪造检测[J]. 信号处理, 2021, 37(4):567-577.
[34]	HARALICK R M, SHANMUGAM K, DINSTEIN I H. Textural Features for Image Classification[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1973, 3(6):610-621.
[35]	SCHADE O H. On the Quality of Color-television Images and the Perception of Color Detail[J]. Journal of the SMPTE, 1958, 67(12):801-819. doi: 10.5594/J16981 URL
[36]	ZHANG Jianyong, JIN Weiqi, ZHOU Yan, et al. Analysis of Human Eye Color Detection Characteristics Based on Colorimetry[J]. Journal of Beijing Institute of Technology, 2003, 23(4):440-443+460.
	张建勇, 金伟其, 周燕, 等. 基于色度学的人眼彩色探测特性分析[J]. 北京理工大学学报, 2003, 23(4):440-443+460.
[37]	LING Hefei, LU Zhengding, YANG Shuangyuan. Practical Technology of Two-dimensional DCT Color Image Digital Watermarking Based on YCbCr Color Space[J]. Small Microcomputer System, 2005, 26(3):482-484.
	凌贺飞, 卢正鼎, 杨双远. 基于YCbCr颜色空间的二维DCT彩色图像数字水印实用技术[J]. 小型微型计算机系统, 2005, 26(3):482-484.
[38]	XUE Juntao, ZONG Yunrui, YANG Zhengling. Gesture Recognition Based on Improved YCbCr Space and Multi-feature Fusion[J]. Computer Applications and Software, 2016, 33(1):151-155.
	薛俊韬, 纵蕴瑞, 杨正瓴. 基于改进的YCbCr空间及多特征融合的手势识别[J]. 计算机应用与软件, 2016, 33(1):151-155.
[39]	JIANG Li, XU Jie, TIAN Li, et al. Adaptive Skin Color Segmentation Method Based on YcbCr Space[J]. Science and Technology Communication, 2010, 26(14):235-236.
	姜丽, 许杰, 田丽, 等. 基于YcbCr空间的自适应肤色分割方法[J]. 科技传播, 2010, 26(14):235-236.
[40]	WANG Wei, DONG Jing, TAN Tieniu. Effective Image Splicing Detection Based on Image Chroma[C]//IEEE. 2009 16th IEEE International Conference on Image Processing (ICIP), November 7-10, 2009, Cairo, Egypt. New York: IEEE, 2009: 1257-1260.
[41]	ZHANG Miaolan, FU Xinwen. Research on a Texture Image Classification Method[J]. Chinese Journal of Image Graphics, 1999, 4(8):62-65.
	张妙兰, 付新文. 一种纹理图象分类方法的研究[J]. 中国图象图形学报, 1999, 4(8):62-65.
[42]	TAN Minxing, LE Q V. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks[EB/OL]. https://arxiv.org/abs/1905.11946v5 , 2020-09-11.
[43]	KARRAS T, LAINE S, AITTALA M, et al. Analyzing and Improving the Image Quality of Stylegan[C]//IEEE. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 13-19, 2020, Seattle, WA, USA. New York: IEEE, 2020: 8110-8119.
[44]	BROCK A, DONAHUE J, SIMONYAN K. Large Scale GAN Training for High Fidelity Natural Image Synthesis[EB/OL]. https://arxiv.org/abs/1809.11096v2 , 2019-02-25.

参数名称	数值
学习率	0.0003
Batch size	64
优化器	Adam
lrf	0.1
灰阶阈值	256

灰阶阈值	准确率	平均处理速度(张/秒)
100	96.73%	75
150	97.22%	67
200	97.75%	60
256	98.03%	58

算法名称	算法结构	准确率
本文所提算法	双色域纹理+EfficientNetB0	98.03%
文献[21]算法	Scharr+MobileNetV2	96.48%
文献[4]算法	SRM滤波+CNN	81.80%

算法名称	算法结构	准确率
本文所提算法	双色域特征+EfficientNetB0	90.43%
Cb、Cr改进算法	Scharr+EfficientNetB0	87.73%
文献[21]算法	Scharr+MobileNetV2	56.80%

参数名称	数值
学习率	0.0001
Batch size	64
优化器	Adam
γ	0.99
灰阶阈值	256