A Code Obfuscation System against Symbolic Execution Attacks

doi:10.3969/j.issn.1671-1122.2021.07.003

Abstract

Abstract:

Symbolic execution technology has made great progress in the areas of assisting the discovery of software vulnerabilities and software de-obfuscation. As one of the most effective means to protect software security, the existing code obfuscation system can be hardly resilient against symbolic execution attacks. To solve the problem mentioned above, starting from the function extension of the obfuscation system and the weakness exploitation of the symbolic execution tool, this paper enriches the synonymous instruction replacement set and the identical truth predicate set, and designs opaque predicates to obfuscate the control flow and reinforce the branching conditions of the software, and proposes a code obfuscation system that is compatible with multiple programming languages, scalable, and can resist symbolic execution attacks. The experimental results show that this obfuscation system can induce symbolic execution tools to enter unreachable paths, or protect some right paths from being explored by symbolic execution tools without changing the original functions of the software, so that it can be resilient against the attacks of symbolic execution tools. Compared with traditional obfuscation systems, this system has obvious advantages in combating symbolic execution attacks.

Key words: code obfuscation, symbolic execution, opaque predicate, OLLVM

CLC Number:

TP309

WEN Weiping, FANG Ying, YE He, CHEN Xiarun. A Code Obfuscation System against Symbolic Execution Attacks[J]. Netinfo Security, 2021, 21(7): 17-26.

Figures/Tables 12

References 24

[1]	COLLBERG C, THOMBORSON C, LOW D. Breaking Abstractions and Unstructuring Data Structures[C]// IEEE. The 1998 International Conference on Computer Languages, May 14-16, 1998, Chicago, IL, USA. Los Alamitos: IEEE Comp Soc, 1998: 28-38.
[2]	BANESCU S, COLLBERG C, GANESH V, et al. Code Obfuscation Against Symbolic Execution Attacks[C]// ACSA. 32nd Annual Computer Security Applications Conference, December 5-9, 2016, Los Angeles, CA, USA. New York: Association for Computing Machinery, 2016: 189-200.
[3]	BOYER R S, ELSPAS B, LEVITT K N. SELECT—A Formal System for Testing and Debugging Programs by Symbolic Execution[J]. ACM SigPlan Notices, 1975, 10(6):234-245. doi: 10.1145/390016.808445 URL
[4]	CLARKE L A. A Program Testing System[C]// ACM. 1976 ACM Annual Conference, October 20-22, 1976, Houston, TX, USA. New York: Association for Computing Machinery, 1976: 488-491.
[5]	KING J C. Symbolic Execution and Program Testing[J]. Communications of the ACM, 1976, 19(7):385-394. doi: 10.1145/360248.360252 URL
[6]	SALWAN J, BARDIN S, POTET M L. Symbolic Deobfuscation: From Virtualized Code Back to the Original[C]// SIDAR. 15th International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment, DIMVA 2018, June 28-29, 2018, Saclay, France. New York: Springer Verlag, 2018: 372-392.
[7]	YADEGARI B, DEBRAY S. Symbolic Execution of Obfuscated Code[C]// ACM SIGSAC. 22nd ACM SIGSAC Conference on Computer and Communications Security, CCS 2015, October 12-16, 2015, Denver, CO, USA. New York: Association for Computing Machinery, 2015: 732-744.
[8]	YADEGARI B, JOHANNESMEYER B, WHITELY B, et al. A Generic Approach to Automatic Deobfuscation of Executable Code[C]// IEEE Computer Society's Technical Committee (TC). 36th IEEE Symposium on Security and Privacy, May 18-20, 2015, San Jose, CA, USA. Piscataway: Institute of Electrical and Electronics Engineers Inc., 2015: 674-691.
[9]	BRUMLEY D, HARTWIG C, LIANG Zhenkai, et al. Automatically Identifying Trigger-based Behavior in Malware[M]. New York: Springer, 2008.
[10]	MING Jiang, XU Dongpeng, WANG Li, et al. Loop: Logic-oriented Opaque Predicate Detection in Obfuscated Binary Code[C]// ACM SIGSAC. 22nd ACM SIGSAC Conference on Computer and Communications Security, CCS 2015, October 12-16, 2015, Denver, CO, USA. New York: Association for Computing Machinery, 2015: 757-768.
[11]	TOFIGHI S R, CHRISTOFI M, ELBAZ-VINCENT P, et al. Dose: Deobfuscation Based on Semantic Equivalence[C]// ACM. 8th Software Security, Protection, and Reverse Engineering Workshop, SSPREW 2018, December 3-4, 2018, San Juan, PR, USA. New York: Association for Computing Machinery, 2018: 1-12.
[12]	EYROLLES N, GOUBIN L, VIDEAU M. Defeating Mba-based Obfuscation[C]// ACM SIGSAC. 2nd International Workshop on Software PROtection, SPRO 2016,October 28, 2016, Vienna, Austria. New York: Association for Computing Machinery, 2016-27-37.
[13]	GABRIEL F. Deobfuscation: Recovering An OLLVM-protected Program[EB/OL]. https://blog.quarkslab.com/deobfuscation-recovering-an-ollvm-protected-program.html, 2014-12-04.
[14]	SCHRITTWIESER S, KATZENBEISSER S, KINDER J, et al. Protecting Software through Obfuscation: Can It Keep Pace with Progress in Code Analysis?[J]. ACM Computing Surveys (CSUR), 2016, 49(1):1-37.
[15]	R1KK3R. Obfuscator-LLVM[EB/OL]. https://github.com/obfuscator-LLVM/obfuscator, 2017-06-29.
[16]	WJQ. Armariris[EB/OL]. https://github.com/GoSSIP-SJTU/Armariris, 2019-06-03.
[17]	NAVILLE. HikariObfuscator[EB/OL]. https://github.com/HikariObfuscator/Hikari, 2020-01-30.
[18]	ZHOU Yongxin, MAIN Alec, GU Yuanxiang, et al. Information Hiding in Software with Mixed Boolean-arithmetic Transforms[EB/OL]. https://link.springer.com/chapter/10.1007/978-3-540-77535-5_5, 2007-08-27.
[19]	XU Hui, ZHOU Yangfan, KANG Yu, et al. Manufacturing Resilient Bi-opaque Predicates against Symbolic Execution[C]// IEEE. 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2018, June 25-28, 2018, Luxembourg City, Luxembourg. Piscataway: Institute of Electrical and Electronics Engineers Inc., 2018: 666-677.
[20]	SHARIF M I, LANZI A, GIFFIN J T, et al. Impeding Malware Analysis Using Conditional Code Obfuscation[C]// NDSS. 15th Annual Network and Distributed System Security Symposium, February 10-13, 2008, San Diego, CA, USA. Reston: Internet Society, 2008: 321-333.
[21]	WANG Zhi, MING Jiang, JIA Chunfu, et al. Linear Obfuscation to Combat Symbolic Execution[C]// ESORICS. 16th European Symposium on Research in Computer Security, September 12-14, 2011, Leuven, Belgium. Berlin: Springer Verlag, 2011: 210-226.
[22]	STEPHENS J, YADEGARI B, COLLBERG C, et al. Probabilistic Obfuscation through Covert Channels[C]// IEEE. 3rd IEEE European Symposium on Security and Privacy, EURO S and P 2018, April 24-26, 2018, London, UK. Piscataway: Institute of Electrical and Electronics Engineers Inc., 2018: 243-257.
[23]	JIEWEIWEI. MD5[EB/OL]. https://github.com/JieweiWei/md5, 2014-10-24.
[24]	RSWINDELL. SHA1[EB/OL]. https://github.com/clibs/sha1, 2021-03-24.

原指令	替换后
a = b c	a = (b d) (c-d)
	a = (b-r r) (c r-r)
	a = (bh ch) (bl cl)
	a = b c zero
a = b-c	a = b d; a = a-c; a = a-d
	a = b-d; a = a-c; a = a d
	a = (bh-ch) (bl-cl)
	a = (b r-r)- (c-r r)
	a = b-c-zero
a = b * c	a = (b-r r)*c
a = b * c	a = (b-r r)* (c-r r)
a = b * c	a = (b zero)* (c-zero)
a = b << c	a = (b r-r) << c
a = b << c	a = (bh << c)% 1 << 16 ( bl << c)% 1 << 16
a = b >> c	a = ( b r-r) << c
a = b & c	a = (b r-r)& (c r-r)
	a = (b zero)& (c-zero)
	a = bl & cl bh & ch
a = b \| c	a = (b r-r) \| (c r-r)
	a = (b zero) \| (c-zero)
	a = bl \| cl bh \| ch
a = b ^ c	a = (b r-r) ^ (c r-r)
	a = (b zero) ^ (c-zero)
	a = bl ^ cl bh ^ ch

类型	版本
操作系统	Ubuntu 20.04.1 LTS
LLVM	LLVM version 10.0.0
KLEE	tags/v2.1
angr	tags/v9.0.6488
Python	Python 3.7.0
CMake	CMake version 3.16.3
IDA pro	version 7.0.170914

测试项目	测试结果
coreutils-8.32源代码随机4个文件	正常执行,测试通过
gzip1.6源代码	正常执行,测试通过
diffutils3.7源代码	正常执行,测试通过
glibc2.33源代码随机4个文件	正常执行,测试通过
MD5开源实现代码^[23]	正常执行,测试通过
SHA1开源实现代码^[24]	正常执行,测试通过

混淆模块	KLEE	angr
未混淆	T	T
OLLVM	T	T
Armariris	T	T
Hikari	T	T
分支复杂化模块	T	T
指令复杂化模块	T	T
文件截断模块	FN	T
Hash函数模块	FP	FP
DLP函数模块	FP	FP
费马小定理模块	FP	FP