Abstract:

Existing Android malicious sample analysis methods need to obtain the sample program to be checked in advance. When the object to be checked is an android smart terminal instead of a sample program, it is impossible to determine which process in the smart terminal is the malicious process to be checked, which affects the effective application of the sample analysis method. Existing detection methods for malicious encrypted traffic can achieve high recognition accuracy, but it is impossible to determine the mapping relationship between malicious traffic and malicious processes in the android terminal, i.e. Which process generates malicious encrypted traffic cannot be determined, and further the specific location information of malicious processes cannot be locked. In order to solve the above problems, this paper proposes an android malicious process identification method based on anomalous encrypted traffic annotation. By monitoring the network communication data generated by android terminals, DNS characteristics, TLS handshake negotiation characteristics and flow statistical characteristics are extracted, and binary classifier based on random forest algorithm is adopted to identify malicious encrypted communication flow. Then, by extracting the characteristics of the flow 5-tuple, a one-to-one mapping is established between the malicious encrypted communication stream and the android terminal process to determine the specific location of the malicious process in the terminal. The experimental results show that the detection accuracy of the proposed method for unknown malicious encrypted traffic in complex network environment is 97.46%, and malicious processes in android terminals can be located according to the detected malicious encrypted data flow.

Key words: TLS protocol, encrypted traffic annotation, 5-tuple, random forest, malicious process identification