With the continuous improvement of network security awareness and the widespread application of encryption technology, encrypted traffic in the network is growing exponentially. Although encryption technology plays an important role in protecting user privacy and data security, it also provides a means for malicious actors to hide their attack behaviors, bringing great challenges to network security supervision and protection. With the increasing amount of encrypted traffic, traditional malicious traffic detection methods are no longer applicable. Deep learning, with its advantages in automatic feature extraction and complex data processing, has become a key technology to improve detection performance. Therefore, this paper systematically reviewed the latest achievements of deep learning in encrypted malicious traffic detection. Firstly, a general encrypted traffic detection framework was proposed according to the general steps of encrypted traffic detection. Secondly, it introduced aspects such as data collection and processing, feature extraction and selection, model methods, and evaluation metrics applied to encrypted malicious traffic detection. It also organized and analyzed existing public datasets and discusses solutions to the problem of data imbalance. Then, from the three perspectives of supervised learning, unsupervised learning, and semi-supervised learning, it compared and analyzed the advantages, disadvantages, and classification performance of different detection methods, and summarized the strengths and weaknesses of different learning methods. Finally, it discussed the open problems in the field of encrypted malicious traffic detection and looked forward to future research directions.
