Abstract:

Anomaly detection of program behavior is an important part of network anomaly detection. In traditional HMM, the probability of transition and the probability of the observed value is only related to the previous state, which leads to high false alarm rate and low detection rate. In this paper an improved 2HMM detection method is proposed, which is based on local regularity of the system calls. And in order to reduce the training time, the model uses a more simple parameter estimation algorithm. Finally, through the experiment, compared with the traditional HMM and traditional 2HMM,the superiority of the model is proved.

Key words: program behavior, anomaly detection, HMM