Abstract: Cloud computing is developing fast and widely used, as an important support for cloud computing, virtualization has improved the efficiency of resource utilization and management capability for a platform. As an open source software for virtualization, the unique design and excellent performance make Xen adopted by many could service providers, which are also troubled by the security problems of Xen hypervisor. The privilege interfaces provided by Xen can be utilized by malicious code of virtual machine, which can be used by intruders to attack Xen or virtual machines running above. To solve the problem of hypercalls of Xen to be abused by malicious code inside guest kernel, a method to analyze the execution path of guest kernel is provided, which is used to trace the execution path of guest kernel that has launched this hypercall, compared with the training set constructed at the beginning, preventing hypercalls being misused by malicious code of guest kernel becomes possible. By tracking stack information of guest kernel, the execution path of virtual machine is reconstructed and built up with the help of instruction analysis and symbol table of guest kernel, unexpected execution paths of hypervalls are avoided with this method. We experimented our idea on Xen platform, a new virtual machine was created to get its training set during its running time. Then when this heprcall happens, the corresponding execution path is constructed dynamically, compared with the training set, unforeseen invoking to hypervalls is avoided.

Key words: Xen, kernel stack, function call graph, instruction analysis