Newest papers

VMSpec: A Decompiler-based Approach against Virtual Machine Obfuscation

Fabian Kilger, Leonard Auer, Manuel Andreas · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29566 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

恶意行为者经常掩饰他们的代码,以阻止调查反转工程努力。最可怕的技术之一是虚拟机(VM)掩饰:程序被翻译成一个自定义VM的完整字体代码,通常采用随机的虚拟指令集,并与相应的解释器一起分布。手动分析和解掩饰规模由于VM掩饰代码的复杂性和多样化技术,使每个VM和解释器都是独一无二的。现有的方法主要集中在简化执行痕迹和识别虚拟执行界限上。然而,相对较少的关注已经被给予了从VM掩饰代码中恢复更高级别的程序语义学。为了解决这个差距,我们提出VMSpec,一个新的基于解编程的解编程方法,完全自动化了VM处理器规格的

Machine translation (Meta M2M100); the English original follows.

Abstract

Malicious actors frequently obfuscate their code to hinder investigative reverse engineering efforts. Among the most formidable techniques is virtual machine (VM) obfuscation: programs are translated into bytecode for a custom VM, often with a randomized virtual instruction set and distributed alongside a corresponding interpreter. Manual analysis and deobfuscation scale poorly due to the complexity of VM obfuscation and the diversification techniques that render each VM and interpreter unique. Existing methods primarily focus on simplifying execution traces and identifying virtual execution boundaries. However, comparatively little attention has been given to recovering higher-level program semantics from VM-obfuscated code. To address this gap, we propose VMSpec, a novel decompiler-based approach that automates the construction of VM processor specifications, enabling existing, well-known decompilers to analyze the VM-obfuscated code. We evaluate our prototype of VMSpec on 147 programs from an established obfuscation benchmark, comprising hash functions, standard algorithms, and randomized functions, obfuscated with different configurations of the Tigress and Code Virtualizer obfuscators. To demonstrate real-world applicability, we additionally include six domain generation algorithms stemming from real-world malware. VMSpec completely strips VM-obfuscation from nearly all obfuscated samples, producing output comparable to Ghidra's decompilation of the original code. Furthermore, it is the first method capable of reconstructing the complete program behavior of VM-obfuscated programs with infinite, input-dependent paths and recursive calls.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Obfuscation, Bytecode, Reverse engineering, Programming language, Virtual machine

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange