Are our clone detectors good enough? An empirical study of code effects by obfuscation

Abstract Clone detection has received much attention in many fields such as malicious code detection, vulnerability hunting, and copyright infringement detection. However, cyber criminals may obfuscate to impede violation To date, few studies have investigated the robustness of clone detectors, especially in-fashion deep learning-based ones, against obfuscation. Meanwhile, most these only measure difference between one snippet its obfuscation version. reality, attackers modify original before obfuscating it. Then what we should evaluate is obfuscated from cloned code, not code. For this, conduct a comprehensive study evaluating 3 popular deep-learning based detectors 6 commonly used traditional ones. Regarding data, collect 6512 pairs five types dataset BigCloneBench program each pair via 64 strategies state-of-art commercial obfuscators. We also 1424 non-clone false positives. In sum, benchmark 524,148 (either or not) are generated, which passed for evaluation. automate evaluation, develop uniform evaluation framework, integrating The results bring us interesting findings on how affects performance detectors. addition, manual reviews uncover root cause phenomenon give suggestions users different perspectives.