Measuring Malware Detection Capability for Security Decision Making

Organizations face an urgent need to bolster their cybersecurity defenses against the rising threat of ransomware. Implementing advanced antivirus and antimalware tools is crucial for proactive identification and mitigation of malicious software. However, adversaries constantly refine malware to evade detection increasing the complexity of the threat. Hence, developing an effective strategy is nontrivial. To address this challenge, this study conducts various analyses on scan results of publicly shared malware samples. Utilizing metadata from 635K samples sourced from MalwareBazaar and scan results from VirusTotal, we assign family labels using AVClass. Additionally, we examine
a 90-day longitudinal dataset alongside the main dataset. Our findings demonstrate that while over 60% of scanner engines detect 67% of samples, certain malware families consistently exhibit lower detection rates. Detection capability improves over time, particularly within the initial 30 days, but remains
inadequate for specific families. Furthermore, we observe that some scanner engines demonstrate nearly flawless detection capability across all malware families, while the majority struggle with efficiently detecting certain types. Moreover, we performed Monte Carlo simulations and revealed that employing multiple scanner engines substantially enhances detection capability, with 3 to 7 scanners being optimal. Finally, simulation analysis in a case study highlights the
significant impact of hard-to-detect malware on risk and performance, underscoring the importance of effective malware strategies.