Machine Learning-Based Malware Detection

In this chapter, we begin to get serious about applying data science to cybersecurity. We will begin by learning how to perform static and dynamic analysis on samples. Building on this knowledge, we will learn how to featurize samples in order to construct a dataset with informative features. The highlight of the chapter is learning how to build a static malware detector using the featurization skills we have learned. Finally, you will learn how to tackle important machine learning challenges that occur in the domain of cybersecurity, such as class imbalance and false positive rate (FPR) constraints.

The chapter covers the following recipes:

• Malware static analysis
• Malware dynamic analysis
• Using machine learning to detect the file type
• Measuring the similarity between two strings
• Measuring the similarity between two files
• Extracting N-grams
• Selecting the best N-grams
• Building a static malware detector
• Tackling class imbalance
• Handling type I and type II errors