Design and Development of an Efficient Malware Detection Using ML

Contents
• Abstract
• Existing System
• Proposed System
• Literature Survey
• Requirements
• System Design
• Results
Contents
A Novel Malware Detection Technique using Machine Learning

Abstract
Abstract
• Enormous growth and generation of data is happening in every day from various sources. The
generated data is presented in various formats i.e., in structured, unstructured, semi-structured,
Pdf’s,Doc’s,Csv’s and Raw file formats.
• All these files are not genuine/pure in all scenarios cause which is generated from identified and
unidentified sources.
• One of the major challenges that antimalware companies face today is to detect, classify and removal
of malware on the vast amounts of data.
Contd….

Abstract
• The modern malware is designed with mutation characteristics that means it can change its behavior
based on the properties of physical file. It is contraction from malicious software.
• The tremendous growth of the data is very helpful to the Malware designers to execute the malware
files such as Virus, Trojans, and Ransomware in any file.
• The formation of modern malware poses a variety of challenges to the antivirus industries.
Contd….

Abstract
• All the antivirus software’s, automatic and manual signature extraction paradigms are only identifying
the known malwares. But malware changes its properties day by day when new data is arrived.
• That is impossible to collect new data every day to train the antivirus software’s and Signatures to
detect malware and manual updation of software’s by security experts to detect malware formation on
data. Because of analyzing all the data manually to update the system is a time and cost effective
process.
Contd….
• To achieve this major defect, companies adapting machine learning technology to detect the malware.
Training the model with the data, it can able to detect the malware while it is occur, with the most
accurate performance.

Existing System
Existing System
• In existing system, the model is well trained but the maintenance of the trade-off between
complexity and performance is a key issue. Very often, then gain in performances of complex
system on validation data is negligible compared to the performances of less complex ones.

Proposed System
Proposed System
• In this proposed system we are going to design a light-weight model to accurately detect the
malware for industrial use with a high accuracy and maintaining the trade off between the
complexity of data and performance of model.

Literature Survey
A Novel Malware Detection Technique using Machine Learning Literature Survey
S.NO Title & Author Method of Developing Remarks
1 Nir Nissim et al’s.
Novel active learning methods for enhanced PC
malware detection,2014.
• Static and Dynamic Features
• An Active Learning Framework which is
developed to detect the malware by
using Support vector machines model
• SVM-Margin AL Method
showed a decrease
performance while AL methods
continued to increase to
acquire malwares.
2 Liu LIU et al’s.
Automatic malware classification and new malware
detection using machine learning,2017.
• incremental malware detection system
to classify malware families and to
detect new malware.
• shared nearest neighbour (SNN) to find
new malware on static features.
• Resulting with 86.7% when
detecting the new malware.
3 Dragos Gavrilut et al’s.
Malware Detection Using Machine learning,2011.
• Data that could be derived in train, test
and scaled up datasets to improve the
performance.
• Multilayer Perceptron algorithms are
used to detect the malware on byte
features
• Overfitting over fitting was
occurred when this algorithm
applied to large datasets.
• Resulted in an average of
89.77%

Literature Survey
S.NO Title & Author Method of Developing Remarks
4 Yujie Fan et al’s.
Malicious sequential pattern mining for automatic
malware detection,2016.
• API calls as features.
• Heuristic based detection method,
which utilizes data mining as well as
machine learning techniques.
• Aims to learn special patterns that
capture the characteristics of malware
• Unable to mine discriminative
features.
• Classification of malware can’t
be done by this model.
5 Mansour Ahmadi et al’s.
Novel Feature Extraction, Selection and Fusion for
Effective Malware Family Classification,2016.
• Static Analysis on Assembly and Byte
Features.
• Feature Selection and Feature Fusion
• Learning based system to classify the
malware using Random Forest and
XGBoost Algorithm.
• 99.7% accuracy resulted in Test Results
with 5-Fold cross validation.
• Method not yet been tested
for robustness against evasion
attacks or poisoning attacks.
• Many Features are used to
classify the malware

Malware Detection
and classification
Dynamic
Analysis
Static
Analysis
Assembly
Features
Byte
Features
API Call
Features
K Nearest
Neighbour
Naive
Bayes
Support Vector
Machines
Decision
Trees
Artificial Neural
Networks
Logistic
Regression
Random
Forest
Machine learning
Methodologies
Literature Survey Diagram

Requirements
Requirements
Hardware Requirements:
The Hardware consists of the physical components of the computer that input storage
processing control, output devices. The kind of hardware used in the project is
• Processor : Intel I3
• Hard Disk : 1TB
• RAM : 4GB

Requirements
Requirements
Software Requirements:
Software is a set of programs to do a particular task. Software is an essential requirement of
computer systems. The kind of software used in the project is
• Operating System : Windows, Linux, Mac, Ubuntu.
• IDE : Anaconda
• Script Book : Jupyter Notebook
• Coding Language : Python

System Design
• Prior to the development of signatures for anti-malware products, the two main tasks that have to be
carried out within the scope of malware analysis are malware detection and malware classification.
• While the goal of malware detection mechanisms is to catch the malware in wild, malware
classification systems assign each sample to the correct malware family.
• In this project, we are going to propose a model which uses different malware characteristics to
effectively assign malware samples to their corresponding families without doing any deobfuscation
and unpacking process.
A Novel Malware Detection Technique using Machine Learning System Design

Steps to design a Machine Learning Model
• Gathering the data
• Prepare the data
• Choosing a model
• Training
• Evaluation
• Hyper Parameter Tuning
• Prediction

System Design
Train/
Test
Split
Process
Execution
Feature
Engineering
Statistic
Analysis
Feature Selection
& Feature Fusion
Testing
Data
Training
Data
Prediction
Classification
Model
Feature Extraction
.asm .byte
Malware Classification System Architecture
Evaluation Measure
Parameter Tuning

System Design
Data Description
Compiler
Human
Understandable
Code
Assembly Level
Instruction
Assembler Byte Code
.asm Files (Assembly View)
.text:00419B02 2B CE sub ecx, esi
.text:00419B07 8D 4C 85 E0l ea ecx, [ebp+eax*4+var_20]
.text:00419B0B 8B 31 mov esi, [ecx]
.text:00419B0D 8D 3C 16 lea edi, [esi+edx]
.byte files (Hex-View)
00401000 56 8D 44 24 08 50 8B F1 E8 1C 1B 00 00 C7 06 08
00401010 BB 42 00 8B C6 5E C2 04 00 CC
00401020 C7 01 08 BB 42 00 E9 26 1C 00 00 CC
00401030 56 8B F1 C7 06 08 BB 42 00 E8 13 1C 00 00 F6
• Spilt the dataset randomly into three parts Train, Cross Validation and Test with 64%, 16%, 20% of data respectively

Malware Classes
• Ramnit
• Lollipop
• Kelihos_ver3
• Vundo
• Simda
• Tracur
• Kelihos_ver1
• Obfuscator.ACY
• Gatak

System Design
Feature Engineering(Feature Selection & Feature Fusion)
• N-gram
• Metadata
.asm file features
.byte file features
• Symbol
• Operation Code
• Register
• Section

System Design
Classification Model
• K-Nearest Neighbour
• Logistic Regression
• Random Forest
• XGBoost
Evaluation Measures
• Confusion Matrix
• Multi Class Log-Loss Function

Conclusion
System Design
• In this we are going to identifying 9 different types of malwares on huge amount of data (0.5TB) that
is provided by Microsoft.
• Malware detection and classification techniques are two different tasks, that are performed by anti-
malware companies.
• Firstly, an executable needs to be analysed to detect if it exhibits any malicious content: Then, in the
case a malware is detected, it is assigned to the most appropriate malware family through a
classification mechanism.

Results
Malware Probability Count

Results
Malware Train and Test Data Values Count

Results
Malware Cross Validation Data Values Count

Results
KNN Malware Class Count on Byte Malware Data

Results
Logistic Regression Malware Class Count on Byte Malware Data

Results
Random Forest Malware Class Count on Byte Malware Data

Results
Xgboost Malware Class Count on Byte Malware Data

Results
Accuracy Values on Byte Malware Data
Classifier Test Log-loss Misclassification Rate Accuracy
K-NN 0.24% 4.50% 95.50
Logistic
Regression
0.528% 12.32% 77.68
Random Forest 0.085% 2.02% 97.98
XGboost 0.078% 1.24% 98.76

Results

Results
K-NN Malware Class Count on ASM Malware Data

Results
Logistic Regression Malware Class Count on ASM Malware Data

Results
Random Forest Malware Class Count on ASM Malware Data

Results
Xgboost Malware Class Count on ASM Malware Data

Results
Accuracy Values on ASM Malware Data
Classifier Test Log-loss Misclassification Rate Accuracy
K-NN 0.089% 2.02% 97.98
Logistic Regression 0.415% 9.61% 90.39
Random Forest 0.057% 1.15% 98.85
XGboost 0.048% 0.87% 99.13

Results
Accuracy Values on ASM Malware Data

Thank You.

Design and Development of an Efficient Malware Detection Using ML

More Related Content

What's hot (20)

Similar to Design and Development of an Efficient Malware Detection Using ML (20)

Recently uploaded (20)

Design and Development of an Efficient Malware Detection Using ML