SlideShare a Scribd company logo
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 06 Issue: 03 | Mar 2019 www.irjet.net p-ISSN: 2395-0072
© 2019, IRJET | Impact Factor value: 7.211 | ISO 9001:2008 Certified Journal | Page 3869
Supervised Learning Classification Algorithms Comparison
Aditya Singh Rathore
B.Tech, J.K. Lakshmipat University
-------------------------------------------------------------***---------------------------------------------------------
Abstract: Under supervised machine learning, classification tasks are one of the most important tasks as a part of data
analysis. It gives a lot of actionable insights to data scientists after using different machine learning algorithms. For study
purpose the Titanic dataset has been chosen. Through this paper, an effort has been made to evaluate different classification
models’ performance on the dataset using the scikit-learn library. The classifiers chosen are Logistic Regression, K-Nearest
Neighbor, Decision Tree, Random Forest, Support Vector Machine and Naïve Bayes classifier. In the end evaluation
parameters like confusion matrix, precision, recall, f1-score and accuracy have been discussed.
Key Words: Numpy, Pandas, classification, knn, logistic regression, decision tree, random forest, Naïve Bayes, SVM,
sklearn.
1. INTRODUCTION:
Machine learning has been getting a lot of attention for the past few years because it holds the key to solve the problems of
the future. In the 21st Century, we are living in a world where data is everywhere and as we go by of our day we are
generating data, be it text messaging or simply walking down the street and different antennas picking GPS signals. The
majority of practical machine learning uses supervised learning. In supervised learning, the machine learning models are
first trained over dataset with associated correct labels. The objective of training the models first is to create a very good
mapping between the independent and dependent variables, so that when that model is run on a new dataset, it can
predict the output variable. Classification is a type of problem under supervised learning. A classification problem is when
the output is a category and when given one or more inputs a classification model will try to predict the value based on
what relation it developed between the input and output. Below are the classifiers chosen for evaluation:
1.1 Logistic Regression:
This type of technique proves to be more appropriate when the dependent variable/output is in the form of binary output,
i.e. 1 or 0. This technique is mainly used when one wants to see the relationship between the dependent variable and one
or more independent variable provided that the dataset is linear. The output of this technique gives out a binary output
which is a better type of output than absolute values as it elucidates the output. In order to limit our output between 0 and
1 we will be using Sigmoid Function.
Below is the mathematical representation of Sigmoid Function:
f(x)= 1/(1+e^-x)
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 06 Issue: 03 | Mar 2019 www.irjet.net p-ISSN: 2395-0072
© 2019, IRJET | Impact Factor value: 7.211 | ISO 9001:2008 Certified Journal | Page 3870
1.2 K- Nearest Neighbor:
This type of technique is one of the simplest classification algorithm. It first stores the entire data in its training phase as
an array of data points. . Whenever a prediction is required for a new data it searches through the entire training dataset
for a number of most similar instances and the data with most similar instance is returned as prediction. This number is
defined by an integer ‘k’. The k denotes the number of neighbours which are the determining class of the test data. For
example, if k=3, then the labels of the 3 closest instances are checked and the most common label is assigned to the test
data. The distance measured from test data to the closest instance is the least distance measure. This can be achieved using
various measuring tools such as Euclidean Distance. The Euclidean distance between point p and q is calculated as
below:
1.3 Decision Tree:
A decision tree is a graphical representation of all possible solutions to a decision based on certain conditions. It consists of
a Root/Parent node which is the base node or the first node of a tree. Then comes the splitting which is basically dividing
the Root/Child nodes into different parts on a certain condition. These parts are called the Branch/ Sub Tree. To remove
unwanted branches from the tree we make use of Pruning. The last node is called the leaf node when the data cannot be
segregated to further levels. We will use CART (Classification and Regression Tree) algorithm to our dataset. Below are
some terminologies used by CART algorithm in order to select the root node :
a) Gini Index: It is the measure of impurity used to build a decision tree
b) Information Gain: It is the decrease in entropy after a dataset is split on the basis of a criteria.
c) Reduction in Variance: The split with lower variance is selected as a criteria.
d) Chi Square: It is the measure of statistical significance between the differences between sub-nodes and parent
node.
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 06 Issue: 03 | Mar 2019 www.irjet.net p-ISSN: 2395-0072
© 2019, IRJET | Impact Factor value: 7.211 | ISO 9001:2008 Certified Journal | Page 3871
1.4 Random Forest:
This classifier is one of the most used algorithms today because it can be used for classification as well as regression. The
main problem with decision trees is that it can over fit the data. One solution to prevent this is to create multiple decision
trees on random subsamples of the data. Now when a new data point is to be predicted then that will be computed by all
decision trees and a majority vote will be calculated. This vote will determine the class of the data. This in turns gives
higher accuracy when compared to decision tree alone. More decision trees directly correlates with more accuracy.
1.5 Support Vector Machine (SVM):
This classifier uses a hyperplane/decision boundary that separates the classes. This hyperplane is positioned in
such a way that it maximises the margin i.e. the space between the decision boundary and the points in each of
the classes that are closest to the decision boundary. These points are called Support Vectors. A hyperplane is a
decision surface that splits the space into 2 parts i.e. it is a binary classifier. This hyperplane in R^n space is an
n-1 dimensional subspace.
SVMs can easily be used for linear classification. However, in order to use SVM for non-linear classification once can use a
kernel trick with which the data is transformed in another dimension to determine the appropriate position of hyperplane.
1.6 Naïve Bayes:
This classifier is used primarily for text classification which generally involves training on high dimensional datasets. This
classifier is relatively faster than other algorithms in making predictions. It is called “naïve” because it makes an
assumption that a feature in the dataset is independent of the occurrence of other features i.e. it is conditionally
independent. This classifier is based on the Bayes Theorem which states that the probability of an event Y given X is equal
to the probability of the event X given Y multiplied by the probability of X upon probability of Y.
P(X|Y) =(P(Y|X) ⋅ P(X))/P(Y)
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 06 Issue: 03 | Mar 2019 www.irjet.net p-ISSN: 2395-0072
© 2019, IRJET | Impact Factor value: 7.211 | ISO 9001:2008 Certified Journal | Page 3872
2. METHODOLOGY:
2.1 Dataset:
The data has been split into two groups:
a) training set (train.csv)
b) test set (test.csv)
The training set is used to build and train the machine learning models. The test set is used to see how well the model
performed on unseen data. We will use the trained models to predict whether or not the passengers in the test set
survived the sinking of the Titanic.
Below is the original structure of the dataset:
Data Dictionary:
 Survived: 0 = No, 1 = Yes
 pclass: Ticket class 1 = 1st, 2 = 2nd, 3 = 3rd
 sibsp: # of siblings / spouses aboard the Titanic
 parch: # of parents / children aboard the Titanic
 ticket: Ticket number
 cabin: Cabin number
 embarked: Port of Embarkation C = Cherbourg, Q = Queenstown, S = Southampton
After pre-processing of the data and feature engineering, below is the dataset we would be finally be working with:
2.2 Evaluation Parameters:
2.2.1 Confusion Matrix:
On a classification problem a confusion matrix gives out a summary of predicted results. The basic idea behind it is to
count the number of times the instances of class A are classified as class B.
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 06 Issue: 03 | Mar 2019 www.irjet.net p-ISSN: 2395-0072
© 2019, IRJET | Impact Factor value: 7.211 | ISO 9001:2008 Certified Journal | Page 3873
Below is the sample confusion matrix:
Pridicted Yes Pridicted No
Actual Yes True Positive False Negative
Actual No False Positive True Negative
2.2.2 Precision:
It is the ratio of total number of true positives divided by total sum of true positives and false positives. In summary it
describes how many of the returned hits were true positive i.e. how many of the found were correct hits.
Precision = True Positives / (True Positives + False Positives)
2.2.3 Recall:
It is the ratio of the total number of true positives divided by total sum of the true positives and the false negatives. It is
also called sensitivity. In summary it describes how many of the true positives were recalled (found), i.e. how many of the
correct hits were also found.
Recall = True Positives / (True Positives + False Negatives)
2.2.4 F1- Score:
It is simply the weighted average of precision and recall wherein the best value for a model is 1 and the worst value is 0.
Below is the formula to calculate the same:
F1-score = 2 * (precision * recall) / (precision + recall)
2.3 Experimental Results:
Below are the experimental results of different Classifiers:
Classifier Class Classification Report Confusion
Matrix
Precision Recall f1-score Accuracy
Logistic
Regression
0 0.81 0.86 0.84 78.90% [[120 19]
[ 28 56]]1 0.75 0.67 0.7
KNN 0 0.73 0.83 0.77 69.90% [[115 24]
[ 43 41]]1 0.63 0.49 0.55
Decision Tree 0 0.84 0.78 0.81 77.17% [[108 31]
[ 20 64]]1 0.67 0.76 0.72
Random Forest 0 0.87 0.88 0.88 84.30% [[123 16]
[ 19 65]]1 0.8 0.77 0.79
SVM 0 0.73 0.76 0.74 67.20% [[106 33]
[ 40 44]]1 0.57 0.52 0.55
Naïve Bayes 0 0.83 0.83 0.83 78.47% [[115 24]
[ 24 60]]1 0.71 0.71 0.71
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 06 Issue: 03 | Mar 2019 www.irjet.net p-ISSN: 2395-0072
© 2019, IRJET | Impact Factor value: 7.211 | ISO 9001:2008 Certified Journal | Page 3874
3. CONCLUSION:
In this paper we experimented with 6 types of classifiers on the Titanic dataset. In order to evaluate the above the train
dataset was split into train & test dataset, since there is no true output for test dataset. From the above the random forest
seems to be most effective with highest precision and recall values and has the least number of false positive and false
negatives.
4. REFERENCES:
[1].P-N. Tan, M. Steinbach, V. Kumar, “Introduction to Data Mining,” Addison-Wesley Publishing, 2006.
[2]. Simm, J., Magrans de Abril, I., & Sugiyama, M., Tree-based ensemble multi-task learning method for classification and
regression, 2014.
[3] Takeuchi, I. & Sugiyama, M, Target neighbor consistent feature weighting for nearest neighbor classification.
[4] M. Kantardzic, “Data Mining: Concepts, Models, Methods, and Algorithms,” John Wiley & Sons Publishing, 2003.

More Related Content

PPTX
2.mathematics for machine learning
PDF
IRJET- Performance Evaluation of Various Classification Algorithms
PDF
An Algorithm Analysis on Data Mining-396
PDF
IRJET - Rainfall Forecasting using Weka Data Mining Tool
PDF
Research scholars evaluation based on guides view using id3
PDF
Principal Component Analysis and Clustering
PDF
Principal Component Analysis
PPTX
2.mathematics for machine learning
IRJET- Performance Evaluation of Various Classification Algorithms
An Algorithm Analysis on Data Mining-396
IRJET - Rainfall Forecasting using Weka Data Mining Tool
Research scholars evaluation based on guides view using id3
Principal Component Analysis and Clustering
Principal Component Analysis

What's hot (14)

PDF
THE IMPLICATION OF STATISTICAL ANALYSIS AND FEATURE ENGINEERING FOR MODEL BUI...
PDF
Pca analysis
PDF
A tour of the top 10 algorithms for machine learning newbies
PDF
Slides distancecovariance
PDF
PCA (Principal component analysis)
PPT
CART Classification and Regression Trees Experienced User Guide
PDF
Principal component analysis and lda
PPTX
Principal Component Analysis (PCA) and LDA PPT Slides
PDF
Statistical Data Analysis on a Data Set (Diabetes 130-US hospitals for years ...
PDF
Classification Based Machine Learning Algorithms
PPTX
AI: Belief Networks
PDF
84cc04ff77007e457df6aa2b814d2346bf1b
PDF
KNN and ARL Based Imputation to Estimate Missing Values
PDF
IRJET- Evaluation of Classification Algorithms with Solutions to Class Imbala...
THE IMPLICATION OF STATISTICAL ANALYSIS AND FEATURE ENGINEERING FOR MODEL BUI...
Pca analysis
A tour of the top 10 algorithms for machine learning newbies
Slides distancecovariance
PCA (Principal component analysis)
CART Classification and Regression Trees Experienced User Guide
Principal component analysis and lda
Principal Component Analysis (PCA) and LDA PPT Slides
Statistical Data Analysis on a Data Set (Diabetes 130-US hospitals for years ...
Classification Based Machine Learning Algorithms
AI: Belief Networks
84cc04ff77007e457df6aa2b814d2346bf1b
KNN and ARL Based Imputation to Estimate Missing Values
IRJET- Evaluation of Classification Algorithms with Solutions to Class Imbala...
Ad

Similar to IRJET- Supervised Learning Classification Algorithms Comparison (20)

PDF
IRJET- Performance Evaluation of Various Classification Algorithms
PDF
IRJET- Comparison of Classification Algorithms using Machine Learning
PDF
Machine Learning: An introduction โดย รศ.ดร.สุรพงค์ เอื้อวัฒนามงคล
PDF
A detailed analysis of the supervised machine Learning Algorithms
PPTX
UNIT 3: Data Warehousing and Data Mining
PPTX
Unit 4 Classification of data and more info on it
PDF
A Survey Ondecision Tree Learning Algorithms for Knowledge Discovery
PPTX
Reuqired ppt for machine learning algirthms and part
PDF
IRJET- Unabridged Review of Supervised Machine Learning Regression and Classi...
PPTX
CST413 KTU S7 CSE Machine Learning Supervised Learning Classification Algorit...
PPT
Data mining techniques unit iv
PDF
Machine Learning Algorithms Introduction.pdf
PPTX
Deep learning from mashine learning AI..
PPTX
BAS 250 Lecture 8
PPTX
Machine learning Chapter three (16).pptx
PDF
introducatio to ml introducatio to ml introducatio to ml
PDF
machine_learning.pptx
PDF
20MEMECH Part 3- Classification.pdf
PDF
IRJET- Study and Evaluation of Classification Algorithms in Data Mining
PDF
Shriram Nandakumar & Deepa Naik
IRJET- Performance Evaluation of Various Classification Algorithms
IRJET- Comparison of Classification Algorithms using Machine Learning
Machine Learning: An introduction โดย รศ.ดร.สุรพงค์ เอื้อวัฒนามงคล
A detailed analysis of the supervised machine Learning Algorithms
UNIT 3: Data Warehousing and Data Mining
Unit 4 Classification of data and more info on it
A Survey Ondecision Tree Learning Algorithms for Knowledge Discovery
Reuqired ppt for machine learning algirthms and part
IRJET- Unabridged Review of Supervised Machine Learning Regression and Classi...
CST413 KTU S7 CSE Machine Learning Supervised Learning Classification Algorit...
Data mining techniques unit iv
Machine Learning Algorithms Introduction.pdf
Deep learning from mashine learning AI..
BAS 250 Lecture 8
Machine learning Chapter three (16).pptx
introducatio to ml introducatio to ml introducatio to ml
machine_learning.pptx
20MEMECH Part 3- Classification.pdf
IRJET- Study and Evaluation of Classification Algorithms in Data Mining
Shriram Nandakumar & Deepa Naik
Ad

More from IRJET Journal (20)

PDF
Enhanced heart disease prediction using SKNDGR ensemble Machine Learning Model
PDF
Utilizing Biomedical Waste for Sustainable Brick Manufacturing: A Novel Appro...
PDF
Kiona – A Smart Society Automation Project
PDF
DESIGN AND DEVELOPMENT OF BATTERY THERMAL MANAGEMENT SYSTEM USING PHASE CHANG...
PDF
Invest in Innovation: Empowering Ideas through Blockchain Based Crowdfunding
PDF
SPACE WATCH YOUR REAL-TIME SPACE INFORMATION HUB
PDF
A Review on Influence of Fluid Viscous Damper on The Behaviour of Multi-store...
PDF
Wireless Arduino Control via Mobile: Eliminating the Need for a Dedicated Wir...
PDF
Explainable AI(XAI) using LIME and Disease Detection in Mango Leaf by Transfe...
PDF
BRAIN TUMOUR DETECTION AND CLASSIFICATION
PDF
The Project Manager as an ambassador of the contract. The case of NEC4 ECC co...
PDF
"Enhanced Heat Transfer Performance in Shell and Tube Heat Exchangers: A CFD ...
PDF
Advancements in CFD Analysis of Shell and Tube Heat Exchangers with Nanofluid...
PDF
Breast Cancer Detection using Computer Vision
PDF
Auto-Charging E-Vehicle with its battery Management.
PDF
Analysis of high energy charge particle in the Heliosphere
PDF
A Novel System for Recommending Agricultural Crops Using Machine Learning App...
PDF
Auto-Charging E-Vehicle with its battery Management.
PDF
Analysis of high energy charge particle in the Heliosphere
PDF
Wireless Arduino Control via Mobile: Eliminating the Need for a Dedicated Wir...
Enhanced heart disease prediction using SKNDGR ensemble Machine Learning Model
Utilizing Biomedical Waste for Sustainable Brick Manufacturing: A Novel Appro...
Kiona – A Smart Society Automation Project
DESIGN AND DEVELOPMENT OF BATTERY THERMAL MANAGEMENT SYSTEM USING PHASE CHANG...
Invest in Innovation: Empowering Ideas through Blockchain Based Crowdfunding
SPACE WATCH YOUR REAL-TIME SPACE INFORMATION HUB
A Review on Influence of Fluid Viscous Damper on The Behaviour of Multi-store...
Wireless Arduino Control via Mobile: Eliminating the Need for a Dedicated Wir...
Explainable AI(XAI) using LIME and Disease Detection in Mango Leaf by Transfe...
BRAIN TUMOUR DETECTION AND CLASSIFICATION
The Project Manager as an ambassador of the contract. The case of NEC4 ECC co...
"Enhanced Heat Transfer Performance in Shell and Tube Heat Exchangers: A CFD ...
Advancements in CFD Analysis of Shell and Tube Heat Exchangers with Nanofluid...
Breast Cancer Detection using Computer Vision
Auto-Charging E-Vehicle with its battery Management.
Analysis of high energy charge particle in the Heliosphere
A Novel System for Recommending Agricultural Crops Using Machine Learning App...
Auto-Charging E-Vehicle with its battery Management.
Analysis of high energy charge particle in the Heliosphere
Wireless Arduino Control via Mobile: Eliminating the Need for a Dedicated Wir...

Recently uploaded (20)

PDF
Enhancing Cyber Defense Against Zero-Day Attacks using Ensemble Neural Networks
PPT
Project quality management in manufacturing
PPTX
OOP with Java - Java Introduction (Basics)
PPTX
KTU 2019 -S7-MCN 401 MODULE 2-VINAY.pptx
PPTX
web development for engineering and engineering
PPTX
UNIT-1 - COAL BASED THERMAL POWER PLANTS
PPTX
additive manufacturing of ss316l using mig welding
PDF
BMEC211 - INTRODUCTION TO MECHATRONICS-1.pdf
PPTX
MET 305 2019 SCHEME MODULE 2 COMPLETE.pptx
PDF
Well-logging-methods_new................
PPTX
CH1 Production IntroductoryConcepts.pptx
PPTX
Foundation to blockchain - A guide to Blockchain Tech
PDF
Digital Logic Computer Design lecture notes
PPTX
UNIT 4 Total Quality Management .pptx
PDF
PRIZ Academy - 9 Windows Thinking Where to Invest Today to Win Tomorrow.pdf
PDF
Model Code of Practice - Construction Work - 21102022 .pdf
PDF
R24 SURVEYING LAB MANUAL for civil enggi
PPTX
Sustainable Sites - Green Building Construction
PPTX
Engineering Ethics, Safety and Environment [Autosaved] (1).pptx
PDF
Automation-in-Manufacturing-Chapter-Introduction.pdf
Enhancing Cyber Defense Against Zero-Day Attacks using Ensemble Neural Networks
Project quality management in manufacturing
OOP with Java - Java Introduction (Basics)
KTU 2019 -S7-MCN 401 MODULE 2-VINAY.pptx
web development for engineering and engineering
UNIT-1 - COAL BASED THERMAL POWER PLANTS
additive manufacturing of ss316l using mig welding
BMEC211 - INTRODUCTION TO MECHATRONICS-1.pdf
MET 305 2019 SCHEME MODULE 2 COMPLETE.pptx
Well-logging-methods_new................
CH1 Production IntroductoryConcepts.pptx
Foundation to blockchain - A guide to Blockchain Tech
Digital Logic Computer Design lecture notes
UNIT 4 Total Quality Management .pptx
PRIZ Academy - 9 Windows Thinking Where to Invest Today to Win Tomorrow.pdf
Model Code of Practice - Construction Work - 21102022 .pdf
R24 SURVEYING LAB MANUAL for civil enggi
Sustainable Sites - Green Building Construction
Engineering Ethics, Safety and Environment [Autosaved] (1).pptx
Automation-in-Manufacturing-Chapter-Introduction.pdf

IRJET- Supervised Learning Classification Algorithms Comparison

  • 1. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 03 | Mar 2019 www.irjet.net p-ISSN: 2395-0072 © 2019, IRJET | Impact Factor value: 7.211 | ISO 9001:2008 Certified Journal | Page 3869 Supervised Learning Classification Algorithms Comparison Aditya Singh Rathore B.Tech, J.K. Lakshmipat University -------------------------------------------------------------***--------------------------------------------------------- Abstract: Under supervised machine learning, classification tasks are one of the most important tasks as a part of data analysis. It gives a lot of actionable insights to data scientists after using different machine learning algorithms. For study purpose the Titanic dataset has been chosen. Through this paper, an effort has been made to evaluate different classification models’ performance on the dataset using the scikit-learn library. The classifiers chosen are Logistic Regression, K-Nearest Neighbor, Decision Tree, Random Forest, Support Vector Machine and Naïve Bayes classifier. In the end evaluation parameters like confusion matrix, precision, recall, f1-score and accuracy have been discussed. Key Words: Numpy, Pandas, classification, knn, logistic regression, decision tree, random forest, Naïve Bayes, SVM, sklearn. 1. INTRODUCTION: Machine learning has been getting a lot of attention for the past few years because it holds the key to solve the problems of the future. In the 21st Century, we are living in a world where data is everywhere and as we go by of our day we are generating data, be it text messaging or simply walking down the street and different antennas picking GPS signals. The majority of practical machine learning uses supervised learning. In supervised learning, the machine learning models are first trained over dataset with associated correct labels. The objective of training the models first is to create a very good mapping between the independent and dependent variables, so that when that model is run on a new dataset, it can predict the output variable. Classification is a type of problem under supervised learning. A classification problem is when the output is a category and when given one or more inputs a classification model will try to predict the value based on what relation it developed between the input and output. Below are the classifiers chosen for evaluation: 1.1 Logistic Regression: This type of technique proves to be more appropriate when the dependent variable/output is in the form of binary output, i.e. 1 or 0. This technique is mainly used when one wants to see the relationship between the dependent variable and one or more independent variable provided that the dataset is linear. The output of this technique gives out a binary output which is a better type of output than absolute values as it elucidates the output. In order to limit our output between 0 and 1 we will be using Sigmoid Function. Below is the mathematical representation of Sigmoid Function: f(x)= 1/(1+e^-x)
  • 2. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 03 | Mar 2019 www.irjet.net p-ISSN: 2395-0072 © 2019, IRJET | Impact Factor value: 7.211 | ISO 9001:2008 Certified Journal | Page 3870 1.2 K- Nearest Neighbor: This type of technique is one of the simplest classification algorithm. It first stores the entire data in its training phase as an array of data points. . Whenever a prediction is required for a new data it searches through the entire training dataset for a number of most similar instances and the data with most similar instance is returned as prediction. This number is defined by an integer ‘k’. The k denotes the number of neighbours which are the determining class of the test data. For example, if k=3, then the labels of the 3 closest instances are checked and the most common label is assigned to the test data. The distance measured from test data to the closest instance is the least distance measure. This can be achieved using various measuring tools such as Euclidean Distance. The Euclidean distance between point p and q is calculated as below: 1.3 Decision Tree: A decision tree is a graphical representation of all possible solutions to a decision based on certain conditions. It consists of a Root/Parent node which is the base node or the first node of a tree. Then comes the splitting which is basically dividing the Root/Child nodes into different parts on a certain condition. These parts are called the Branch/ Sub Tree. To remove unwanted branches from the tree we make use of Pruning. The last node is called the leaf node when the data cannot be segregated to further levels. We will use CART (Classification and Regression Tree) algorithm to our dataset. Below are some terminologies used by CART algorithm in order to select the root node : a) Gini Index: It is the measure of impurity used to build a decision tree b) Information Gain: It is the decrease in entropy after a dataset is split on the basis of a criteria. c) Reduction in Variance: The split with lower variance is selected as a criteria. d) Chi Square: It is the measure of statistical significance between the differences between sub-nodes and parent node.
  • 3. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 03 | Mar 2019 www.irjet.net p-ISSN: 2395-0072 © 2019, IRJET | Impact Factor value: 7.211 | ISO 9001:2008 Certified Journal | Page 3871 1.4 Random Forest: This classifier is one of the most used algorithms today because it can be used for classification as well as regression. The main problem with decision trees is that it can over fit the data. One solution to prevent this is to create multiple decision trees on random subsamples of the data. Now when a new data point is to be predicted then that will be computed by all decision trees and a majority vote will be calculated. This vote will determine the class of the data. This in turns gives higher accuracy when compared to decision tree alone. More decision trees directly correlates with more accuracy. 1.5 Support Vector Machine (SVM): This classifier uses a hyperplane/decision boundary that separates the classes. This hyperplane is positioned in such a way that it maximises the margin i.e. the space between the decision boundary and the points in each of the classes that are closest to the decision boundary. These points are called Support Vectors. A hyperplane is a decision surface that splits the space into 2 parts i.e. it is a binary classifier. This hyperplane in R^n space is an n-1 dimensional subspace. SVMs can easily be used for linear classification. However, in order to use SVM for non-linear classification once can use a kernel trick with which the data is transformed in another dimension to determine the appropriate position of hyperplane. 1.6 Naïve Bayes: This classifier is used primarily for text classification which generally involves training on high dimensional datasets. This classifier is relatively faster than other algorithms in making predictions. It is called “naïve” because it makes an assumption that a feature in the dataset is independent of the occurrence of other features i.e. it is conditionally independent. This classifier is based on the Bayes Theorem which states that the probability of an event Y given X is equal to the probability of the event X given Y multiplied by the probability of X upon probability of Y. P(X|Y) =(P(Y|X) ⋅ P(X))/P(Y)
  • 4. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 03 | Mar 2019 www.irjet.net p-ISSN: 2395-0072 © 2019, IRJET | Impact Factor value: 7.211 | ISO 9001:2008 Certified Journal | Page 3872 2. METHODOLOGY: 2.1 Dataset: The data has been split into two groups: a) training set (train.csv) b) test set (test.csv) The training set is used to build and train the machine learning models. The test set is used to see how well the model performed on unseen data. We will use the trained models to predict whether or not the passengers in the test set survived the sinking of the Titanic. Below is the original structure of the dataset: Data Dictionary:  Survived: 0 = No, 1 = Yes  pclass: Ticket class 1 = 1st, 2 = 2nd, 3 = 3rd  sibsp: # of siblings / spouses aboard the Titanic  parch: # of parents / children aboard the Titanic  ticket: Ticket number  cabin: Cabin number  embarked: Port of Embarkation C = Cherbourg, Q = Queenstown, S = Southampton After pre-processing of the data and feature engineering, below is the dataset we would be finally be working with: 2.2 Evaluation Parameters: 2.2.1 Confusion Matrix: On a classification problem a confusion matrix gives out a summary of predicted results. The basic idea behind it is to count the number of times the instances of class A are classified as class B.
  • 5. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 03 | Mar 2019 www.irjet.net p-ISSN: 2395-0072 © 2019, IRJET | Impact Factor value: 7.211 | ISO 9001:2008 Certified Journal | Page 3873 Below is the sample confusion matrix: Pridicted Yes Pridicted No Actual Yes True Positive False Negative Actual No False Positive True Negative 2.2.2 Precision: It is the ratio of total number of true positives divided by total sum of true positives and false positives. In summary it describes how many of the returned hits were true positive i.e. how many of the found were correct hits. Precision = True Positives / (True Positives + False Positives) 2.2.3 Recall: It is the ratio of the total number of true positives divided by total sum of the true positives and the false negatives. It is also called sensitivity. In summary it describes how many of the true positives were recalled (found), i.e. how many of the correct hits were also found. Recall = True Positives / (True Positives + False Negatives) 2.2.4 F1- Score: It is simply the weighted average of precision and recall wherein the best value for a model is 1 and the worst value is 0. Below is the formula to calculate the same: F1-score = 2 * (precision * recall) / (precision + recall) 2.3 Experimental Results: Below are the experimental results of different Classifiers: Classifier Class Classification Report Confusion Matrix Precision Recall f1-score Accuracy Logistic Regression 0 0.81 0.86 0.84 78.90% [[120 19] [ 28 56]]1 0.75 0.67 0.7 KNN 0 0.73 0.83 0.77 69.90% [[115 24] [ 43 41]]1 0.63 0.49 0.55 Decision Tree 0 0.84 0.78 0.81 77.17% [[108 31] [ 20 64]]1 0.67 0.76 0.72 Random Forest 0 0.87 0.88 0.88 84.30% [[123 16] [ 19 65]]1 0.8 0.77 0.79 SVM 0 0.73 0.76 0.74 67.20% [[106 33] [ 40 44]]1 0.57 0.52 0.55 Naïve Bayes 0 0.83 0.83 0.83 78.47% [[115 24] [ 24 60]]1 0.71 0.71 0.71
  • 6. International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 03 | Mar 2019 www.irjet.net p-ISSN: 2395-0072 © 2019, IRJET | Impact Factor value: 7.211 | ISO 9001:2008 Certified Journal | Page 3874 3. CONCLUSION: In this paper we experimented with 6 types of classifiers on the Titanic dataset. In order to evaluate the above the train dataset was split into train & test dataset, since there is no true output for test dataset. From the above the random forest seems to be most effective with highest precision and recall values and has the least number of false positive and false negatives. 4. REFERENCES: [1].P-N. Tan, M. Steinbach, V. Kumar, “Introduction to Data Mining,” Addison-Wesley Publishing, 2006. [2]. Simm, J., Magrans de Abril, I., & Sugiyama, M., Tree-based ensemble multi-task learning method for classification and regression, 2014. [3] Takeuchi, I. & Sugiyama, M, Target neighbor consistent feature weighting for nearest neighbor classification. [4] M. Kantardzic, “Data Mining: Concepts, Models, Methods, and Algorithms,” John Wiley & Sons Publishing, 2003.