![Using Python
#Import Library of Gaussian Naive Bayes model
from sklearn.naive_bayes import GaussianNB
import numpy as np
#assigning predictor and target variables
X= np.array([[-3,7],[1,5], [1,2], [-2,0], [2,3], [-4,0], [-1,1], [1,1], [-2,2],
[2,7], [-4,1], [-2,7]])
y = np.array([3, 3, 3, 3, 4, 3, 3, 4, 3, 4, 4, 4])
#Create a Gaussian Classifier
model = GaussianNB()
# Train the model using the training sets
model.fit(X, y)
#Predict Output
predicted= model.predict([[1,2],[3,4]])
print(predicted)](https://image.slidesharecdn.com/naive-220827042219-07a0c734/85/Naive-pdf-9-320.jpg)
Naive.pdf
- 1. Naive Bayes Techno Main Salt Lake By Name : Mahim Majee Stream : CSE Semester : 7th Section : A Roll No. : 13000119055 Registration No. : 016704 of 2019-20 1
- 2. Introduction Classification technique based on Bayes’ Theorem With “naive” assumption of independence among predictors. Easy to build Particularly useful for very large data sets Known to outperform even highly sophisticated classification methods a. e.g. Earlier method for spam detection
- 3. Introduction - Bayes theorem ● P(c|x) - the posterior probability of class (c, target) given predictor (x, attributes). ● P(c) - the prior probability of class. ● P(x|c) - is the likelihood which is the probability of predictor given class. ● P(x) - is the prior probability of predictor.
- 4. How Naive Bayes algorithm works We are training data set of weather and corresponding target variable ‘Play’ (suggesting possibilities of playing). Now, we need to classify whether players will play or not based on weather condition.
- 5. Convert the data set into a frequency table Create Likelihood table by finding the probabilities like Overcast probability = 0.29 and probability of playing is 0.64. Now, use Naive Bayesian equation to calculate the posterior probability for each class. The class with the highest posterior probability is the outcome of prediction.
- 6. Problem: Players will play if weather is sunny. Is this statement is correct? P(Yes | Sunny) = P( Sunny | Yes) * P(Yes) / P (Sunny) Here we have P (Sunny |Yes) = 3/9 = 0.33, P(Sunny) = 5/14 = 0.36, P( Yes)= 9/14 = 0.64 Now, P (Yes | Sunny) = 0.33 * 0.64 / 0.36 = 0.60, (high probability)
- 7. Problem: Players will play if weather is sunny. Is this statement is correct? P(No | Sunny) = P( Sunny | No) * P(No) / P (Sunny) Here we have P (Sunny |No) = 2/5 = 0.4, P(Sunny) = 5/14 = 0.36, P( No)= 5/14 = 0.36 Now, P (No | Sunny) = 0.4 * 0.36 / 0.36 = 0.40, (low probability)
- 8. Zero Frequency Problem What if any of the count is 0? Add 1 to all counts It is a form of Laplace smoothing
- 9. Using Python #Import Library of Gaussian Naive Bayes model from sklearn.naive_bayes import GaussianNB import numpy as np #assigning predictor and target variables X= np.array([[-3,7],[1,5], [1,2], [-2,0], [2,3], [-4,0], [-1,1], [1,1], [-2,2], [2,7], [-4,1], [-2,7]]) y = np.array([3, 3, 3, 3, 4, 3, 3, 4, 3, 4, 4, 4]) #Create a Gaussian Classifier model = GaussianNB() # Train the model using the training sets model.fit(X, y) #Predict Output predicted= model.predict([[1,2],[3,4]]) print(predicted)
- 10. Tips to improve the Naive Bayes Model If continuous features do not have normal distribution, we should use transformation or different methods to convert If test data set has zero frequency issue, apply smoothing techniques “Laplace smoothing” Remove correlated features, as the highly correlated features are voted twice in the model and it can lead to over inflating importance. Naive Bayes classifier has limited options for parameter tuning Can’t be ensembled - because there is no variance to reduce
- 11. Variants Gaussian: It is used in classification and it assumes that features follow a normal distribution. Multinomial: It is used for discrete counts. Implements the naive Bayes algorithm for multinomially distributed data. It is one of the two classic naive Bayes variants used in text classification Bernoulli: The binomial model is useful if your feature vectors are binary (i.e. zeros and ones). One application would be text classification with ‘bag of words’ model where the 1s & 0s are “word occurs in the document” and “word does not occur in the document” respectively.
- 12. Applications Real time Prediction: Naive Bayes is an eager learning classifier and it is sure fast. Thus, it could be used for making predictions in real time. Multi class Prediction: Well known for multi class prediction feature. Text classification/ Spam Filtering/ Sentiment Analysis: Mostly used in text classification. Have higher success rate as compared to other algorithms. Widely used in Spam filtering (identify spam e-mail) and Sentiment Analysis. Recommendation System: Naive Bayes Classifier and Collaborative Filtering together builds a Recommendation System that uses machine learning and data mining techniques to filter unseen information and predict whether a user would like a given resource or not
- 13. Conclusions The naive Bayes model is tremendously appealing because of its simplicity, elegance, and robustness. It is one of the oldest formal classification algorithms, and yet even in its simplest form it is often surprisingly effective. It is widely used in areas such as text classification and spam filtering. A large number of modifications have been introduced, by the statistical, data mining, machine learning, and pattern recognition communities, in an attempt to make it more flexible. But someone has to recognize that such modifications are necessarily complications, which detract from its basic simplicity.
- 14. References 1.9. Naive Bayes — scikit-learn 1.1.1 documentation https://en.wikipedia.org/wiki/Bayes%27_theorem Naive Bayes classifier – Wikipedia http://en.wikipedia.org/wiki/Naive_Bayes_classifier http://www.cs.cmu.edu/afs/cs.cmu.edu/project/theo20/www/mlbo ok/ch6.pdf Data Mining: Concepts and Techniques, 3rd Edition, Han & Kamber & Pei ISBN: 9780123 814791