Text classification-php-v4
5 likes3,723 views
This document summarizes text classification in PHP. It discusses what text classification is, common natural language processing terminology like tokenization and stemming, Bayes' theorem and how it relates to naive Bayes classification. It provides examples of tokenizing, stemming, stopping words, and building a naive Bayes classifier in PHP using the NlpTools library. Key steps like training and testing a classifier on sample text data are demonstrated.
![Classification (training 2/2)
// iterate over training array
foreach ($training as $trainingDocument){
// add to our training set
$trainingSet->addDocument(
// class
$trainingDocument[0],
// document
new TokensDocument($tokenizer->tokenize($trainingDocument[1]))
);
}
// train our Naive Bayes Model
$bayesModel = new FeatureBasedNB();
$bayesModel->train($features, $trainingSet);](https://image.slidesharecdn.com/text-classification-php-v4-160111204742/85/Text-classification-php-v4-42-320.jpg)
![Classification (classifying)
$testSet = array(
array('us','i want to see the statue of liberty'),
array('uk','i saw the big ben yesterday’),
…
);
// init our Naive Bayes Class using the features and our model
$classifier = new MultinomialNBClassifier($features, $bayesModel);
// iterate over our test set
foreach ($testSet as $testDocument){
// predict our sentence
$prediction = $classifier->classify(
array('new york','us'), // the classes that can be predicted
new TokensDocument($tokenizer->tokenize($testDocument[1])) // the sentence
);
printf("sentence: %s | class: %s | predicted: %sn”,
$testDocument[1], $testDocument[0], $prediction );
}](https://image.slidesharecdn.com/text-classification-php-v4-160111204742/85/Text-classification-php-v4-43-320.jpg)
Text classification-php-v4
- 1. Text classification in PHP
- 2. Who am I ? Glenn De Backer (twitter: @glenndebacker) Web developer @ Dx-Solutions 32 years old originally from Bruges, now living in Meulebeke Interested in machine learning, (board) games, electronics and have a bit of a creative bone… Blog: http://www.simplicity.be
- 3. What will we cover today ? What is text classification NLP terminology Bayes theorem Some PHP code
- 4. What is text classification ? Text classification is the process of assigning classes to documents This can be done manually or by using machine learning (algorithmically) Today`s talk will be about classifying text using a supervised machine learning algorithm: Naive bayes
- 5. Supervised vs unsupervised machine learning ? Supervised means in simple terms that we need to feed our algorithm examples of data and what they represent Free gift card -> spam The server is down -> ham Unsupervised means that we work with algorithms that finds hidden structure in unlabelled data. For example clustering documents
- 6. Some possible use cases Spam detection (classic) Assigning categories, topics, genres, subjects, … Determine authorship Gender classification Sentiment analysis Identifying languages …
- 7. Personal project Nieuws zonder politiek
- 8. Personal project Nieuws zonder politiek Fun project from 2010 Related to the 589 days with no elected government. We had a lot of political related non-news items that I wanted to filter out as an experiment. News aggregator that fetched news from different flemish newspapers Classified those items into political and non political news
- 9. Personal project Wuk zeg je ?
- 10. Personal project Wuk zeg je ? Fun project released at the end of 2015 Inspired by a contest of the province of West Flanders to find foreign words that sounded West-Flemish Can recognise the West-Flemish dialect… but also Dutch, French and English Uses character n-grams instead of words
- 11. NLP terminology
- 12. Tokenization Before any real text processing can be done we need to execute the task of tokenization. Tokenisation is the task of dividing text into words, sentences, symbols or other elements called tokens. They often talk about features instead of tokens.
- 13. N-grams N-gram are sequences of tokens of length N Can be words, combination of words, characters, … . Depending on the size it also sometimes called a unigram (1 item), bigram (2 items) or a trigram (3 items). Character n-grams are very suited for language classification
- 14. Stop words Are words (or features) that are particular common in a text corpus for example the, and, on, in, … Are considered uninformative A list of stopwords is used to remove or ignore words from the document we are processing Optional but recommended
- 15. Stemming Stemming is the process of reducing words to their word stem, base or root. Not a required step but it can certainly help in reducing the number of features and improving the task of classifying text (e.g. speed or quality) The most used is the Porter stemmer which contains support for English, French, Dutch, …
- 16. Bag Of Words (BOW) model Is a simple representation of text features Can be words, combination of words, sounds, … . A Bow model contains a vocabulary including a vocabulary count
- 17. Training / test set A training set is just a collection of a labeled data used for classifying data. Free gift card -> spam The server is down -> ham A test set is simply to test the accuracy of our classifier
- 18. A typical flow PHP is a server-side scripting language designed for web development
- 19. A typical flow PHP | is | a | server-side | scripting | language | designed | for | web | development
- 20. A typical flow PHP | is | a | server-side | scripting | language | designed | for | web | development
- 21. A typical flow PHP | server-side | scripting | language | designed | web | development
- 22. A typical flow PHP : 1 server-side : 1 scripting : 1 language : 1 designed : 1 web : 1 development : 1
- 23. Bayes theorem
- 24. Some history trivia Discovered by a British minister Thomas Bayes in 1740. Rediscovered independently by a French scholar Piere Simon Laplace who gave it its modern mathematical form. Alan Turing used it to decode the German Enigma Cipher which had a big influence on the outcome of World War 2.
- 25. Bayes theorem In probability theory or statistics Bayes theorem describes the probability of an event based on conditions that might relate to that event. E.g. how probable it is that an article is about sports (and that based on certain words that the article contains).
- 26. Naive Bayes Naive Bayes classifiers are a family of simple probabilistic classifiers based on applying Bayes theorem The naive part is the fact that it strongly assume independence between features (words in our case)
- 27. Bayes and text classification We can modify the standard Bayes formule as: Where C is the class… and D is the document We can drop P(D) as this is a constant in this case. This is a very common thing to do when using Naive Bayes for classification problems.
- 28. Probability of a class Where Dc is the number of documents in our training set that have this class… and Dt is the total number of documents in our training set
- 29. Probability of a class given a document Where wx are the words of our text What is the (joint) probability of word 1, word 2, word 3, … given our class
- 30. Enough abstract formulas for today, 2 simplified examples
- 31. We have the following data* word good bad total server 5 6 11 crashed 2 14 16 updated 9 1 10 new 8 1 9 total 24 22 46 * in reality your data will contain a lot more words and higher counts
- 32. word good bad total server 5 6 11 crashed 2 14 16 … … … … total 24 22 46 The server has crashed (We applied a stopword filter that removes the words “the” and “has”)
- 33. word good bad total server 5 6 11 updated 9 1 10 new 8 1 9 … … … … total 24 22 46 The new server is updated (We applied a stopword filter that removes the words “the” and “is”)
- 34. NLP in PHP
- 35. NlpTools NlpTools is a library for natural language processing written in PHP Classes for classifying, tokenizing, stemming, clustering, topic modeling, … . Released under the WTFL license (Do what you want)
- 36. Tokenizing a sentence // text we will be converting into tokens $text = "PHP is a server side scripting language."; // initialize Whitespace and punctuation tokenizer $tokenizer = new WhitespaceTokenizer(); // print array of tokens print_r($tokenizer->tokenize($text));
- 37. Dealing with stop words // text we will be converting into tokens $text = "PHP is a server side scripting language."; // define a list of stop words $stop = new StopWords(array("is", "a", "as")); // initialize Whitespace tokenizer $tokenizer = new WhitespaceTokenizer(); // init token document $doc = new TokensDocument($tokenizer->tokenize($text)); // apply our stopwords $doc->applyTransformation($stop); // print filtered tokens print_r($doc->getDocumentData());
- 38. Dealing with stop words
- 39. Stemming words // init PorterStemmer $stemmer = new PorterStemmer(); // stemming variants of upload printf("%sn", $stemmer->stem("uploading")); printf("%sn", $stemmer->stem("uploaded")); printf("%sn", $stemmer->stem("uploads")); // stemming variants of delete printf("%sn", $stemmer->stem("delete")); printf("%sn", $stemmer->stem("deleted")); printf("%sn", $stemmer->stem("deleting"));
- 40. Stemming words
- 41. Classification (training 1/2) $training = array( array('us','new york is a hell of a town'), array('us','the statue of liberty'), array('us','new york is in the united states'), array('uk','london is in the uk'), array('uk','the big ben is in london’), … ); // hold our training documents $trainingSet = new TrainingSet(); // our tokenizer $tokenizer = new WhitespaceTokenizer(); // will hold the features we will be working $features = new DataAsFeatures();
- 42. Classification (training 2/2) // iterate over training array foreach ($training as $trainingDocument){ // add to our training set $trainingSet->addDocument( // class $trainingDocument[0], // document new TokensDocument($tokenizer->tokenize($trainingDocument[1])) ); } // train our Naive Bayes Model $bayesModel = new FeatureBasedNB(); $bayesModel->train($features, $trainingSet);
- 43. Classification (classifying) $testSet = array( array('us','i want to see the statue of liberty'), array('uk','i saw the big ben yesterday’), … ); // init our Naive Bayes Class using the features and our model $classifier = new MultinomialNBClassifier($features, $bayesModel); // iterate over our test set foreach ($testSet as $testDocument){ // predict our sentence $prediction = $classifier->classify( array('new york','us'), // the classes that can be predicted new TokensDocument($tokenizer->tokenize($testDocument[1])) // the sentence ); printf("sentence: %s | class: %s | predicted: %sn”, $testDocument[1], $testDocument[0], $prediction ); }
- 44. Classification
- 45. Some tips It is a best practice to split your data in a training and test set instead of training on your whole dataset! If you train your classifier against the whole dataset it can happen that it will be very accurate on the dataset but performs badly on unseen data, this is also called overfitting in machine learning. There isn’t a best split but 80-20 (Pareto principle) or 70-30 are safe ratio’s. The numbers tells the tale! There are multiple ways of telling how accurate your classifier performs but precision and recall are a good start ! - http://www.kdnuggets.com/faq/ precision-recall.html
- 46. Some online PHP resources http://www.php-nlp-tools.com/ - The homepage of NlpTools http://www.phpir.com - Contains a lot of tutorials regarding information retrieval in PHP https://github.com/camspiers/statistical- classifier - An alternative Bayes Classifier but also supports SVM
- 47. Reading material Code examples written in Java and Python but concepts can easily be applied in other languages…
- 48. PHP NLP projects released as open source php-dutch-stemmer: is a PHP class that stems Dutch words. Based on Porters algorithm. https://github.com/simplicitylab/php-dutch-stemmer php-luhn-summarize: is a class that provides a basic implementation of Luhn’s algorithm. This algorithm can automatically create a summary of a given text. https://github.com/simplicitylab/php-luhn-summarize
- 50. Thank you !