Lecture 02 ml supervised and unsupervised
- 1. Machine Learning types Dr. Mostafa A. Elhosseini YouTube Channel
- 2. ML Jargons ≡ Training set: the examples that the system uses to learn ≡ Training instance: Training sample -> each training example ≡ Label: spam or not ≡ feature = attribute + value ≡ Predictors: features ≡ Ex. Price of a car ▪ Set of features (mileage, age, brand) called predictors ▪ Feature : Mileage = 15,000 ≡ Performance measure P: ≡ Ratio of correctly classified emails
- 3. Machine learning definition ≡ Machine Learning is the science (and art) of programming computers so they can learn from data ≡ The field of study that gives computers the ability to learn without being explicitly programmed – Arthur Samuel, 1959 ≡ A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E. Tom Mitchell, 1997
- 4. ML types ≡ Whether or not they are trained with human supervision ▪ supervised, ▪ unsupervised, ▪ semisupervised, and ▪ Reinforcement Learning ≡ Whether or not they can learn incrementally on the fly ▪ online versus batch learning ≡ Whether they work by simply comparing new data points to known data points, or instead detect patterns in the training data and build a predictive model, much like scientists do ▪ instance-based versus model-based learning
- 5. Supervised / Unsupervised ≡ In supervised learning, the training data you feed to the algorithm includes the desired solutions, called labels ▪ Classification ▪ Regression ≡ Classification ▪ The spam filter is a good example of this: it is trained with many example emails along with their class (spam or ham), and it must learn how to classify new
- 6. Supervised / Unsupervised ≡ Regression ▪ predict a target numeric value, such as the price of a car, given a set of features (mileage, age, brand, etc.) called predictors. ≡ some regression algorithms can be used for classification as well ▪ Logistic Regression is commonly used for classification, as it can output a value that corresponds to the probability of belonging to a given class
- 7. Supervised / Unsupervised ≡ The training data is unlabeled. ≡ The system tries to learn without a teacher. ≡ Most important unsupervised Applications ▪ Clustering ▪ Blog visitors ▪ Visualization and dimensionality reduction ▪ Association rule learning
- 8. Unsupervised// Dimensionality Reduction ≡ Simplify the data without losing too much information ≡ One way to do this is to merge several correlated features into one ▪ car’s mileage may be very correlated with its age, so the dimensionality reduction algorithm will merge them into one feature that represents the car’s wear and tear. This is called feature extraction ≡ It is often a good idea to try to reduce the dimension of your training data ▪ It will run much faster, ▪ The data will take up less disk and memory space, and in some cases ▪ It may also perform better
- 9. Unsupervised // anomaly detection ≡ Detecting unusual credit card transactions to prevent fraud, catching manufacturing defects, or automatically removing outliers from a dataset before feeding it to another learning algorithm. ≡ The system is trained with normal instances, and when it sees a new instance it can tell whether it looks like a normal one or whether it is likely an anomaly
- 10. Unsupervised // Association rule learning ≡ The goal is to dig into large amounts of data and discover interesting relations between attributes. ≡ For example, suppose you own a supermarket. Running an association rule on your sales logs may reveal that people who purchase barbecue sauce and potato chips also tend to buy steak. Thus, you may want to place these items close to each other.
- 11. Batch/ Online learning ≡ Incapable of learning incrementally ≡ Trained using all the available data ≡ Take a lot of time and computing resources ≡ Done offline ≡ Ex. If a new type of spam -> you need to train a new version of the system from scratch on the full dataset
- 12. Batch/ Online learning ≡ you train the system incrementally by feeding it data instances sequentially, either individually or by small groups called mini- batches. ≡ Each learning step is fast and cheap, so the system can learn about new data on the fly, as it arrives ≡ This whole process is usually done offline (i.e., not on the live system), so online learning can be a confusing name. Think of it as incremental learning. ≡ Learning rate ▪ How fast they should adapt to changing data
- 13. Instance-based // Model-based ≡One more way to categorize Machine Learning systems is by how they generalize ▪ Needs to be able to generalize to examples it has never seen before ≡ Having a good performance measure on the training data is good, but insufficient; the true goal is to perform well on new instances ≡ There are two main approaches to generalization: instance-based learning and model-based learning
- 14. Instance-based // Model-based ≡Simply to learn by heart ▪ For Spam filter: just flag all emails that are identical to emails that have already been flagged by users ▪ Not the worst solution, but certainly not the best. ≡ Instead of just flagging emails that are identical to known spam emails, your spam filter could be Programmed to also flag emails that are very similar to known spam emails. ≡ This requires a measure of Similarity between two emails ≡ The system would flag an email as spam if it has many words in common with a known spam email
- 16. Instance-based // Model-based ≡ Another way to generalize from a set of examples is to build a model of these examples, then use that model to make predictions. This is called model-based learning