Introduction to Machine Learning - An overview and first step for candidate deep-divers and future developers
Download as PPTX, PDF1 like168 views
The document provides a comprehensive introduction to machine learning, discussing its relevance, methodologies, and applications, including supervised, unsupervised, and reinforcement learning. It details the data science workflow, tools, and algorithms, emphasizing the importance of big data, open data, and platforms like Oracle and Apache Spark. The summary also highlights the democratization of machine learning and its business implications, encouraging practitioners to explore and adopt these technologies.
![X = [X1,X2,X3,…,XN]](https://image.slidesharecdn.com/introductionoverviewmachinelearning-siggilde-7may2018-180511141254/85/Introduction-to-Machine-Learning-An-overview-and-first-step-for-candidate-deep-divers-and-future-developers-6-320.jpg)
![AGENDA
• What is Machine Learning?
• Why could it be relevant [to you]?
• What does it entail?
• With which algorithms, tools and technologies?
• Oracle and Machine Learning?
• How do you embark on Machine Learning?
• Dinner
• Handson
• Functional/non-technical
• Technical](https://image.slidesharecdn.com/introductionoverviewmachinelearning-siggilde-7may2018-180511141254/85/Introduction-to-Machine-Learning-An-overview-and-first-step-for-candidate-deep-divers-and-future-developers-8-320.jpg)
![MACHINE LEARNING
• Analyze Historical Data (input and result – training set) to discover
Patterns & Models
• Iteratively apply Models to [additional] Input (test set) and compare
model outcome with known actual result to improve the model
• Use Model to predict
outcome for
entirely new data
10](https://image.slidesharecdn.com/introductionoverviewmachinelearning-siggilde-7may2018-180511141254/85/Introduction-to-Machine-Learning-An-overview-and-first-step-for-candidate-deep-divers-and-future-developers-10-320.jpg)
![OPEN DATA – SOME EXAMPLES
• Kaggle - Data Sets and [Samples of] Data Discovery: www.kaggle.com
• US, EU and UK Government Data: data.gov, open-data.europa.eu and data.gov.uk
• Open Images Data Set: www.image-net.org
• Open Data From World Bank: data.worldbank.org
• Historic Football Data: api.football-data.org
• New York City Open Data - opendata.cityofnewyork.us
• Airports, Airlines, Flight Routes: openflights.org
• Open Database – machine counterpart to Wikipedia: www.wikidata.org
• Google Audio Set (manually annotated audio events)
- research.google.com/audioset/
• Movielens - Movies, viewers and ratings:
files.grouplens.org/datasets/movielens/](https://image.slidesharecdn.com/introductionoverviewmachinelearning-siggilde-7may2018-180511141254/85/Introduction-to-Machine-Learning-An-overview-and-first-step-for-candidate-deep-divers-and-future-developers-46-320.jpg)
![DEMO: CONFERENCE ABSTRACT
CLASSIFICATION CHALLENGE
• Take all conference abstracts for
• Train a Classification Model on
picking the Conference Track
• Based on Title, Summary [, Speaker, Level,…]
• Use the Model to pick the Track
for sessions at](https://image.slidesharecdn.com/introductionoverviewmachinelearning-siggilde-7may2018-180511141254/85/Introduction-to-Machine-Learning-An-overview-and-first-step-for-candidate-deep-divers-and-future-developers-53-320.jpg)
![MANY CLOUD SERVICES AROUND BIG DATA &
[PREDICTIVE] ANALYTICS & MACHINE LEARNING
60](https://image.slidesharecdn.com/introductionoverviewmachinelearning-siggilde-7may2018-180511141254/85/Introduction-to-Machine-Learning-An-overview-and-first-step-for-candidate-deep-divers-and-future-developers-59-320.jpg)
![SUMMARY
• R and Python are most popular technologies for data exploration
and ML model discovery [on small subsets of Big Data]
• Apache Spark (on Hadoop) is frequently used to powercrunch data
(wrangling) and run ML models on Big Data sets
• Notebooks are a popular vehicle in the Data Science lab
• To explore and report
• Oracle is quite active on Machine Learning
• Power PaaS and SaaS with ML
• Provide us with the Machine Learning Data Lab & Run Time (on the cloud)
• Getting started on Machine Learning is fun, smart & well supported](https://image.slidesharecdn.com/introductionoverviewmachinelearning-siggilde-7may2018-180511141254/85/Introduction-to-Machine-Learning-An-overview-and-first-step-for-candidate-deep-divers-and-future-developers-65-320.jpg)
Introduction to Machine Learning - An overview and first step for candidate deep-divers and future developers
- 1. THE ART OF INTELLIGENCE – A PRACTICAL INTRODUCTION MACHINE LEARNING AMIS SIG & Conclusion Gilde AI & Machine Learning Mei 2018
- 8. AGENDA • What is Machine Learning? • Why could it be relevant [to you]? • What does it entail? • With which algorithms, tools and technologies? • Oracle and Machine Learning? • How do you embark on Machine Learning? • Dinner • Handson • Functional/non-technical • Technical
- 9. LEARNING • How do we learn? • Try something (else) => get feedback => learn • Eventually: • We get it (understanding) so we can predict the outcome of a certain action in a new situation • Or we have experienced enough situations to predict the outcome in most situations with high confidence • Through interpolation, extrapolation, etc. • We remain clueless 9
- 10. MACHINE LEARNING • Analyze Historical Data (input and result – training set) to discover Patterns & Models • Iteratively apply Models to [additional] Input (test set) and compare model outcome with known actual result to improve the model • Use Model to predict outcome for entirely new data 10
- 11. WHY IS IT RELEVANT (NOW)? • Data • big, fast, open • Machine Learning has become feasible and accessible • Available • Affordable (software & hardware) • Doable (Citizen Data Scientist) • Fast enough • Business Cases & Opportunities => Demands • End users, Consumers, Competitive pressure, Society
- 12. WHY IS IT RELEVANT (NOW)?
- 13. GARTNER – STRATEGIC TECHNOLOGY TRENDS 2018
- 14. EXAMPLE USE CASES • Speech recognition • Identify churn candidates • Intent & Sentiment analysis on social media • Upsell & Cross Sell • Target Marketing • Customer Service • Chat bots & voice response systems • Predictive Maintenance • Gaming • Captcha • Medical Diagnosis • Anomaly Detection (find the odd one out) • Autonomous Cars • Voter Segment Analysis • Customer Recommendations • Smart Data Capture • Face Detection • Fraud Prevention • (really good) OCR • Traffic light control • Navigation • Should we investigate | do lab test? • Spam filtering • Propose friends | contacts • Troll detection • Auto correct • Photo Tagging and Album organization
- 15. READY-TO-RUN ML APPS Someone else selected, configured and trained an ML model and makes it available for you to use against your own data
- 16. READY TO RUN ML APPS – SAAS POWERED BY ML #DevoxxMA
- 17. PRODUCTS WITH ML INSIDE #DevoxxMA
- 18. Do It Yourself Machine Learning
- 19. THE DATA SCIENCE WORKFLOW • Set Business Goal – research scope, objectives • Gather data • Prepare data • Cleanse, transform (wrangle), combine (merge, enrich) • Explore data • Model Data • Select model, train model, test model • Present findings and recommend next steps • Apply: • Make use of insights in business decisions • Automate Data Gathering & Preparation, Deploy Model, Embed Model in operational systems
- 20. DATA DISCOVERY 20 A B C D E F G 1104534 ZTR 0.1 anijs 2 36 T 631148 ESE 132 rivier 0 21 S -3 WGN 71 appel 0 1 - 1262300 ZTR 56 zes 2 41 T 315529 HVN 1290 hamer 0 11 - 788914 ASM 676 zwaluw 0 26 T 157762 HVN 9482 wie 0 6 - 946681 DHG 42 rond 1 31 T -31539 WGN 2423 bruin 0 0 - 47338 HVN 54 hamer 0 16 P
- 21. SCATTER PLOT ATTRIBUTE F (Y-AXIS)VS ATTRIBUTE A 21 0 5 10 15 20 25 30 35 40 45 -200000 0 200000 400000 600000 800000 1000000 1200000 1400000 Y-Values Y-Values
- 22. SCATTER PLOT ATTRIBUTE F (Y-AXIS)VS ATTRIBUTE A 22 0 5 10 15 20 25 30 35 40 45 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010 2015 Age of Lucas Jellema vs Year Y-Values
- 23. DATA DISCOVERY – ATTRIBUTES IDENTIFIED 23 Time of Birth City ? ? #Kids Age Level of Education 1104534 ZTR 0.1 anijs 2 36 T 631148 ESE 132 rivier 0 21 S -3 WGN 71 appel 0 1 - 1262300 ZTR 56 zes 2 41 T 315529 HVN 1290 hamer 0 11 - 788914 ASM 676 zwaluw 0 26 T 157762 HVN 9482 wie 0 6 - 946681 DHG 42 rond 1 31 T -31539 WGN 2423 bruin 0 0 - 47338 HVN 54 hamer 0 16 P
- 24. TYPES OF MACHINE LEARNING • Supervised • Train and test model from known data (both features and target) • Unsupervised • Analyze unlabeled data – see if you can find anything • Semi-Supervised • Interactive flow, for example human identifying clusters • Reinforcement • Continuously improve algorithm (model) as time progresses, based on new experience
- 25. MACHINE LEARNING ALGORITHMS • Clustering • Hierarchical k-means, Orthogonal Partitioning Clustering, Expectation-Maximization • Feature Extraction/Attribute Importance/Principal Component Analysis • Classification • Decision Tree, Naïve Bayes, Random Forest, Logistic Regression, Support Vector Machine • Regression • Multiple Regression, Support Vector Machine, Linear Model, LASSO, Random Forest, Ridgre Regression, Generalized Linear Model, Stepwise Linear Regression • Association & Collaborative Filtering (market basket analysis, apriori) • Reinforcement Learning – brute force, value function, Monte Carlo, temporal difference, .. • Neural network and Deep Learning with Deep Neural Network • Can be used for many different use cases
- 26. MODELING PHASE • Select a model to try to create a fit with (predict target well) • Set configuration parameters for model • Divide data in training set and test set • Train model with training set • Evaluate performance of trained model on the test set • Confusion matrix, mean square error, support, lift, false positives, false negatives • Optionally: tweak model parameters, add attributes, feed in more training data, choose different model • Eventually (hopefully): pick model plus parameters plus attributes that will reliably predict the target variable given new data • Possibly combine multiple models to collaborate on target value
- 27. OPTICAL DIGIT RECOGNITION == CLASSIFICATION Predicted Actual 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 Naïve Bayes Decision Tree Deep Neural Network
- 28. CLASSIFICATION GONE WRONG • Machine learning applied to millions of drawings on QuickDraw • to classify drawings • For example: drawings of beds • See for example: • https://aiexperiments.withgoogle.com/quick-draw
- 29. MACHINE LEARNING OPERATIONAL SYSTEMS • “We have a model that will choose best chess move based on certain input”
- 30. MACHINE LEARNING OPERATIONAL SYSTEMS • Discovery => Model => Deploy • “We have a model that will predict a class (classification) or value (regression) based on certain input with a meaningful degree of accuracy” – how can we make use of that model?
- 31. DEPLOY MODEL AND EXPOSE • Model is usually created on Big Data in Data Science environment using the Data Scientist’s tools • Model itself is typically fairly small • Model will be applied in operational systems against single data items (not huge collections nor the entire Big Data set) • Running the model online may not require extensive resources • Implementing the model at production run time • Export model (from Data Scientist environment) and import (into production environment) • Reimplement the model in the development technology and deploy (in the regular way) to the production environment • Expose model through API
- 32. 80M PICTURES OF ROAD
- 33. BIG DATA => SMALL ML MODELS
- 34. DEPLOY MODEL AND EXPOSE REST API
- 35. MODEL MANAGEMENT • Governance (new versions, testing and approval) • A/B testing • Auditing (what did the model decide and why? notifying humans? ) • Evaluation (how well did the model’s output match the reality) to help evolve the model • for example recommendations followed • Monitor self learning models (to detect rogue models)
- 36. WHAT TO DO IT WITH? • Mathematics (Statistics) • Gauss (normal distribution) • Bayes’ Theorem • Euclidean Distance • Perceptron • Mean Square Error
- 37. WHAT TO DO IT WITH?
- 38. TOOLS AND LIBRARIES IMPLEMENTING MACHINE LEARNING ALGORITHMS +
- 40. HOW TO PICK TOOLS FOR THE JOB • What are the jobs? • Gather data • Prepare data • Explore and (hopefully) Discover • Present • Embed & Deploy Model • What are considerations? • Volume • Speed and Time • Skills • Platform • Cost
- 42. POPULAR FRAMEWORKS & LIBRARIES • TensorFlow • MXNet • Caffe • DL4J • Keras • … many more… Oracle Database Option Advanced Analytics #DevoxxMA
- 43. NOTEBOOK – THE LAB JOURNAL FROM THE DATALAB • Common format for data exploration and presentation • User friendly interface on top of powerful technologies • Most popular implementations • Jupyter (fka IPython) • Apache Zeppelin • Spark Notebook • Beaker • SageMath (SageMathCloud => CoCalc) • Oracle Machine Learning Notebook UI • Try out Jupyter at: https://mybinder.org/
- 45. OPEN DATA • Governments and NGOs, scientific and even commercial organizations are publishing data • Inviting anyone who wants to join in to help make sense of the data – understand driving factors, identify categories, help predict • Many areas • Economy, health, public safety, sports, traffic & transportation, games, environment, maps, …
- 46. OPEN DATA – SOME EXAMPLES • Kaggle - Data Sets and [Samples of] Data Discovery: www.kaggle.com • US, EU and UK Government Data: data.gov, open-data.europa.eu and data.gov.uk • Open Images Data Set: www.image-net.org • Open Data From World Bank: data.worldbank.org • Historic Football Data: api.football-data.org • New York City Open Data - opendata.cityofnewyork.us • Airports, Airlines, Flight Routes: openflights.org • Open Database – machine counterpart to Wikipedia: www.wikidata.org • Google Audio Set (manually annotated audio events) - research.google.com/audioset/ • Movielens - Movies, viewers and ratings: files.grouplens.org/datasets/movielens/
- 47. WHAT IS HADOOP? • Big Data means Big Computing and Big Storage • Big requires scalable => horizontal scale out • Moving data is very expensive (network, disk IO) • Rather than move data to processor – move processing to data: distributed processing • Horizontal scale out => Hadoop: distributed data & distributed processing • HDFS – Hadoop Distributed File System • Map Reduce – parallel, distributed processing • Map-Reduce operates on data locally, then persists and aggregates results
- 48. WHAT IS SPARK? • Developing and orchestrating Map-Reduce on Hadoop is not simple • Running jobs can be slow due to frequent disk writing • Spark is for managing and orchestrating distributed processing on a variety of cluster systems • with Hadoop as the most obvious target • through APIs in Java, Python, R, Scala • Spark uses lazy operations and distributed in-memory data structures – offering much better performance • Through Spark – cluster based processing can be used interactively • Spark has additional modules that leverage distributed processing for running prepackaged jobs (SQL, Graph, ML, …)
- 50. EXAMPLE RUNNING AGAINST SPARK • https://github.com/jadianes/spark-movie-lens/blob/master/notebooks/building-recommender.ipynb
- 51. WHAT IS ORACLE DOING AROUND MACHINE LEARNING? • Oracle Advanced Analytics in Oracle Database • Data Mining, Enterprise R • Text (ESA), Spatial, Graph • SQL
- 53. DEMO: CONFERENCE ABSTRACT CLASSIFICATION CHALLENGE • Take all conference abstracts for • Train a Classification Model on picking the Conference Track • Based on Title, Summary [, Speaker, Level,…] • Use the Model to pick the Track for sessions at
- 54. DEMONSTRATION OF ORACLE ADVANCED ANALYTICS • Using Text Mining and Naives Bayes Data Mining Classification • Train model for classifying conference abstracts into tracks • Use model to propose a track for new abstracts • Steps • Gather data • Import, cleanse, enrich, … • Prepare training set and test set • Select and configure model • Combining Text and Mining using Naive Bayes • Train model • Test and apply model
- 55. TRAIN MODEL DECLARE xformlist dbms_data_mining_transform.TRANSFORM_LIST; BEGIN DBMS_DATA_MINING_TRANSFORM.SET_TRANSFORM( xformlist, 'abstract', NULL, 'abstract', NULL, 'TEXT(TOKEN_TYPE:NORMAL)'); DBMS_DATA_MINING.CREATE_MODEL ( model_name => 'SESSION_CLASS_NB' , mining_function => dbms_data_mining.classification , data_table_name => 'J1_SESSIONS' , case_id_column_name => 'session_title' , target_column_name => 'session_track' , settings_table_name => 'session_class_nb_settings' , xform_list => xformlist); END;
- 56. APPLY MODEL
- 57. APPLY MODEL
- 58. BIG DATA SQL ORACLE DATABASE AS SINGLE POINT OF ENTRY
- 59. MANY CLOUD SERVICES AROUND BIG DATA & [PREDICTIVE] ANALYTICS & MACHINE LEARNING 60
- 60. WHAT IS ORACLE DOING AROUND MACHINE LEARNING? • Big Data Discovery (fka Endeca), Big Data Preparation and Big Data Compute • Big Data Appliance • Data Visualization Cloud • Analytics Cloud • Industry specific Analytics Clouds (Sales, Marketing, HCM) on top of SaaS • RTD – Real Time Decisions • DaaS • Oracle Labs (labs.oracle.com) • Machine Learning Research Group (link) • Machine Learning CS – “Oracle Notebook”
- 61. HUMANS LEARNING MACHINE LEARNING: YOUR FIRST STEPS #DevoxxMA
- 62. HUMANS LEARNING MACHINE LEARNING: YOUR FIRST STEPS • Jupyter Notebooks and Python – https://mybinder.org/ • HortonWorks Sandbox VM – Hadoop & Spark & Hive, Ambari • DataBricks Cloud Environment with Apache Spark (free trial) • KataKoda – tutorials & live environment for TensorFlow • Oracle Big Data Lite – Prebuilt Virtual Machine • Data Visualization Desktop – ready to run desktop tool • Tutorials, Courses (Udacity, Coursera, edX) • Books • Introducing Data Science • Learning Apache Spark 2 • Python Machine Learning
- 63. THE AMIS & CONCLUSION MACHINE LEARNING JOURNEY – STARTING TODAY • General introduction • Use case • Handson • Functional (non-programming) • Technical: R & Rstudio – Decision Trees • Deep dive sessions • 14th June: Random Forests, K-Means Clustering – with R • … • … • … (Python, TensorFlow, Neural Network, PCA, Linear Regression)
- 64. SUMMARY • IoT, Big Data, Machine Learning => AI • Recent and Rapid Democratization of Machine Learning • Algorithms, Storage and Compute Resources, High Level Machine Learning Frameworks, Education resources , Open Data, Trained ML Models, Out of the Box SaaS capabilities – powered by ML • Produce business value today • Machine Learning by computers helps us(ers) understand historic data and apply that insight to new data • Developers have to learn how to incorporate Machine Learning into their applications – for smarter Uis, more automation, faster (p)reactions
- 65. SUMMARY • R and Python are most popular technologies for data exploration and ML model discovery [on small subsets of Big Data] • Apache Spark (on Hadoop) is frequently used to powercrunch data (wrangling) and run ML models on Big Data sets • Notebooks are a popular vehicle in the Data Science lab • To explore and report • Oracle is quite active on Machine Learning • Power PaaS and SaaS with ML • Provide us with the Machine Learning Data Lab & Run Time (on the cloud) • Getting started on Machine Learning is fun, smart & well supported
- 66. HANDS ON • Alle materialen staan in: https://github.com/AMIS-Services Non Technical Technical Decision Trees
- 67. HANDS ON • Alle materialen staan in: https://github.com/AMIS-Services Non Technical
- 68. HANDS ON • Alle materialen staan in: https://github.com/AMIS-Services Non Technical Technical Decision Trees
- 69. • Blog: technology.amis.nl • Email: [email protected] • : lucasjellema • : lucas-jellema • : www.amis.nl, [email protected] +31 306016000 Edisonbaan 15, Nieuwegein
- 70. REFERENCES • AI Adventures (Google) https://www.youtube.com/watch?v=RJudqel8DVA • Twitch TV https://www.twitch.tv/videos/179940629 and sources on GitHub: https://github.com/sunilmallya/dl-twitch-series • Tensor Flow & Deep Learning without a PhD (Devoxx) https://www.youtube.com/watch?v=vq2nnJ4g6N0 • KataKoda Browser Based Runtime for TensorFlow https://www.katacoda.com/courses/tensorflow • And many more #DevoxxMA
Editor's Notes
- #2: Our technology has gotten smart and fast enough to make predictions and come up with recommendations in near real time. Machine Learning is the art of deriving models from our Big Data collections – harvesting historic patterns and trends – and applying those models to new data in order to rapidly and adequately respond to that data. This presentation will explain and demonstrate in simple, straightforward terms and using easy to understand practical examples what Machine Learning really is and how it can be useful in our world of applications, integrations and databases. Hadoop and Spark, real time and streaming analytics, Watson and Cloud Datalab, Jupyter Notebooks, Oracle Machine Learning CS and the Citizen Data Scientists will all make their appearance, as will SQL.
- #4: Why do we study history? To understand the present and predict the future (from current events)
- #12: IoT Social Media
- #13: IoT Social Media
- #26: Market Basket Analysis: https://www.linkedin.com/pulse/using-machine-learning-market-basket-analysis-thomsen
- #28: http://yann.lecun.com/exdb/mnist/ MNIST – handwritten images
- #29: https://aiexperiments.withgoogle.com/quick-draw
- #32: https://www.slideshare.net/databricks/apache-spark-model-deployment
- #35: https://www.slideshare.net/databricks/apache-spark-model-deployment
- #36: https://www.slideshare.net/databricks/apache-spark-model-deployment
- #44: https://www.slideshare.net/AshishBansal17/tensorflow-vs-mxnet
- #46: https://github.com/lucasjellema/theArtOfMachineLearning/blob/master/LinearRegression.ipynb https://github.com/lucasjellema/jupyter-notebook-eredivisie/blob/master/EredivisieResults_2016_2017.ipynb https://github.com/jadianes/spark-movie-lens/blob/master/notebooks/building-recommender.ipynb https://github.com/justmarkham/DAT4/blob/master/notebooks/08_linear_regression.ipynb
- #47: https://openflights.org/data.html - airports, airlines, flight routes Google Audio Set - https://research.google.com/audioset/ (A large-scale dataset of manually annotated audio events) Open Images Data Set - https://github.com/openimages/dataset , www.image-net.org http://api.football-data.org/index UK Data - https://data.gov.uk/ Open Data Sets - https://www.kaggle.com/datasets CBS Open Data - https://www.cbs.nl/nl-nl/onze-diensten/open-data Open Data Sets for Deep learning - https://deeplearning4j.org/opendata Data.gov The home of the US Government’s open data https://open-data.europa.eu/ The home of the European Commission’s open data https://www.wikidata.org (in part originated out of Freebase.org An open database that retrieves its information from sites like Wikipedia, MusicBrains, and the SEC archive ) Data.worldbank.org Open data initiative from the World Bank Aiddata.org Open data for international development Open.fda.gov Open data from the US Food and Drug Administration Google Knowledge Graph API - https://developers.google.com/knowledge-graph/ Detroit Open Data Portal https://data.detroitmi.gov/ Example: Detroit Police Crime statistics: https://data.detroitmi.gov/Public-Safety/-Archived-All-Crime-Incidents-2009-May-5-2017/b4hw-v6w2
- #48: https://openflights.org/data.html - airports, airlines, flight routes Google Audio Set - https://research.google.com/audioset/ (A large-scale dataset of manually annotated audio events) Open Images Data Set - https://github.com/openimages/dataset , www.image-net.org http://api.football-data.org/index http://files.grouplens.org/datasets/movielens/ml-latest-small-README.html UK Data - https://data.gov.uk/ Open Data Sets - https://www.kaggle.com/datasets CBS Open Data - https://www.cbs.nl/nl-nl/onze-diensten/open-data Open Data Sets for Deep learning - https://deeplearning4j.org/opendata Data.gov The home of the US Government’s open data https://open-data.europa.eu/ The home of the European Commission’s open data https://www.wikidata.org (in part originated out of Freebase.org An open database that retrieves its information from sites like Wikipedia, MusicBrains, and the SEC archive ) Data.worldbank.org Open data initiative from the World Bank Aiddata.org Open data for international development Open.fda.gov Open data from the US Food and Drug Administration Google Knowledge Graph API - https://developers.google.com/knowledge-graph/ Detroit Open Data Portal https://data.detroitmi.gov/ Example: Detroit Police Crime statistics: https://data.detroitmi.gov/Public-Safety/-Archived-All-Crime-Incidents-2009-May-5-2017/b4hw-v6w2
- #52: https://github.com/jadianes/spark-movie-lens/blob/master/notebooks/building-recommender.ipynb
- #53: https://www.oracle.com/big-data/big-data-discovery/index.html https://labs.oracle.com/pls/apex/f?p=labs:49:::::P49_PROJECT_ID:7 https://technology.amis.nl/2004/10/16/hidden-plsql-gem-in-10g-dbms_frequent_itemset-for-plsql-based-data-mining/ http://oracledmt.blogspot.nl/2006/05/sql-of-analytics-1-data-mining.html
- #62: https://www.oracle.com/big-data/big-data-discovery/index.html https://labs.oracle.com/pls/apex/f?p=labs:49:::::P49_PROJECT_ID:7
- #64: http://tmpnb.org http://www.oracle.com/technetwork/database/bigdata-appliance/oracle-bigdatalite-2104726.html https://www.udacity.com/course/intro-to-machine-learning--ud120 https://www.coursera.org/learn/machine-learning#%20 https://www.edx.org/course/machine-learning-columbiax-csmm-102x-0 https://technology.amis.nl/2017/05/06/the-hello-world-of-machine-learning-with-python-pandas-jupyter-doing-iris-classification-based-on-quintessential-set-of-flower-data/ https://github.com/rhiever/Data-Analysis-and-Machine-Learning-Projects/blob/master/example-data-science-notebook/Example%20Machine%20Learning%20Notebook.ipynb https://databricks.com/try-databricks https://hortonworks.com/products/sandbox/ http://www.oracle.com/technetwork/middleware/oracle-data-visualization/downloads/oracle-data-visualization-desktop-2938957.html