Getting Started with BigQuery ML
Download as PPTX, PDF0 likes234 views
This document provides an overview of machine learning basics and building machine learning models in BigQuery. It introduces BigQuery as a serverless data warehouse that supports petabyte-scale data and enables machine learning through BigQuery ML. It describes the main machine learning problem categories of supervised and unsupervised learning. Finally, it demonstrates how to create, evaluate, and predict from machine learning models using SQL in BigQuery.
Getting Started with BigQuery ML
- 1. Getting Started with BigQuery ML Dan Sullivan GDG Great North DevFest 2020 October 17, 2020
- 2. OVERVIEW • INTRODUCTION TO BIGQUERY • MACHINE LEARNING BASICS • BUILDING MACHINE LEARNING MODELS IN BIGQUERY
- 3. Bio • Principle Engineer, PEAK6 Technologies • Author • Instructor • Udemy • Google Cloud • LinkedIn Learning • Data Science • Machine Learning • Databases & Data Modeling
- 5. What is BigQuery? • Serverless data warehouse • Petabyte scale • Uses SQL but is not a relational database • Analytical database • Other features • BigQuery ML • BigQuery BI Engine • BigQuery GIS
- 6. Datasets & Tables • Datasets • Collection of tables and views • Access control set at dataset and table level • Tables • Supports scalar and nested structures • Stored in columnar format • Partitioning • No indexes
- 7. Views • Projection of one or more tables • Tables can be joined • Views can be materialized
- 10. Command Line
- 11. Federated Data Access • Federated query • Cloud Storage • Parquet • ORC • Bigtable and Cloud SQL • Spreadsheets in Google Drive
- 13. Machine Learning Problem Categories • Unsupervised Learning • Supervised Learning • Reinforcement Learning*
- 14. Unsupervised Learning • Draw inference from data • Previously undetected patterns • Examples • Clustering • Anomaly detection • Principal component analysis
- 15. Supervised Learning • Learn from examples • Goal is to predict category or value • Examples • Classifying tumors from images • Predicting housing prices • Identifying fraudulent credit card transactions
- 16. 2 Approaches to MachineLearning • Symbolic Artificial Intelligence • Neural networks and deep learning
- 17. SymbolicArtificial Intelligence • Symbols represent entities and attributes • Manipulate symbols to make inferences • Variety of ways to manipulate symbols
- 18. SymbolicMachine Learning • Domain of interest is modeled using symbols • Set of features, e.g. • Length of stay in a hospital • Type of operation • Age at time of operation
- 19. Variety ofSymbolicML Algorithms • Decision trees • Random forests • Naïve Bayes • Support vector machines (SVMs) • K Nearest Neighbor
- 20. Deep Learning Builds onNeuron-likeAbstraction • Inputs are numbers • Weights assign importance to inputs • Sum weighted inputs becomes • Input to a non-linear function • Output is result of that function.
- 21. Activation Function • Non-linear function known as activation function • Examples • Sigmoid • TanH • ReLU
- 22. NeuralNetworks • Input layer • Hidden layer • Output
- 23. Deep Learning Networks • More than 3 layers • Challenging to learn weights • Backpropagation algorithm used to adjust weights
- 24. Steps to Creating and Using ML Models • Data engineering • Collection • Quality assessment and cleansing • Transform and format • Feature engineering • Training • Evaluating • Predicting
- 26. BigQuery ML
- 27. WhyDo ML inBigQuery? • Support for data engineering • No need to export data • Use SQL https://commons.wikimedia.org/wiki/File:Supervised_machine_learning_in_a_nutshell.svg
- 28. BigQueryML • Create machine learning models in SQL • Several kinds of models • Linear regression • Binary and multiclass logistic regressions • K-means clustering • Time series forecast • Matrix factorization • Boosted Tree and XGBoost • Tensorflow (imported) • AutoML Tables
- 30. CREATE MODEL Options • Model Type • Input Label Columns • Regularization • Learning rate • Early Stop • Standardize Features • Max Tree Depth • More …