Data science-toolchain
8 likes2,434 views
The document outlines a data science toolchain presented by Jie-Han Chen, covering various programming languages and software such as Python, R, and Java, along with their strengths in data analysis and visualization. It discusses data collection methods, algorithm modeling, and machine learning libraries like Scikit-learn and TensorFlow. Additionally, it highlights tools for data visualization and frameworks like Scrapy for data extraction from websites.
Data science-toolchain
- 1. Data Science Toolchain presented by Jie-Han Chen slide: https://goo.gl/1hXBGk
- 3. Python Open Source Community Package Web Service Good Readability Machine Learning
- 4. R Open Source Community Built-in Statistics Package Standalone computing & data analysis Slower than Python
- 5. High Performance Big Data Poor Visualization, Modeling Java
- 6. Matlab & Octave Powerful built-in math functions Simple Data Visualization tool Prototyping -5 0 5 10 -10 -10 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 -5 0 5 10
- 7. Jupyter Notebook Support 40+ programming language. eg: Python, R, Scala... Excellent for sharing your experiments Markdown, Latex example1 example2
- 10. Data Science Toolchains Data Collection Data Visualization Data Storage Algorithm & Modeling
- 12. Data Collection Using API: Facebook, Wikipedia Web Scraper
- 13. Web Scraper
- 14. Web Scraper HTTP request + HTML Parser
- 15. HTTP: python-requests Better than built-in urllib Sessions with Cookie Persistence Thread-safety
- 18. Web page
- 22. HTML parser
- 24. Scrapy An open source and collaborative framework for extracting the data you need from websites.In a fast, simple, yet extensible way.
- 25. Scrapy $ scrapy startproject tutorial
- 26. Scrapy
- 27. path: /scrapy/dmoz.py crawler name: dmoz Scrapy
- 28. Scrapy
- 29. $ scrapy crawl dmoz Scrapy
- 31. robots.txt
- 33. "I believe that visualization is one of the most powerful means of achieving personal goals." Harvey Mackay
- 36. Matplotlib
- 37. ggplot2
- 38. D3.js Data Visualization Project Interacive Web frontend example1 example2
- 40. Bokeh Python, R, Scala, Julia Interactive Jupyter Notebook
- 41. Tableau
- 47. Programming
- 48. Using GeoJSON with Leaflet , Configurable
- 49. Using GeoJSON with Leaflet
- 57. S3 1. Key-value 2. Permission 3. Data Visualization 4. Big Data (Spark)
- 59. Algorithm & Modeling python-numpy + python-pandas + scikit-learn libsvm spark-Mlib Weka Deep Learning
- 60. Numpy + Pandas + Scikit-learn
- 61. Numpy C
- 62. Numpy - data structure ndarray (n-dim array) ndim size shape dtype
- 68. Numpy - linalg
- 69. numpy Series, DataFrame : csv, json ... nan
- 70. Series -
- 71. Series -
- 72. Series -
- 73. Series -
- 75. Pandas - import
- 76. Pandas - import
- 77. Pandas - import
- 78. Pandas - import
- 79. Pandas - NaN
- 80. Pandas - NaN
- 81. Pandas - NaN
- 86. LIBSVM C Easy to use Support many programming languages Dataset
- 87. LIBSVM - install $ git clone
- 88. LIBSVM - install $ make
- 90. LIBSVM - data format label index , attribute value , attribute
- 91. LIBSVM - data format
- 92. LIBSVM - toy
- 94. MLlib
- 95. MLlib , Hadoop Java, Scala, R, Python
- 96. MLlib
- 97. MLlib , Hadoop Java, Scala, R, Python
- 98. Classification: logistic regression, naive Bayes,... Regression: generalized linear regression, survival regression,... Decision trees, random forests, and gradient-boosted trees Recommendation: alternating least squares (ALS) Clustering: K-means, Gaussian mixtures (GMMs),... Topic modeling: latent Dirichlet allocation (LDA) Frequent itemsets, association rules, and sequential pattern mining MLlib
- 99. Feature transformations: standardization, normalization, hashing,... ML Pipeline construction Model evaluation and hyper-parameter tuning ML persistence: saving and loading models and Pipelines MLlib
- 100. MLlib , Hadoop Java, Scala, R, Python
- 101. MLlib
- 103. Weka Java library Big Data Support GUI
- 105. Theano Base on Numpy Implemented by Cython Dynamic C code generation GPU & CUDA tensor, math expression A CPU and GPU Math Compiler in Python Theano tutorial: http://www.slideshare.net/SergiiGavrylov/theano-tutorial
- 106. Keras Theano, Tensorflow Support GPU prototype High-level neural networks library
- 112. Tool ?
- 114. Homework Github repo Data science Database, Social Network Analytics, ML library, Deep Learning Platform ... READM.md: Repo Demo Code email: [email protected] Google https://goo.gl/forms/PQPz8u2glyunQvfM2