ArangoML Pipeline Cloud - Managed Machine Learning Metadata
0 likes1,144 views
The document introduces ArangoML Pipeline Cloud, a managed solution for machine learning metadata, emphasizing the importance of data in the ML pipeline. It discusses features like a multi-model database, metadata management, performance auditing, and reproducible model building for data scientists. It also provides links for getting started and offers a demo of the capabilities of the platform.
![https://github.com/arangoml/arangopipe
● Python package
● HTTP API
● TFX Integration [coming shortly]](https://image.slidesharecdn.com/arangomlpipelinecloudwebinar-200214133050/85/ArangoML-Pipeline-Cloud-Managed-Machine-Learning-Metadata-28-320.jpg)
ArangoML Pipeline Cloud - Managed Machine Learning Metadata
- 1. + + Copyright © ArangoDB Inc., 2019 - Confidential ArangoML Pipeline Cloud From Data to Managed Metadata
- 2. TL;DR 2 1. Different ways data is important for your machine learning pipeline 2. ArangoML Pipeline Cloud: The managed solution for your ML Metadata
- 3. Jörg Schad, PhD Head of Engineering and ML @ArangoDB ● Suki.ai ● Mesosphere ● Architect @SAP Hana ● PhD Distributed DB Systems ● Twitter: @joerg_schad
- 4. Chris Woodward Developer Relations Engineer @ArangoDB ● Training ● Development ● Community ● Twitter: @cw00dw0rd ● Slack: Chris.ArangoDB
- 7. Copyright©ArangoDBInc.,2019- Confidential 7 Get Data Write intelligent machine learning code Train Model Run Model Repeat What Data Scientists should be doing…
- 8. Copyright©ArangoDBInc.,2019- Confidential 8 Sculley, D., Holt, G., Golovin, D. et al. Hidden Technical Debt in Machine Learning Systems What Data Scientist are doing…
- 11. ● Native Multi Model Database ○ Stores, K/V, Documents & Graphs ● Distributed ○ Graphs can span multiple nodes ● AQL - SQL-like multi-model query language ● ACID Transactions including Multi Collection Transactions
- 12. Databases I
- 13. Databases II
- 15. Why Graph?
- 18. Databases III
- 20. Challenges ● Understand complete provenance of Model a. Understand Provenance b. Complete version history c. Audit ● Find all Models in production derived from dataset x ● Compare performance of different model performance ● Identify reusable steps ● Is my serving data distribution the same as for training data ● ...
- 21. From Data to Metadata…. Common Metadata
- 22. Metadata? https://www.kubeflow.org/docs/components/misc/metadata/ ML Project Dataset Feature Transform Experiment Train Performance Test Performance Model Function Model Serving Performance Notebook
- 23. ArangoML Pipeline “A common extensible metadata layer for ML pipelines which allows Data Scientists and DataOps to manage all information related to their ML pipelines in one place.” https://www.arangodb.com/2019/09/arangoml-pipeline-common-metadata-layer-machine-learning-pipelines/
- 25. Multi-Model Metadata FOR f in featuresets FILTER f.name == 'my_feature' FOR entity IN 1..3 ANY f featureset_dataset RETURN entity
- 26. Find relevant entities for given model Audit Resource accounting Explore Performance differences Trace Data Lineage (e.g., GDPR) Permission tracking Search/reuse existing entities Reproducible Model Building …. ... Detect Data Shift …. Data Scientist DataOps Administrator
- 27. ML Project Dataset Feature Transform Experiment Train Performance Test Performance Model Function ArangoML “Schema” Model Serving Performance Notebook
- 28. https://github.com/arangoml/arangopipe ● Python package ● HTTP API ● TFX Integration [coming shortly]
- 33. How to get started? 33 docker run -p 6529:8529 -p 8888:8888 -p 3000:3000 -it arangopipe/ap_tensor_flow But what about production?
- 34. How to get started? 34 docker run -p 6529:8529 -p 8888:8888 -p 3000:3000 -it arangopipe/ap_tensor_flow But what about production?
- 35. ArangoML Pipeline Cloud 35 1. Fully managed Cloud Solution 2. SLAs a. Temporary cloud instance with no setup b. Production instance
- 37. Thanks for listening! 37 https://www.arangodb.com/• https://github.com/arangoml/arangopipe • Blogpost • Getting Started Notebook Test-drive ArangoDB and ArangoML using Oasis 14-days for free