2024-10-04 - Grace Hopper Celebration Open Source Day - Stefan
0 likes112 views
Milvus is an open-source vector database designed to efficiently manage and search deep learning embeddings, with a focus on scalability and performance. The document outlines a schedule for a workshop detailing Milvus features, contributions, and use cases, emphasizing its advantages over traditional databases. It invites participation in contributing to projects, along with resources for getting started and future discussions on generative AI.
![Contributing Your First Issue
● A few first-time-contributor-suitable issues:
○ [Feature Request]: [Milvus]Collection load
○ [Feature Request]: [Milvus]Shards Number
[Feature Request]: [Milvus]Timeout
[Feature Request]: [Milvus]Partition
● Help add integrations of Milvus and new RAG/agent frameworks:
○ dynamiq-ai/dynamiq
○ CrewAI
○ EmbedAnything
○ Firebase GenKit
○ Cognee
https://zilliz.com/blog/contributing-to-open-source-milvus-beginners-guide](https://image.slidesharecdn.com/2024-10-04-gracehoppercelebrationopensourceday-stefan-241004175230-91c6e392/85/2024-10-04-Grace-Hopper-Celebration-Open-Source-Day-Stefan-30-320.jpg)
2024-10-04 - Grace Hopper Celebration Open Source Day - Stefan
- 2. Milvus: An Open-Source Vector Database Build cool stuff! Learn about Deep Learning embeddings Create a Vector Database
- 3. Welcome!
- 4. Stefan Webb Developer Advocate, Zilliz [email protected] https://www.linkedin.com/in/stefan-webb https://x.com/stefan_webb
- 5. Tim Spann Principal Developer Advocate, Zilliz [email protected] https://www.linkedin.com/in/timothyspann https://x.com/PaaSDev Jiang Chen Head of Ecosystems and Developer Relations, Zilliz [email protected] https://www.linkedin.com/in/jiangc1010 https://x.com/jiangc1010
- 6. Today’s Schedule Morning Session (PT) 9.45 – 10.15 Intro to Milvus and Vector Databases 10.15 – 10.35 Getting Started with Milvus 10.35 – 11.45 Contributing to Milvus; or, Milvus Workshop 13.15 – 13.45 Intro to Milvus and Vector Databases 13.45 – 14.05 Getting Started with Milvus 14.00 – 15.15 Contributing to Milvus; or, Milvus Workshop Afternoon Session (PT)
- 7. Searching the Web with Gen AI
- 9. VS. Apple VS. Rising dough VS. Change car tire Rising Dough Proofing Bread ✔ ❌ Why is Semantic Search Difficult?
- 10. Why is Semantic Search Important? 20% Other newly generated data in 2025 will be unstructured data 80% Data Source: The Digitization of the World by IDC
- 11. Solution: Deep Learning Similarity Search
- 12. New Challenge: Search in Vector Spaces How to Index and Search? ● High-dimensional ● > 1000 dims How to Scale? ● 10-100 million vectors? ● Billions? ● Trillions? ● Billions of users? Multiple Data Types? ● Text ● Images ● Audio ● Graphs ● …
- 13. | © Copyright 2024 Zilliz 13 Milvus is an Open-Source Vector Database to store, index, manage, and use the massive number of embedding vectors generated by deep neural networks and LLMs. contributors 400 stars 30K docker pulls 66M forks 2.7K + Milvus: The most widely-adopted vector database
- 14. Milvus Users
- 15. Integrations Framework Hardware Infrastructure Embedding Models LLMs Software Infrastructure Vector Database
- 16. Why Open-Source? Cost-effective Innovation Community
- 17. Why Not Traditional Databases? Suboptimal Indexing / Search Scaling Inadequate Query & Analytics Support
- 18. All About Vector Databases and Milvus
- 20. How Does Similarity Search Work?
- 21. Deployment Options Milvus Lite ● Locally hosted ● Suitable for prototyping and demos Milvus Standalone ● Single remote/local server ● “Medium” scale ● Simplified setup, maintenance, etc. compared to cluster Milvus Cluster ● Distributed system ● Many different types of nodes ● Scales to 100s of billions of vectors
- 23. Benchmarks Shows 3-20x faster comparing with open source Milvus At least 6x faster than other vector databases https://github.com/zilliztech/VectorDBBench
- 24. Demo Time: What Can you Build with Milvus?
- 25. #1 - Multimodal Image Search ● https://multimodal-demo.milvus.io/
- 26. #2 - Chat with Open Source Software ● https://osschat.io/
- 27. Getting Started with Milvus and Workshop
- 28. Getting Started ● Google Colab notebook
- 29. Choose Your Own Adventure Contribute your first issue Challenge: Mini-projects Workshop: Semantic Search with Milvus Milvus Bootcamp Tutorials
- 30. Contributing Your First Issue ● A few first-time-contributor-suitable issues: ○ [Feature Request]: [Milvus]Collection load ○ [Feature Request]: [Milvus]Shards Number [Feature Request]: [Milvus]Timeout [Feature Request]: [Milvus]Partition ● Help add integrations of Milvus and new RAG/agent frameworks: ○ dynamiq-ai/dynamiq ○ CrewAI ○ EmbedAnything ○ Firebase GenKit ○ Cognee https://zilliz.com/blog/contributing-to-open-source-milvus-beginners-guide
- 31. Contributing Your First Issue You can also contribute to Milvus directly: • Issues · milvus-io/milvus · GitHub by following the contribution instructions, and see • Contributing to Open Source Milvus: A Beginner’s Guide
- 32. Milvus Mini-Projects Call for contribution of projects and demos built with Milvus. Get inspired by examples here. We can feature them on Discord. Suggestions: • RAG with Contextual Retrieval • Introducing Contextual Retrieval Anthropic • RAG for an application that requires structured output • GitHub - dottxt-ai/outlines: Structured Text Generation • Structured Outputs - OpenAI API • RAG on Wikipedia • wikimedia/wikipedia · Datasets at Hugging Face
- 34. Workshop: Building Semantic Search with Milvus ● Google Colab notebook
- 35. Generative AI Resource Hub • Generative AI Resource Hub | Zilliz
- 37. 10/8/2024 San Francisco 10/15/2024 Silicon Valley 10/23/2024 New York Held twice a month
- 38. We’re Hiring! ● Engineering Manager, Database Systems ● Sr SW Engineer, Distributed Systems ● SRE, Cloud Platform ● Staff SW Engineer, Cloud Platform ● Staff SW Engineer, Database Systems https://zilliz.com/careers#open-positions