Stream me to the Cloud (and back) with Confluent & MongoDB
Download as PPTX, PDF0 likes1,222 views
The document discusses current challenges faced by businesses in leveraging data and providing seamless customer experiences, particularly during a crisis that has increased demand for online services. It highlights the need for scalable cloud solutions and the importance of modern architectures like event-driven systems, supported by technologies from Confluent and MongoDB. Joint case studies illustrate how organizations in various sectors, including healthcare and finance, successfully implemented these technologies to improve operational efficiency and responsiveness.
![23
RDBMS vs. Flexible Data Model
Relational MongoDB
{ customer_id : 3,
first_name : ”Danilo",
last_name : ”Nobrega",
city : ”Stockholm",
phones: [
{
number : “46-64-223-9828”,
dnc : true,
type : “home”
},{
number : “46-173-555-
12144”,
type : “cell”
}]
}
Customer ID First Name Last Name City
0 Jonah Rosenboom Hamburg
1 Tim Schojohann München
2 Boris Bialek Zürich
3 Danilo Nobrega Stockholm
4 Ingo Marienfeld Düsseldorf
Phone Number Type DNC Customer ID
49-69-223-9828 home T 3
49-69-143-45986 home T 0
49-173-555-12144 cell (null) 3
49-69-777-1212 home T 1
49-175-698-1213 cell (null) 1
49-162-767-444 cell F 2](https://image.slidesharecdn.com/mongodbconfluentfelixgianlucav3-200423115935/85/Stream-me-to-the-Cloud-and-back-with-Confluent-MongoDB-23-320.jpg)
Stream me to the Cloud (and back) with Confluent & MongoDB
- 1. Stream me to the Cloud (and back) with Confluent & MongoDB Felix Reichenbach MongoDB Gianluca Natali Confluent
- 2. Agenda Current Challenges A Cloud Story Solution Presentation & Architecture Demo Joint use cases & Customer references
- 4. Business Challenges Provider needs: - Tap new revenue streams to stay competitive - Scale with the demand and keep TCO low - Continuously evolve and improve user/customer experience Customer wants: - Consume services anywhere and anytime - Omnichannel with seamless transition (Mobile <-> Browser) - Fast response times and Real Time Interactions (Notifications, Status Updates, Confirmations…)
- 5. Technical Challenges Made worse by this crisis: - Scaling for peak hours requires heavy overprovisioning - Purchasing Hardware may not be possible - Mobility of personnel is limited Evergreen problems: - Making use of data is very difficult - Data is spread across many applications / domains - Low latency access is almost impossible - Agile development methodologies cannot be applied
- 6. Common Initiatives ● Application Modernization ● Microservices Architectures ● Single View of XYZ ● Real-time Analytics ● Content Management ● IoT ● ... Across Verticals: Healthcare Financial Services Telco Technology Retail Manufacturing / Automotive
- 7. a retail story... AllGoodThings Inc.
- 8. Our Meetings are changing... By now probably everyone of us had experienced a crowded business video call from home. Let me introduce you to the Conference call Bingo! 😀
- 9. CEO CTO CFO ● Our online store had 2 main outages last week ● Surge in Customers complaints and escalation are a concern ● We are unable to add/change products as fast as we’d like ● 600% increase in overall traffic to website ● A tweet with a link to an hand sanitizer product in our store went viral ● Adding a product requires downtime that we can’t afford with current queue setup Customer Success Head of Eng. CMO ● Outages due to lack of resources in DC ● Our DBs is cracking under load. ● We limited concurrent users to stay online ● Need to triple our DC to support peak load ● Health category has skyrocketed to #1 ● “Home office” and “Fitness” are trending upwards ● People can’t visit our shops, all the business moved to our online store ● #1 Complaints: “Item was successfully ordered and paid but then received email that is unavailable”, ● #2 Complaint: “Can’t access the website, I am stuck in queue” ● Our Support team is working double shifts, we need to grow the team ASAP ● DC: We can’t triple our DC size, the ROI doesn’t work. We need to maintain our margins ● Support: Can we establish more efficient processes and automate concierge/triage via support app or Chat bot? ● I can’t approve your requests as they are AllTheGoodThings Inc.
- 10. Meeting Conclusion / Requirements ● Fix the scalability issue while limiting the investment on our DC. ● Leverage public Cloud services to reduce TCO. ● Explore the feasibility of an app to allow automatic resolution of simple customer queries ● Increase agility in adding products ● Provide real time stock information
- 11. a few days later... Solution Overview
- 12. Eng.CEO CTOCS CMOCFO ● Dynamic Scalability ● High Availability ● Increased Agility ● Avoid Lock In ● ...
- 15. ETL/Data Integration Messaging Batch Expensive Time Consuming Difficult to Scale No Persistence After Consumption No Replay Highly Scalable Durable Persistent Ordered Real-time
- 16. ETL/Data Integration Messaging Batch Expensive Time Consuming Difficult to Scale No Persistence After Consumption No Replay Highly Scalable Durable Persistent Ordered Real-timeHighly Scalable Durable Persistent Ordered Real-time Event Streaming
- 17. Today
- 18. Real-Time Inventory Real-Time Fraud Detection Real-Time Customer 360 Machine Learning Models Real-Time Data Transformation ... Contextual Event-Driven Applications Universal Event Pipeline Data Stores Logs 3rd Party Apps Custom Apps/Microservices TREAMSSTREAMS CONNECT CLIENTS With Event Streaming and Kafka
- 19. Agility and Scalability with MongoDB
- 21. Isolated data in many different applications / domains and structures
- 22. 22 Data which is accessed together, is stored together... Customer Product Contact Pricing Phone Phone Objects Tables isolated in several applications and domains Lead NameNameParts ARR Address Contact Roles SummaryCustomer Detail Specs How to build a flexible and scalable operational data store??
- 23. 23 RDBMS vs. Flexible Data Model Relational MongoDB { customer_id : 3, first_name : ”Danilo", last_name : ”Nobrega", city : ”Stockholm", phones: [ { number : “46-64-223-9828”, dnc : true, type : “home” },{ number : “46-173-555- 12144”, type : “cell” }] } Customer ID First Name Last Name City 0 Jonah Rosenboom Hamburg 1 Tim Schojohann München 2 Boris Bialek Zürich 3 Danilo Nobrega Stockholm 4 Ingo Marienfeld Düsseldorf Phone Number Type DNC Customer ID 49-69-223-9828 home T 3 49-69-143-45986 home T 0 49-173-555-12144 cell (null) 3 49-69-777-1212 home T 1 49-175-698-1213 cell (null) 1 49-162-767-444 cell F 2
- 24. 24 The document model is the perfect fit because it is... Easy: Work with data in a natural, intuitive way, fully transactional Flexible: Adapt and make changes quickly Fast: Get great performance with less code Versatile: Supports a wide variety of data models and queries
- 25. 25 Multiple data models, rich query functionality... Rich Queries Point | Range | Geospatial | Faceted Search | Aggregations | JOINs | Graph Traversals JSON Documents Tabular Key-Value Text GraphGeospatial
- 26. 26 Mobile App with MongoDB Realm Personalized Marketing Campaign Analytics Real-Time Online Data Store MongoDB Data Platform Real Time Data Sync SPA with MongoDB Stitch
- 27. Freedom of Choice: Run wherever you need Private Cloud Deploy and self manage on premises Deploy via Kubernetes with Kubernetes Operators Multi- Cloud / PaaS Build on top of fully managed services Leverage horizontal and vertical elasticity Avoid cloud provider lock-in Hybrid Cloud Deploy a homogeneous technology stack across on-prem and cloud. Reduce complexity and avoid lock-in
- 28. Joint Reference Architecture Reference Architecture: https://www.confluent.io/resources/confluent-platform-reference-architecture-mongodb/
- 29. DEMO
- 30. Joint References
- 31. Centene, Healthcare Provider Centene has the largest medicare and medicaid managed care providers Challenge Centene’s Core Challenge is growth caused by Mergers & Acquisitions This caused them to reevaluate their Enterprise Data Integration and Data Migration Strategies…They wanted better scalability, availability, faster ETL Solution Build centralized eventing framework for enterprise use across all Centene domains Employing CDC for ingest and leveraging microservices using Confluent, Kafka, and MongoDB
- 32. Financial Services - a large global bank Leading global financial services firm, a major provider of various investment banking, retail banking and financial services. Challenges Need for centralized incident and event management solution Modernize legacy applications Provide complete health dashboards across various systems Streamline payment processing, customer activity tracking and real-time data integration across the bank Solution The bank implemented a secure logging platform running Kafka as a service and MongoDB databases The environment has over 80 clusters and supports data volumes between 200TB and 1PB per day Data collection and normalization is at the edge and data is replicated back to two central data centers for global reporting and trend analysis
- 33. Modernized security information and event management Filter, transform aggregate APP SIEM Index Search Curated streams Forensic Archive MongoDB CDC Syslog Network traffic Firewall logs RDBMS Application logs Sensor Data HTTP proxy logs QRadar Arcsight Splunk Elastic
- 34. Front, rear and top view cameras Parking assistant Environment pointer Ultrasonic Sensors Parking assistant with front and rear camera plus environment indicator Crash Sensors Front protection adaptivity Side protection Tail impact protection Front Camera Audi Active lane assistant Speed limit indicator Adaptive light Infrared Camera Rearview assistance with Pedestrian recognition Front and Rear Radar Sensors ACC with stop and go function Side assist Real time data cluster
- 35. Real-Time Monitoring Logs Reference data (monitoring) CDC Syslog MQTT APP Filter, transform aggregate SIEM Search Aggregated event data / curated streams iOT iOT iOT Kafka Streams/ksqlDB
- 36. Modernize your architectures with Confluent and MongoDB A new generation of technologies is needed to consume and exploit today's real time, fast moving data sources. Run your business in real-time, building real-time applications with historical context. Together, Confluent and MongoDB enable you to ● Build sophisticated data-driven and event-driven applications ● Modernize your application architecture ● Uncover new sources of data ● Derive insights for a competitive advantage & Accelerate your Journey to the Cloud and back! 😉
- 37. Driving innovation and remaining operational in uncertain times Koen Rousseau, Digital Lead, Sanoma Learning Online Education Tomorrow 10:30am CET Tim Nutman, Manager Solutions Architecture, MongoDB mongodb.com/webinars