Role of Analytics in Digital Business
Download as PPTX, PDF0 likes877 views
The document discusses leveraging analytics for digital business transformation, emphasizing the importance of real-time and incremental analytics over traditional batch methods. It highlights various use cases, such as predictive maintenance and sales analytics, along with case studies in machine learning and anomaly detection. The document also covers the role of APIs and data collection methods in optimizing customer experiences and driving insights.
Role of Analytics in Digital Business
- 1. Driving Insights for Your Digital Business With Analytics Srinath Perera (@srinath_perera) VP – Research, WSO2 Member, Apache Foundation
- 3. Big Data Washing • Collect anything that is easy to get • Aggregate and Group • Find a complex but pretty chart • Predict something, but does not measure it’s quality • Claim you just got started!
- 4. Uber • A company worth XX • A taxi company that does not have cars or drivers
- 5. Picture by Dan Ruscoe (CC) https://www.flickr.com/photos/druscoe/8031488298 Game Changers
- 7. New Digital inspired Products and Revenue Streams • New way to do business (e.g. Uber, Amazon Go) • Product as a Service (e.g. IoT Jack hammer, Light as a service) • Progressive Insurance Gadget • Sell insights ( Telcos knows where people are, credit card companies know what people buy and their demographics, navigation apps know traffic)
- 8. Get Close to your Customers • Use analytics to optimize the experience • Predict issues and proactively handle them ( e.g. reschedule automatically when flight has missed) • Predict churn and act • Track the brand and manage it • Target your marketing
- 9. Optimizations • Reduce Fraud • Logistics, day to day operations • Analytics for hiring and Performance appraisal • Predictive maintenance • Sales analytics, demand prediction • Security and surveillance
- 10. Making this real
- 11. Conceptual Architecture • APIs play a key role in data collection • Need to respond to events as fast as possible • Incremental Analysis is key
- 13. Only DAS, which has everything Focus is on CEP ( siddhi), our core differentiator and 80% streaming and 20% batch use cases Integrating with Apache Spark as oppose to bundling it in
- 14. Data Collection Points 1. APIs 2. Instrumentations built into products being used (e.g. SNMP, JMX) 3. Sensors and custom instrumentations 4. Log analysis 5. Social networks and other feeds
- 15. Data Collection API One Sensor API to publish events - REST, Thrift, Java, JMS, Kafka - Java clients, java script clients* First you define streams (think it as a infinite table in SQL DB) Then publish events via Sensor API
- 16. “Publish once, analyze anyway you like”
- 17. KPIs and their Role • KPIs (Key Performance Indicators) are numbers that can give you an idea about performance of something – E.g. Countries have them ( GDP, Per Capita Income, HDI index etc) • Examples – Company Revenue – Lifetime value of a customer – Revenue per Square foot ( in retail industry) • Idea is to define them and monitor them. But defining them is hard work!! • Often one indicator tells half the story, and you need several that cover different angles
- 18. insert overwrite table BusSpeed select hour, average(v) as avgV, busID from BusStream group by busID, getHour(ts); Batch Analytics 1. For simple analytics, you can write Spark SQL (SQL-like) 2. They operates on top of data streams we published 3. Run as MapReduce jobs in Apache Spark
- 19. Picture by Dan Ruscoe (CC) https://www.flickr.com/photos/druscoe/8031488298 Lets go Beyond Batch
- 20. Incremental Analytics • Most “Digital business” use cases are incremental ( data keeps coming, and results should be updated) • Can do just with batch, but slow and lot of work • DAS includes set of incremental operators, works just with streaming in most cases. • Incremental ML is not included yet
- 21. Real-time: Value of some Insights degrade Fast! 1. Stock Markets 2. Fraud 3. Surveillance 4. Patient Monitoring 5. Traffic
- 22. Real Time Analytics with CEP
- 23. Case Study: People Tracking via BLE • Traffic Monitoring • Smart retail • Airport management Track people through • BLE via triangulation • Higher level logic via CEP
- 24. Case Study: Realtime Soccer Analysis Videohttps://www.youtube.com/watch? v=nRI6buQ0NOM
- 25. Machine learning • Given examples build a program that matches those examples • We call that program a “model” • Major improvements in last few years (e.g. deeplearning) Can you “Write a program to drive a Car?” Predictive Analytics
- 26. Machine Learner Wizard is No More • Machine learner provided wizard to build machine learning models • Technology is changing too fast to keep building such a Wizard • We are dropping that and instead support models built with other machine learning tools
- 27. Using ML Models • We support models built with following tools • PMML • Spark • We recommend PySpark as default ( works with DAS) • Models can be used them with both WSO2 CEP and ESB • Tensorflow, H20 models are coming ( can do already by writing an extension)
- 28. Case Study: Predict Wait Time in the Airport • Predicting the time to go through airport using location data • Real-time updates and events to passengers via the App
- 29. Anomaly Detection • Find the Odd one out • Anomalies by value though “Clustering” • Anomalies through time using Markov Chains • Detect Problems are drill in to find details • Available as a solution White paper: Fraud Detection and Prevention: A Data Analytics ApproachImage "Reading" by Creative Stall (cc), Noun Project
- 30. Communicate
- 31. What is a Dashboard? • Think a car dashboard • It give you idea about overall system in a glance • It is boring when all is good, and grab attention when something is wrong • Support for drill down and find root cause
- 32. • Starts with data in tabular format • Map each column to dimension in your plot like X,Y, color, point size, etc • Create a chart with few clicks Powered by VizGrammer lib that uses Vaga undneath (see https://github.com/wso2/Vi zGrammar) Gadget Generation Wizard
- 33. • When data cross security domains, there are security and management concerns • APIs ( e.g. WSO2 APIM) solve these problems Often data are accessed through the network – Mobile Apps – Query interfaces – Data integration – As a Subscription Expose data through API
- 34. Alerts • Done through CEP queries • Notifications ( sent via email, SMS, Pager etc.) • Goal is to give you peace of mind ( not having to check all the time) • They should be specific • They should be infrequent • They should have very low false positives • Let users control sensitivity
- 35. Take the time to Understand!!
- 36. Solutions
- 37. Photo by Tim Evanson (CC) https://www.flickr.com/photos/timevanson/ 6830726558