Data-Drive DevOps: Mining Machine Data for "Metrics that Matter"
5 likes4,514 views
The document discusses the importance of using metrics in DevOps to enhance performance and drive continuous improvement by leveraging machine data. It outlines various categories of metrics that matter, including culture, process, quality, activity, and impact, emphasizing that each organization's journey will be unique. The goal is to use machine data to optimize development, testing, and business impact, while ensuring that teams can identify waste and improve overall efficiency.
Data-Drive DevOps: Mining Machine Data for "Metrics that Matter"
- 1. Andi Mann @AndiMann @Splunk Copyright © 2016 Splunk Inc. Data-Driven DevOps Mining Machine Data for ‘Metrics that Matter’ Andi Mann @AndiMann @Splunk
- 2. Andi Mann @AndiMann @Splunk CAMS – as close to prescriptive as DevOps gets
- 3. Andi Mann @AndiMann @Splunk3 BUT WHAT SHOULD YOU MEASURE?
- 4. Andi Mann @AndiMann @Splunk I’m working super hard!! That’s my stapler.
- 5. Andi Mann @AndiMann @Splunk5 Yeah, but … … what are you achieving? I’m gonna need you to come in Sunday.
- 6. Andi Mann @AndiMann @Splunk Gartner’s DevOps ‘Metrics that Matter’ Gartner Inc., Data-Driven DevOps: Use Metrics to Help Guide Your Journey, 29 May 2014 G00264319, Analyst(s): Cameron Haight | Tapati Bandopadhyay
- 7. Andi Mann @AndiMann @Splunk IDC’s DevOps ‘Metrics that Matter’
- 8. Andi Mann @AndiMann @Splunk Some DevOps Metrics that Might Matter Culture e.g. • Retention • Satisfaction • Callouts Process e.g. • Idea-to-cash • MTTR • Deliver time Quality e.g. • Tests passed • Tests failed • Best/worst Systems e.g. • Throughput • Uptime • Build times Activity e.g. • Commits • Tests run • Releases Impact e.g. • Signups • Checkouts • Revenue
- 9. Andi Mann @AndiMann @Splunk But DevOps Is Always a Unique Journey What Are Your ‘Metrics That Matter’?
- 10. Andi Mann @AndiMann @Splunk Every tool, every process, every component, on-prem or off One Constant: Machine Data
- 11. Andi Mann @AndiMann @Splunk Common Data Fabric 11 API SDKs UI Other Tools Escalation/ Collaboration Visibility Across the Whole Dev Lifecycle Plan Code Build Test/QA Stage Release Config Monitor
- 12. Andi Mann @AndiMann @Splunk Common Data Fabric 12 API SDKs UI Server, Storage. N/W Server Virtualisation Operating Systems Mobile Applications Cloud Services Other Tools Ticketing/Help Desk Custom Applications Visibility Across the Whole Ops Environment API Services Infrastructure Applications
- 13. Andi Mann @AndiMann @Splunk Use Machine Data To Identify ‘Waste’ Plan Develop (UI) Develop (Db) Develop (M’ware) Develop (Backend) Security Test Monitor Build (Prod) Architect Secure/ Comply DeployAccept Unit Test Document Cap Plan Train Feedback Integration Test Configure System Test Launch CAB Develop (APIs) Budget Build (Dev) Mgmt/ Tooling W W W W W W W W W 16 40 52 35 96 40 48 24 --8 2 5 6 8 2 12
- 14. Andi Mann @AndiMann @Splunk Use Machine Data To Manage Testing and QA • Release when ready, not a date! • Best / worst developers • Best / worst providers • Impact of new code on ops • Impact of new code on biz
- 15. Andi Mann @AndiMann @Splunk Use Machine Data To Enable Continuous Improvement Defect Information Capacity Planning Quality Standards Enhancement Requests Integration Requirements Acceptance Metrics Service Levels and KPIs Application Development Test and Acceptance Production BuildCodePlan Test/QA Stage Release Config Monitor Infrastructure Dependencies
- 16. Andi Mann @AndiMann @Splunk Use Machine Data To Accelerate Velocity Pivot & improve with Continuous Insights Product Managers identify new opportunities Continuously delivered to market … and Auditors are “happy”
- 17. Andi Mann @AndiMann @Splunk Use Machine Data To Improve Quality Code quality scans Static security scans White BoxDeveloper checks in code Automated Acceptance Tests Dynamic Security Scans Black Box “Chaos Monkey” tests Test Fail: Return Test Fail: Return X X Production QA Prod Pattern QA Pattern Library Test Pass: Promote Test Pass: Promote to Production Pattern library used for test and QA
- 18. Andi Mann @AndiMann @Splunk Use Machine Data To Align Business Impact
- 19. Andi Mann @AndiMann @Splunk Metrics that Matter Drive Better Feedback Loops Improve Application Velocity Improve Application Quality Improve Application Impact
- 20. Andi Mann @AndiMann @Splunk Copyright © 2016 Splunk Inc. Andi Mann @AndiMann We make machine data accessible, usable, and valuable to everyone