DevOps+Data: Working with Source Control
0 likes916 views
The document provides an overview of DevOps principles and the importance of source control in database management. It discusses the challenges associated with automating databases, the different models of source control (centralized, distributed, and hybrid), and emphasizes the necessity of integrating source control with database changes. Best practices for implementing database source control and the role of modern tools to facilitate this process are also highlighted.
![WHAT IS DEVOPS?
• “DevOps is the practice of operations and development
engineers participating together in the entire service
lifecycle, from design through the development process to
production support.”
• Corollary: “DevOps is also characterized by operations staff
making use [of] many of the same techniques as developers
for their systems work.”
Source: https://theagileadmin.com/what-is-devops/](https://image.slidesharecdn.com/devopsdata1-sourcecontrol-170321220740/85/DevOps-Data-Working-with-Source-Control-5-320.jpg)
DevOps+Data: Working with Source Control
- 1. DEVOPS+DATA: WORKING WITH SOURCE CONTROL Ed Leighton-Dick
- 2. ABOUT ME Email: [email protected] Blog: edleightondick.com Twitter: @eleightondick
- 3. GOALS • Understand DevOps • Understand source control • How to apply source control to data
- 4. DEVOPS
- 5. WHAT IS DEVOPS? • “DevOps is the practice of operations and development engineers participating together in the entire service lifecycle, from design through the development process to production support.” • Corollary: “DevOps is also characterized by operations staff making use [of] many of the same techniques as developers for their systems work.” Source: https://theagileadmin.com/what-is-devops/
- 6. WHAT IS DEVOPS? • Incorporates • Structured processes favored by operations • Flexibility favored by developers • Built around consistent processes and practices • Evolved from need to better support Agile • Often includes Continuous Integration/Continuous Delivery
- 7. WHAT’S THE BIG DEAL? • Improved communication and collaboration • Improved productivity and efficiency • More regular, more consistent software updates • Improved software quality • Reduced time to recover from failed releases • Lower risk
- 8. STAGES • Code • Build • Verify • Package • Release • Configure • Monitor Image Credit: By Kharnagy - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=51215412
- 9. DEVOPS+DATA • Databases are harder to automate • Shared resource • Code mixed in with data • Mix of static and dynamic data • Objects must be modified, not simply replaced • Databases are integral components of most applications
- 10. SOURCE CONTROL
- 11. SOURCE CONTROL • Practices and supporting systems that support the management of changes to application code • Stores history of changes • Facilitates merging of conflicting changes • AKA: version control, source management, revision control
- 12. TYPES OF SOURCE CONTROL • Centralized • PVCS • Microsoft Visual Source Safe • Microsoft Team Foundation Server • Subversion (SVN) • Distributed • Git • Mercurial
- 13. COMPARISON Centralized • One snapshot + deltas • Working copy = Most recent version • Work solely on central repository • One source of truth Distributed • Git: Snapshots • Working copy = Full copy of repository • Work primarily with local repository • ??
- 14. TRADITIONAL • Repository • Working folder • Check-out and check-in • Commits • Branching and merging
- 15. DISTRIBUTED • Repository • Local repository • Cloning • Pull • Push • Branching and merging
- 16. VCS+DATA
- 17. VCS+DATA • Databases should be included in source control • Database code = Application code • Static data = Application code • Most applications cannot run without their database(s)
- 18. VCS+DATA • Database professionals have been slow to adopt source control • Many DBAs have no development experience • Lack of communication about why • Lack of communication/inclusion with development teams • Migration vs deployment • Lack of pre-built tools to facilitate complex database changes
- 19. VCS+DATA: WHY NOW? • The situation has changed • Modern tools now exist to facilitate technical side • DevOps changes the conversation
- 20. VCS+DATA MODELS • Schema-only • Changes-only • Hybrid
- 21. THE SCHEMA-ONLY MODEL • Code for all database objects stored • Mirrors model used by most application code • Can produce a clean copy of a specific version of a database • Cannot migrate to a new version of a database without help • Used by… • Roll-your-own • Most third-party tools* - Microsoft, Redgate, ApexSQL, DBMaestro
- 22. THE CHANGES-ONLY MODEL • Scripts to upgrade between versions are stored • Scripts to create a clean copy may not be stored • Used by… • Roll-your-own (very common)
- 23. THE HYBRID MODEL • Stores scripts used to create objects • Stores scripts used to migrate between versions • Used by… • Redgate SQL Source Control (with migration scripts) • Redgate ReadyRoll • Others?
- 24. FIRST THINGS FIRST… • All changes must be saved to source control for source control to be useful • Database cannot enforce this – Unaware of source control • Source control cannot enforce this – Unaware of changes • External tools may be able to monitor but cannot yet enforce – No control over database or source control • Cannot move further in DevOps without good source control
- 25. FIRST THINGS FIRST… • Decide on your process before implementing tools • What objects are saved to source control? • What changes are saved? • Who saves changes? • How often? • Who/what will enforce process? How?
- 26. SHARED VS DEDICATED • Two models for working with development databases • Shared • Dedicated
- 27. SHARED DEVELOPMENT MODEL • Data kept in central location • More efficient from technical perspective • Less disk space • Less management overhead • Developers must be concerned with conflicts with each others’ work • Integration done at time of development • Refreshing can be difficult
- 28. DEDICATED DEVELOPMENT MODEL • Each developer has their own copy of the database • Developer is able to work more efficiently • No conflicts with other developers’ work • Copy of database is stored locally in many cases • Data is more easily refreshed • Integration occurs between development and testing • Concerns • Transporting and storage of large databases • Security of data • Management overhead
- 29. WHICH IS BETTER? • No right answer • Often dictated by circumstances • My preference: As close to dedicated as possible • Redgate SQL Clone can help facilitate refreshing, management • Solves some problems • Still requires connection to central image
- 30. RELEASING CHANGES • The problem: How do we generate the deltas that matter to us? • Source control deltas = Differences between stored versions (Not useful for migrations) • Databases require change scripts of some sort
- 31. TOOLS • Microsoft Visual Studio Database Projects • Feature of Visual Studio • Developers works with creation scripts, rather than directly with database • “Declarative database development” model • Schema Comparison used to generate change scripts
- 32. TOOLS • Redgate approaches • SQL Source Control • More “behind the scenes” than database projects • Integrated into SQL Server Management Studio • Uses SQL Compare to generate change scripts at time of release • Manually-created migration scripts override generated change scripts • ReadyRoll • Integrated into Visual Studio • Uses SQL Compare to generate migration scripts at time of commit • Edit generated migration scripts to override
- 33. DEMO
- 34. BEST PRACTICES • Make database source control part of your team’s standard development process • Don’t make it only the DBA’s responsibility • One object/change per file • Check-in changes on a regular basis • Find a solution that integrates with what your developers already use • Don’t allow code changes outside of the process • Store the database build number in the database
- 35. WRAP-UP
- 36. GOALS • Understand DevOps • Understand source control • How to apply source control to data
- 37. QUESTIONS? Email: [email protected] Blog: edleightondick.com Twitter: @eleightondick
- 38. THANK YOU FOR ATTENDING! ~ Please remember to fill out your comment cards ~