Geek Sync | Data Integrity Demystified - Deborah Melkin | IDERA
Download as PPTX, PDF1 like44 views
The document is a presentation by Deborah Melkin discussing data integrity in databases, covering its definition, significance, and types such as entity, referential, and domain integrity. Key points include the structure, naming conventions, and constraints necessary for maintaining data integrity and performance, along with examples of primary and unique key constraints. The presentation concludes by acknowledging that while achieving 100% data integrity may not be possible, striving for it is essential for effective data management.
Geek Sync | Data Integrity Demystified - Deborah Melkin | IDERA
- 1. Data Integrity Demystified Deborah Melkin sheher March 10, 2021
- 2. About Me • 20 years as a DBA • Mainly work with SQL Server • Mainly work with OLTP but have worked with some data marts. • NESQL Board Member • SQL SaturdayUser Group Speaker • IDERA ACE Class of 2020 • Speaker Idol Winner 2019 • Microsoft MVP – Data Platform Random facts:
- 3. Agenda • Setting the Foundation • Designing for the Data • Data Integrity & Performance
- 6. “Data Integrity” …the maintenance of, and the assurance of, the accuracy and consistency of data over its entire life- cycle, and is a critical aspect to the design, implementation and usage of any system which stores, processes, or retrieves data.… https://en.wikipedia.org/wiki/Data_integrity
- 7. What Does It Mean??? ? ? ? ? ? ? ?
- 8. Why Does This Matter? • “Garbage In – Garbage Out” • Trust of data • Not independent of other data • Accessed by multiple sources • Solve problems before they happen
- 10. Data Storage & Data Engines • Relational Database • NoSQL Database • Data Lake • Excel • Text File • etc….
- 11. Naming Conventions • Capitalization and punctuation • Character limits • Singular or Plural table name • TLA (three letter aliases) or other prefixes • Schemas
- 13. Foundations Create Integrity By… • Defining the structure of the data • Understanding the type of data at the start • Looking at names to easily understand what the data represents by • Creating consistency and a structure to the definitions used in the database
- 14. Designing for the Data…
- 15. Different Types of Integrity • Entity Integrity – About the table (entity) • Referential Integrity – About the relationships between tablesentities • Domain Integrity – About the column (attribute)
- 16. Entity Integrity
- 17. Relational Database Design • Normal Form vs. Star Schema • Determines how to group the different pieces of data in a logical way • Provides additional meaning based on the column’s “role” on the table
- 18. Primary Key Constraint • Column or combination of columns that define the uniqueness of each record of the table • All columns must be NOT NULL • Only 1 primary key per table • Logical vs Physical
- 19. Tip: Identify Primary Key Column(s) Ask the following question: What does this table hold using 1 or 2 sentences?
- 20. Surrogate Primary Key https://www.merriam-webster.com/dictionary/surrogate
- 21. Do Surrogate PKs Create Integrity? They Don’t!
- 22. Do Surrogate PKs Create Integrity? “If you stand for nothing, you will fall for anything.” If your surrogate primary key stands for nothing, your data integrity will “fall.”
- 23. Unique Key Constraint • Column or combinations of columns that create a unique set in addition to the Primary Key. • Not as strict as Primary Key – allows for NULL columns. • Can have multiple unique combinations.
- 24. Unique Key Constraint (cont’d) • Difference between a Unique Key and a Unique Index: • Use of filtered indexes • Use of included columns
- 25. What does Entity Integrity do? • Defines the relationship between… • the columns on the table • the tables themselves • Creates a clear definition of what each record is about
- 26. What does Entity Integrity do? • Creates all of the alternate primary key combinations or the logical primary key in case of a surrogate primary key • Enforces additional business rules
- 28. Foreign Key Constraint • Creates relationship between two tables • Ensures that the value in the field exists in the referenced table • Can define actions that occur when the referenced column is changed or the row is removed
- 29. Foreign Key Example Need to add a “Discontinued” column
- 30. What does Referential Integrity do? • Ensures that data that is related to other tables is valid • Defines rules for handling changes with the referenced column to maintain the data integrity
- 31. Domain Integrity
- 32. Data Types • Sets the type of data we are dealing with • Creates constraints on the data goes inside the attribute • Can guarantee valid data
- 33. Data Types
- 34. NOT NULL • NULL is the absence of a value • Sets the requirement as to whether the attribute must have a value • ‘’ (empty string) <> NULL
- 35. NOT NULL
- 36. Default Constraint • Set the value for a column if the value for the column is not specified • Ignored if the value is specified as NULL • Applies to Inserts or Foreign Key with SET DEFAULT
- 37. Check Constraint • Creates restrictions on the data that goes in the column • Ensuring the data in the field matches a given criteria • Can be created on the column or table level • Table level means that multiple columns can be checked at once
- 38. What Does Domain Integrity Do? • Defines the type of data expected for each column • Ensures the data in the column matches what should be there • Sets whether the field is required to have a value
- 39. What Does Domain Integrity Do? • Establishes the common known value for the field when the record is first inserted • Ensures that only the values that we expect to find in the column (or table) are there • Eliminates a way for bad data to get in our tables
- 41. Clustered Index: Integrity or Performance Clustered index defines the order of how the data in the table is physically stored. Integrity Performanc e
- 42. Integrity and the Query Optimizer • NOT NULL • Primary Key Constraints • Unique Constraints • Foreign Key Constraints • Check Constraints
- 43. Data Types Performance • Database size bloat • Memory Grant • Implicit Conversions
- 44. Demos!
- 46. Is Data Integrity 100% Achievable? No! • But it is worthwhile to figure out what is needed to create true data integrity to better understand and work with your data.
- 48. So What Did It Stand for?
- 50. References (cont’d) • https://debthedba.wordpress.com/2018/10/31/how-i- really-feel-about-surrogate-primary-keys/ • https://www.scarydba.com/2015/09/09/yes-foreign- keys-help-performance/ • https://www.sqlservercentral.com/articles/trust-or- verify-%E2%80%93-how-fks-can-or-cannot-help- performance
- 51. Thank you for Coming! Email: [email protected] Twitter: @dgmelkin Blog: DebtheDBA.wordpress.com GitHub: https://tinyurl.com/y5dpvue2