Building scalable application with sql server
Download as pptx, pdf0 likes779 views
Chris Adkin has 15 years of IT experience and 14 years of experience as a DBA working with various sectors. He has over 10 years of experience with SQL Server from version 2000. He provides his email and Twitter contact information. The document then discusses various topics related to database design and performance including OLTP vs OLAP characteristics, data modeling best practices, indexing strategies, query optimization techniques, and concurrency control methods.
![ If problems exist, long PAGELATCH_*
and PAGEIOLATCH waits will be
visible.
Common causes of latch contention
include:
◦ Heavy CRUD activity affecting rows
in the same page.
◦ Solution is to pad rows out with
char[X] columns.](https://image.slidesharecdn.com/buildingscalableapplicationwithsqlserver-120628172936-phpapp02/85/Building-scalable-application-with-sql-server-33-320.jpg)
Building scalable application with sql server
- 1. Chris Adkin
- 2. 15 years of IT experience 14 years of DBA experience across a variety of vertical sectors. Over 10 years experience of SQL Server from 2000. Contact ◦ Email: [email protected] ◦ Twitter: ChrisAdkin8
- 3. The following material contains general rules of thumb. Always test your assumptions. There will always be edge cases in which conventional wisdom may not apply.
- 4. Characteristic OLTP OLAP I/O Random Serial Transactions Short Longer during ETL Execution plans Simple Complex Plan nature Serial with index seeks Parallel with index/table /partition scans SQL Complexity Simpler Complex, often generated by BI tools Number of queries Finite Highly variable (re)compiles impact Critical Execution plan run on performance time more important SQL Selectivity Queries typically only Queries aggregate affect a few rows many rows and affect many rows during ETL
- 5. Where the foundation for scalability is laid. Consider your most performance critical queries and the number of joins they require.
- 6. Data model design “Usual suspects”: ◦ Over normalisation. ◦ 1:1 relationships, where most / all the attributes from both entities are retrieved together. ◦ Logical data model sprawl, as the model evolves over time.
- 7. 1:1 relationships edge cases: ◦ Sparse columns, which cannot be compressed or used in clustered indexes. ◦ Infrequently accessed columns with wide data types.
- 8. “Walking with keys” We only want columns from tables A and E and have to go via B, C and D in order to facilitate the join. The exception is when entities are introduced to resolve many to many relationships. A B C D E
- 9. Test drive your logical model design as early as possible in the project. Failure to do so may result in performance issues built into the foundations of the application.
- 10. “Chattiness” => heavy network conversation volume between software components. Chatter takes place via linked servers and /or between tiers. More time is spent on the network than anywhere else.
- 11. Low IIS and SQL Server CPU utilisation is a symptom. Cache static / relatively static data outside the database. Put CRUD intensive operations in stored procedures. Always put the processing closest to the resource that requires it.
- 12. There is a wider debate about where it is best to put the business logic. This is beyond the scope of this presentation, however you may wish to look at: ◦ To sp Or Not sp by Adam Machanic ◦ Pet vsPet: MS opens .Net benchmarking wars
- 13. Start with a minimal set of indexes and then add. Use filtered indexes for selective SQL. Indexing GUIDs: ◦ Causes page splits and fragmentation ◦ GUIDS are large, 16 bytes ◦ Inflates the size of secondary indexes created on clustered indexes.
- 14. „Scatter gun‟ indexing strategies can lead to indexes that add little or no value. Glenn Berry‟s diagnostics script includes a query to highlight indexes with lots of writes and little or no reads.
- 15. For GUIDs and clustered indexes, refer to “The Clustered Index Debate” by SQL Skills. If you really need GUIDs, consider: ◦ NEWSEQUENTIALID(). ◦ SSD technology as blogged about by Tony Rogerson. Otherwise avoid GUIDs in general.
- 16. Know your critical indexes and when to rebuild and re-organise them. The maintenance plan index rebuild tasks rebuilds ALL indexes. Use index fragmentation and index read intensity to determine rebuild / reorg order. Some good free tools are available, the procedures from Ola Hallengren to name one.
- 17. A compile takes place when a plan cannot be found for a statement in the plan cache. Recompiles take place when the execution engine deems that a plan is no longer valid. All are to be avoided where possible.
- 18. Factor Causes A Compile Causes A Recompile Use Of temp tables √ Changes in statistics √ DBCC FREEPROCCACHE / instance √ restart Schema changes √ Use of the RECOMPILE hint √ Non parameterised SQL √ Dynamic SQL √ Plans aged out the plan cache √ Changes in SET options √ (Refer to Erland Sommarskog‟s blog)
- 19. There are two hints that help to minimise recompiles. The KEEP PLAN hint causes the execution engine to treat temporary tables as permanent tables. The KEEPFIXED PLAN hint will prevent recompiles if there are changes to the statistics of base tables used in a query. Forced parameterisation can minimise compiles, but use it with caution.
- 20. There are many causes of RBAR. Cursors. User defined functions, table level functions are the exception. EXISTS sub queries. Recursive CTEs. . . . and many more, search on SQL Server Central for RBAR
- 21. Avoid RBAR wherever possible. Row by agonising row, as coined by Jeff Moden, is covered extensively in his articles on SQL Server Central.
- 22. Segregate different types of querying activity. I/O best practise is to segregate random from sequential I/O. Segregate OLAP/reporting queries from OLTP queries.
- 23. ETL aside, OLTP applications are more write intensive than OLAP ones. Optimisations for reads can decimate write performance: ◦ Heavy indexing ◦ Covering indexes ◦ Index views ◦ Column store indexes ◦ Aggregation
- 24. The Command Query Responsibility Segregation model provides a solution. Queries insert data into a database optimised for fast writes. The same data is queued (service broker). The data is de-queued and written to a database optimised for fast reads.
- 25. An online retailer services thousands of concurrent customers. A single table holds the balance for the retailer in a single row. If all transactions can update the balance table at the same time, this creates a unique point of serialisation.
- 26. Queue the debits credits from the user transactions, two options. Table with a clustered index, as per Remus Rasanu‟s blog and DELETE OUTPUT destructive read. Service broker is the 2nd option. De-queue values and update the table.
- 27. Prior to SQL Server 2005, unless dirty reads are used, readers block writers. This throttles back OTLP through put. The solution; multi-version concurrency control. Facilitated by two new isolation levels available in SQL Server 2005 onwards.
- 28. Read committed snapshot ◦ Data is consistent with beginning of the statement. ◦ An extension of the read committed isolation level. Snapshot isolation ◦ Data is consistent with the beginning of transaction.
- 29. Snapshot / RCSI isolation cannot be enabled for databases with filestream file groups. Queries may have to traverse the row version store => more logical and physical reads. This Microsoft paper provides guidance as to which isolation level to use.
- 30. 14 bytes of information is added to each row in the database as it is modified or created. Snapshot isolation can lead to update conflicts, retry logic is required to work around this. Ensure tempdb resides on fast disks, as this is where the version store lives.
- 31. Latches protect the integrity of internal SQL Server memory structures. Three latch wait types: ◦ Buffer (BUF) : PAGELATCH_* wait type ◦ Non buffer : LATCH_* wait type ◦ IO latch : PAGEIOLATCH wait type
- 32. When multiple threads try to obtain the same latch in incompatible modes, contention takes place. Some latch contention is to be expected. Refer to this SQLCAT paper for detailed information on latch trouble shooting.
- 33. If problems exist, long PAGELATCH_* and PAGEIOLATCH waits will be visible. Common causes of latch contention include: ◦ Heavy CRUD activity affecting rows in the same page. ◦ Solution is to pad rows out with char[X] columns.
- 34. An index has a leading column populated by an identity, sequence etc. Solution: hash partition the table on the indexes leading column.
- 35. Diagnosing and Resolving Latching Issues by SQL Customer Advisory Team Designing High Scale OLTP Systems by Thomas Kejser Lessons Learned From 128 Core OLTP Testing by Thomas Kejser
- 37. Twitter : ChrisAdkin8 Email : [email protected]