![Primitive Arrays
// Sequential Access = Predictable
for (int i=0; i<someArray.length; i++)
someArray[i]++;](https://image.slidesharecdn.com/performanceandpredictability-140430055652-phpapp02/85/Performance-and-predictability-49-320.jpg)
![Primitive Arrays - Skipping Elements
// Holes Hurt
for (int i=0; i<someArray.length; i += SKIP)
someArray[i]++;](https://image.slidesharecdn.com/performanceandpredictability-140430055652-phpapp02/85/Performance-and-predictability-50-320.jpg)
![Multidimensional Arrays
Multidimensional Arrays are really Arrays of
Arrays in Java. (Unlike C)
Some people realign their accesses:
for (int col=0; col<COLS; col++) {
for (int row=0; row<ROWS; row++) {
array[ROWS * col + row]++;
}
}](https://image.slidesharecdn.com/performanceandpredictability-140430055652-phpapp02/85/Performance-and-predictability-52-320.jpg)
![Bad Access Alignment
Strides the wrong way, bad
locality.
array[COLS * row + col]++;
Strides the right way, good
locality.
array[ROWS * col + row]++;](https://image.slidesharecdn.com/performanceandpredictability-140430055652-phpapp02/85/Performance-and-predictability-53-320.jpg)
Performance and predictability
- 3. Why care about low level rubbish? Branch Prediction Memory Access Storage Conclusions
- 5. Performance Discussion Product Solutions “Just use our library/tool/framework, and everything is web-scale!” Architecture Advocacy “Always design your software like this.” Methodology & Fundamentals “Here are some principles and knowledge, use your brain“
- 6. Do you care? DON'T look at access patterns first Many problems not Pattern Related Networking Database or External Service Minimising I/O Garbage Collection Insufficient Parallelism Harvest the low hanging fruit first
- 7. Low Hanging is a cost-benefit analysis
- 8. So when does it matter?
- 9. Informed Design & Architecture * * this is not a call for premature optimisation
- 11. That 10% that actually matters
- 12. Performance of Predictable Accesses
- 13. Why care about low level rubbish? Branch Prediction Memory Access Storage Conclusions
- 14. What 4 things do CPUs actually do?
- 15. Fetch, Decode, Execute, Writeback
- 16. Pipelining speeds this up
- 17. Pipelined
- 18. What about branches? public static int simple(int x, int y, int z) { int ret; if (x > 5) { ret = y + z; } else { ret = y; } return ret; }
- 19. Branches cause stalls, stalls kill performance
- 21. Can we eliminate branches?
- 22. Naive Compilation if (x > 5) { ret = z; } else { ret = y; } cmp x, 5 jmp L1 mov z, ret br L2 L1: mov y, ret L2: ...
- 23. Conditional Branches if (x > 5) { ret = z; } else { ret = y; } cmp x, 5 mov z, ret cmovle y, ret
- 24. Strategy: predict branches and speculatively execute
- 25. Static Prediction A forward branch defaults to not taken A backward branch defaults to taken
- 27. Static Hints (Pentium 4 or later) __emit 0x3E defaults to taken __emit 0x2E defaults to not taken don’t use them, flip the branch
- 28. Dynamic prediction: record history and predict future
- 29. Branch Target Buffer (BTB) a log of the history of each branch also stores the address its finite!
- 30. Local record per conditional branch histories Global record shared history of conditional jumps
- 31. Loop specialised predictor when there’s a loop (jumping in a cycle n times) Function specialised buffer for predicted nearby function returns Two level Adaptive Predictor accounts for up patterns of up to 3 if statements
- 32. Measurement and Monitoring Use Performance Event Counters (Model Specific Registers) Can be configured to store branch prediction information Profilers & Tooling: perf (linux), VTune, AMD Code Analyst, Visual Studio, Oracle Performance Studio
- 33. Approach: Loop Unrolling for (int x = 0; x < 100; x+=5) { foo(x); foo(x+1); foo(x+2); foo(x+3); foo(x+4); }
- 34. Approach: minimise branches in loops
- 35. Approach: Separation of Concerns Little branching on a latency critical path or thread Heavily branchy administrative code on a different path Helps adjust for the inflight branch prediction not tracking many targets Only applicable with low context switching rates Also easier to understand the code
- 36. One more thing ...
- 37. Division/Modulus index = (index + 1) % SIZE; vs index = index + 1; if (index == SIZE) { index = 0; }
- 38. / or % are 92 Cycles on Core i*
- 39. Summary CPUs are Super-pipelined and Superscalar Branches cause stalls Branches can be removed, minimised or simplified Frequently get easier to read code as well
- 40. Why care about low level rubbish? Branch Prediction Memory Access Storage Conclusions
- 41. The Problem Very Fast Relatively Slow
- 42. The Solution: CPU Cache Core Demands Data, looks at its cache If present (a "hit") then data returned to register If absent (a "miss") then data looked up from memory and stored in the cache Fast memory is expensive, a small amount is affordable
- 43. Multilevel Cache: Intel Sandybridge Shared Level 3 Cache Level 2 Cache Level 1 Data Cache Level 1 Instruction Cache Physical Core 0 HT: 2 Logical Cores .... Level 2 Cache Level 1 Data Cache Level 1 Instruction Cache Physical Core N HT: 2 Logical Cores
- 44. How bad is a miss? Location Latency in Clockcycles Register 0 L1 Cache 3 L2 Cache 9 L3 Cache 21 Main Memory 150-400
- 45. Cache Lines Data transferred in cache lines Fixed size block of memory Usually 64 bytes in current x86 CPUs Between 32 and 256 bytes Purely hardware consideration
- 46. Prefetch = Eagerly load data Adjacent Cache Line Prefetch Data Cache Unit (Streaming) Prefetch Problem: CPU Prediction isn't perfect Solution: Arrange Data so accesses are predictable Prefetching
- 47. Sequential Locality Referring to data that is arranged linearly in memory Spatial Locality Referring to data that is close together in memory Temporal Locality Repeatedly referring to same data in a short time span
- 48. General Principles Use smaller data types (-XX:+UseCompressedOops) Avoid 'big holes' in your data Make accesses as linear as possible
- 49. Primitive Arrays // Sequential Access = Predictable for (int i=0; i<someArray.length; i++) someArray[i]++;
- 50. Primitive Arrays - Skipping Elements // Holes Hurt for (int i=0; i<someArray.length; i += SKIP) someArray[i]++;
- 51. Primitive Arrays - Skipping Elements
- 52. Multidimensional Arrays Multidimensional Arrays are really Arrays of Arrays in Java. (Unlike C) Some people realign their accesses: for (int col=0; col<COLS; col++) { for (int row=0; row<ROWS; row++) { array[ROWS * col + row]++; } }
- 53. Bad Access Alignment Strides the wrong way, bad locality. array[COLS * row + col]++; Strides the right way, good locality. array[ROWS * col + row]++;
- 54. Full Random Access L1D - 5 clocks L2 - 37 clocks Memory - 280 clocks Sequential Access L1D - 5 clocks L2 - 14 clocks Memory - 28 clocks
- 55. Primitive Collections (GNU Trove, GS-Coll) Arrays > Linked Lists Hashtable > Search Tree Avoid Code bloating (Loop Unrolling) Custom Data Structures Judy Arrays an associative array/map kD-Trees generalised Binary Space Partitioning Z-Order Curve multidimensional data in one dimension Data Layout Principles
- 56. Data Locality vs Java Heap Layout 0 1 2 class Foo { Integer count; Bar bar; Baz baz; } // No alignment guarantees for (Foo foo : foos) { foo.count = 5; foo.bar.visit(); } 3 ... Foo bar baz count
- 57. Data Locality vs Java Heap Layout Serious Java Weakness Location of objects in memory hard to guarantee. GC also interferes Copying Compaction
- 58. Summary Cache misses cause stalls, which kill performance Measurable via Performance Event Counters Common Techniques for optimizing code
- 59. Why care about low level rubbish? Branch Prediction Memory Access Storage Conclusions
- 60. Hard Disks Commonly used persistent storage Spinning Rust, with a head to read/write Constant Angular Velocity - rotations per minute stays constant
- 61. A simple model Zone Constant Angular Velocity (ZCAV) / Zoned Bit Recording (ZBR) Operation Time = Time to process the command Time to seek Rotational speed latency Sequential Transfer TIme
- 62. ZBR implies faster transfer at limits than centre
- 63. Seeking vs Sequential reads Seek and Rotation times dominate on small values of data Random writes of 4kb can be 300 times slower than theoretical max data transfer Consider the impact of context switching between applications or threads
- 64. Fragmentation causes unnecessary seeks
- 65. Sector alignment is disk-level irrelevant
- 68. Summary Simple, sequential access patterns win Fragmentation is your enemy Alignment can be relevant in RAID scenarios SSDs are a different game
- 69. Why care about low level rubbish? Branch Prediction Memory Access Storage Conclusions
- 70. Software runs on Hardware, play nice
- 71. Predictability is a running theme
- 73. YMMV: measured incremental improvements >>> going nuts