Algorithms summary (English version)
2 likes762 views
This document provides an overview of common algorithms and data structures. It begins by defining an algorithm and discussing algorithm analysis techniques like asymptotic analysis. It then covers basic data structures like arrays, linked lists, stacks, queues, and trees. It explains various sorting algorithms like selection sort, bubble sort, insertion sort, quicksort, and mergesort. It also discusses search techniques like binary search and binary search trees. Other topics covered include priority queues, hashing, graphs, graph traversal, minimum spanning trees, shortest paths, and string searching algorithms.
![BINARY SEARCH
• Rules
• your key: k
• you know value v at i: array[i] = v
• if k=v you found it
• if k>v the left part of array[i] can be ignored
• if k<v the right part can be ignored
• Algorithm
• 1. compare key and the middle of the list
• 2.
• if matched return the index of the middle
• if not matched remove the left or right from the middle in
accordance with the rule above rules
• 3. goto 1 while the list is remained](https://image.slidesharecdn.com/algorithmssummary-140611015001-phpapp02/85/Algorithms-summary-English-version-19-320.jpg)
![GRAPH TRAVERSAL
• Depth-first vs. Breadth-first search
• stack vs. queue
A
B
C
D
E
F
G
H
I
DFS
Stack
[A] [AB] [ABC] [AB] [ABD] [ABDE] [ABDEF] [ABDE] [ABDEG] [ABDEGH] [ABDEG] [ABDGI]
Traversal: A, B, C, D, E, F, G, H, I
BFS
Queue
Traversal: A, B, C, D, E, G, F, H, I
[A] [B] [CD] [D] [EGH] [GHF] [HFI]
A
B
C
D
E
F
G
H
I
A
B
C
D
E
F
G
H I
DFS tree
BFS tree](https://image.slidesharecdn.com/algorithmssummary-140611015001-phpapp02/85/Algorithms-summary-English-version-32-320.jpg)
![STRING SEARCH
RABIN-KARP METHOD
• Rabin-Karp
• T: text with size of m characters
• p: pattern with n characters
• h = hash(p)
• hi: hash value of n-character substring of T starting at T[i]
• if h = hi, string “T[i]…T[i+n-1]” can be p
• i=0 to m-n
• Hash
hi =
n 1X
j=0
T[i + j] · 2n 1 j
hi+1 = hi T[i] · 2n 1
+ T[i + n]
abcdefghijklmnopqrstuvwxyz
hash value = h1
hash value = h2
h2 = h1 b · 210
+ m](https://image.slidesharecdn.com/algorithmssummary-140611015001-phpapp02/85/Algorithms-summary-English-version-35-320.jpg)
Algorithms summary (English version)
- 1. ALGORITHMS - BASICS Young-Min Kang DEPT. GAME ENG., TONGMYONG UNIV., BUSAN, KOREA.
- 2. ALGORITHM • “step-by-step” procedure for calculation • input/output • simple and clear • finite • executable
- 3. ALGORITHM ANALYSIS • Asymptotic analysis • analysing the limiting behaviours of functions ! • the terms and are insignificant when n is very large • f(n) is asymptotically equivalent to • big-Oh, big-Omega, big-Theta f(n) = n3 + 20n2 + 10n 20n2 10n n3 f(n) = O(g(n) : f is bounded above by g f(n) = ⌦(g(n)) : f is bounded below by g f(n) = ⇥(g(n)) : f is bounded both above and below by g upper bound lower bound
- 4. BASIC DATA STRUCTURES • array vs. linked list • random access (time) • array • memory efficiency (space) • linked list
- 5. LINKED LIST STRUCTURES • singly linked list • 1 node 1 link (next) • doubly linked list • 1 node 2 links (prev. and next) • commonly used data • head/tail pointers
- 6. BASIC DATA STRUCTURES • Stack • Last in, First out. • data • top: index • array: stacked data • operations • push • pop
- 7. BASIC DATA STRUCTURES • Queue • First in, First out. • data • front, rear: indices • array: queued data • operations • add • remove rear add remove
- 8. TREES • Graph G = {V, E} • a set of vertices V and edges E ! ! • Tree • acyclic graph
- 9. TREES • rooted or not? • rooted tree • hierarchical structure root
- 10. HIERARCHICAL TREE • 1 root • multiple children for each node • n = maximum number of children • n-ary tree • binary tree • maximum number of children = 2 • level, height… level 1 level 2 level 3 height of the node 10 = 2 height of the root = tree height
- 11. SORTING - SELECTION SORT • list L = { sorted sublist L(+), unsorted sublist L(-) } • initialisation: L = L(+)L(-) • L(+) = nil • L(-) = L • Loop: • find “min” in L(-), swap “min” and the 1st element of L(-) • move the 1st element of L(-) to L(+) • repeat this loop until L(+)=L L(+) L(-)
- 12. SORTING - BUBBLE SORT • list L = { sorted sublist L(+), unsorted sublist L(-) } • initialisation: L = L(-)L(+) • L(-) = L • L(+) = nil • Loop: • from left to right in L(-) • compare two adjacent items and sort them • the biggest item will be moved to the right end • move the right-most element of L(-) to L(+) • repeat this loop until L(+)=L L(-) L(+) = 0 max will be here L(-) L(-) compare compare compare ...
- 13. SORTING - INSERTION SORT • list L = { sorted sublist L(+), unsorted sublist L(-) } • initialisation: L = L(+)L(-) • L(+) = nil • L(-) = L • Loop • take the 1st element of L(-) • find the proper location in L(+), and insert it • repeat this loop until L(+)=L
- 14. WHICH ONE IS BETTER? • average time complexity ! • almost-sorted list • insertion sort works better • because “proper location” can be quickly found • in many cases • “almost-sorted” lists are given (collision detection…) • insertion sort rather than bubble sort • http://http.developer.nvidia.com/GPUGems3/elementLinks/32fig01.jpg O(n2 )
- 15. QUICK SORT: A QUICKER SORT? • Quick sort • list L to be sorted • pivot element: a item x in L • L(-): list of items less than x • L(+): list of items greater than x • algorithm • select x in L • partition L into L(-), x, L(+) • recursive calls, one with L(-) and another with L(+)
- 16. HOW QUICK IS ‘QUICK SORT’? • If you are lucky • pivot divides the list evenly • recursive call depth = lgn • O(n) tasks for every level of the recursive call tree • overall time complexity: • When you’re down on your luck (already sorted list) • pivot has the minimum or the maximum value • O(n) recursive calls • overall time complexity: • Average: O(n lg n) O(n lg n) O(n2 )
- 17. MERGE SORT - GUARANTEED • algorithm • Loop 1: • evenly divide all lists • repeat Loop 1 until all the lists are atomic • Loop 2: • merge adjacent lists • repeat Loop 2 until we have only 1 list O(n lg n)
- 18. SEARCH • Sequential search • O(n) • more efficient methods are needed • self-organising list • frequently accessed items are moved to the front area • you are not going to be lucky forever
- 19. BINARY SEARCH • Rules • your key: k • you know value v at i: array[i] = v • if k=v you found it • if k>v the left part of array[i] can be ignored • if k<v the right part can be ignored • Algorithm • 1. compare key and the middle of the list • 2. • if matched return the index of the middle • if not matched remove the left or right from the middle in accordance with the rule above rules • 3. goto 1 while the list is remained
- 20. BINARY SEARCH TREE • simple rule for a node • all items in the left subtree is less than the node • all items in the right subtree is greater than the node • operations • if lucky • search: O(lgn) • add: O(lgn) • remove (next slide) • trivial cases • deleting leaf or a node with only one subtree • non trivial case • replace the removed item with the biggest in the left subtree or the smallest in the right subtree
- 21. BST NODE REMOVAL • Delete x • if it’s a leaf • just remove it • internal node case 1 • if it has only one subtree • link its parent and child • internal node case 2 • if it has two subtrees • find the min in the right subtree • copy ‘min’ to x • recursively delete ‘min’
- 22. 2-3-4 TREE • why it is needed • BST can be skewed • how to balance the tree: 2-3-4 tree • 3 kinds of nodes • 2 node • 1 key with 2 links (binary tree node) • 3 node • 2 keys with 3 links • 4 node • 3 keys with 4 links Patrick Prosser, 2-3-4 trees and red-black tree, course slide, http://www.dcs.gla.ac.uk/~pat/52233/slides/234_RB_trees1x1.pdf
- 23. 2-3-4 TREE: HOW IT WORKS • find the leaf in the same way as BST • if the leaf is 2- or 3-node, make it 3- or 4-node • if the leaf is 4-node • divide the 4-node into two 2-nodes • move the middle element upwards • put the new item • chained 4-node division? • pre-divide 4-nodes during the leaf search • node removal: a bit complicated 2-3-4 tree is balanced! guaranteed O(lg n) search Patrick Prosser, 2-3-4 trees and red-black tree, course slide, http://www.dcs.gla.ac.uk/~pat/52233/slides/234_RB_trees1x1.pdf
- 24. RED-BLACK TREE • 2-3-4 tree: different node structures • better implementation • red-black tree • 2-node • 3-node • 4-node no consecutive red edges balanced if you just count the black edges Patrick Prosser, 2-3-4 trees and red-black tree, course slide, http://www.dcs.gla.ac.uk/~pat/52233/slides/234_RB_trees1x1.pdf
- 25. RED-BLACK TREE • insertion • find leaf and add the new item • mark red the edge from the parent to the new node • keep “no consecutive red edges” rule • cases right rotation left rotation left-right double rotation right-left double rotation promotion rule if incoming edges of p and its sibling are red, turn them black and mark the incoming edge of g red Patrick Prosser, 2-3-4 trees and red-black tree, course slide, http://www.dcs.gla.ac.uk/~pat/52233/slides/234_RB_trees1x1.pdf
- 26. PRIORITY QUEUE - HEAP • types • max-heap: any of my descendants are less than me • min-heap: any of my descendants are greater than me • heap is a complete binary tree • always balanced • array implementation • i-th node • children: 2*i th and 2*i+1 th • parent: i 2 ⌫
- 27. PRIORITY QUEUE - HEAP • insert
- 28. PRIORITY QUEUE - HEAP • delete
- 29. HASH • hash function • any function that maps data of arbitrary size to data of fixed size with slight differences in input data producing very big difference in output data (Wikipedia, http://en.wikipedia.org/wiki/Hash_function) • hash functions are typically not invertible • hash table • data is stored as hash(data)-th element of the table • collision • if hash function is not perfect • solution • chaining • open addressing • linear probing, quadric probing, double hashing, random probing
- 30. HASH FUNCTIONS • hash functions • multiplication ! • Knuth suggested ! • division: hash(k) = k mod m • middle of square • middle p-bits of • folding • break key to pieces and combine them 0 hash(k) < m k2 A = p 5 1 2 hash(k) = m · (k · A bk · Ac) Golden ratio
- 31. GRAPH • graph G = {V,E} • V: set of vertices • E: set of edges linking two vertices • adjacent • if two nodes are connected with an edge • data structures for graph • adjacent matrix or adjacent list • graph types • directed or undirected • weighted or non-weighted undirected vs. directed weighted vs. non-weighted
- 32. GRAPH TRAVERSAL • Depth-first vs. Breadth-first search • stack vs. queue A B C D E F G H I DFS Stack [A] [AB] [ABC] [AB] [ABD] [ABDE] [ABDEF] [ABDE] [ABDEG] [ABDEGH] [ABDEG] [ABDGI] Traversal: A, B, C, D, E, F, G, H, I BFS Queue Traversal: A, B, C, D, E, G, F, H, I [A] [B] [CD] [D] [EGH] [GHF] [HFI] A B C D E F G H I A B C D E F G H I DFS tree BFS tree
- 33. MINIMUM COST SPANNING TREE • Greedy Algorithm • Prim’s method • two vertices sets: V, U • select a node v0 and initialise V={v0} • U = { all other vertices} • find the minimum edge (v in V, u in U) that links V and U • if it does not create a cycle, add u into V • Kruskal’s method • sort the edge list • add edge in increasing order • if it creates a cycle, throw it away
- 34. DIJKSTRA’S SHORTEST PATH • single source - multiple destination subgraph with vertices of which shortest path was found subgraph with vertices of which shortes paths are not known yet G+ G start u : newly added vertex d+ = 0 d+ u : found v1 v2 vn ... c1 c2 cn dv2 dv1 dvn initialisation d+ start = 0 dvi = 1 G+ = start G = G start u = start Loop d+ u + ci < dvi ) dvi d+ u + ci for all i find minimum dv in G add v into G+ , remove it from G fix dv to be d+ v u v While G is not empty
- 35. STRING SEARCH RABIN-KARP METHOD • Rabin-Karp • T: text with size of m characters • p: pattern with n characters • h = hash(p) • hi: hash value of n-character substring of T starting at T[i] • if h = hi, string “T[i]…T[i+n-1]” can be p • i=0 to m-n • Hash hi = n 1X j=0 T[i + j] · 2n 1 j hi+1 = hi T[i] · 2n 1 + T[i + n] abcdefghijklmnopqrstuvwxyz hash value = h1 hash value = h2 h2 = h1 b · 210 + m
- 36. STRING SEARCH KNUTH-MORRIS-PRATT METHOD text pattern mismatch 1 character shift naive approach text pattern mismatch jump to the mismatched location? we want... ANY PROBLEM? Sometimes we cannot jump that far... Why? text pattern mismatch identical ”Jump Table” is required border
- 37. STRING SEARCH BOYER-MOORE METHOD • right to left comparison • bad character shift • good suffix shift text pattern mismatch comparison direction bad character bad character shift text pattern mismatch comparison direction good su x case 1 case 2 good su x is found in the left su x of good su x matches the prefix of pattern
- 38. DYNAMIC PROGRAMMING LONGEST COMMON SUBSEQUENCE • subsequence • a sequence derived from another sequence by deleting some elements without altering the order of the elements • common subsequence • G is common subsequence of X and Y iff G is a subsequence of both X and Y • longest common subsequence • optimal substructure! Xi =< x1, x2, · · · , xi > Yj =< y1, y2, · · · , yj > |LCS(Xi, Yi)| = 0 @ 0 , if i = 0 or j = 0 |LCS(Xi 1, Yj 1)| + 1 , if xi = yj max(|LCS(Xi 1, Yj)|, |LCS(Xi, Yj 1)|) , if xi 6= yj and i, j > 0 1 A
- 39. PROBLEM SOLVING STRATEGIES • Divide & Conquer • Quick sort, Merge sort • Greedy algorithms • Optimal substructure • Minimum cost spanning tree, shortest path • easy to implement • Dynamic Programming • Optimal substructure • solving big problem by breaking down into simple subproblems