algorithm_5divide_and_conquer.pdf

1. Iris Hui-Ru Jiang Fall 2014 CHAPTER 5 DIVIDE AND CONQUER Preliminaries: Mathematical Induction 2 Divide-and-conquer IRIS H.-R. JIANG Weak Induction ¨ Given the propositional P(n) where nÎ À, a proof by mathematical induction is of the form: ¤ Basis step: The proposition P(0) is shown to be true ¤ Inductive step: The implication P(k) ®P(k + 1) is shown to be true for every kÎ À n In the inductive step, the statement P(k) is called the inductive hypothesis Divide-and-conquer 3 IRIS H.-R. JIANG Strong Induction ¨ Given the propositional P(n) where nÎ À, a proof by second principle of mathematical induction (or strong induction) is of the form: ¤ Basis step: The proposition P(0) is shown to be true ¤ Inductive step: The implication P(0) ÙP(1) Ù … Ù P(k) ®P(k + 1) is shown to be true for every kÎ À Divide-and-conquer 4

5. IRIS H.-R. JIANG Example: A Defective Chessboard ¨ Any 8Ŝ8 defective chessboard can be covered with twenty-one triominoes ¨ Q: How? Divide-and-conquer 5 Triomino IRIS H.-R. JIANG Example: A Defective Chessboard ¨ Any 8Ŝ8 defective chessboard can be covered with twenty-one triominoes ¨ Any 2nŜ2n defective chessboard can be covered with 1/3(2nŜ2n- 1) triominoes ¨ Prove by mathematical induction! Divide-and-conquer 6 IRIS H.-R. JIANG Mathematical Induction ¨ The first domino falls. ¨ If a domino falls, so will the next domino. ¨ All dominoes will fall! Divide-and-conquer 7 Courtesy of Prof. C.L. Liu 1 2 3 4 5 IRIS H.-R. JIANG Proof by Mathematical Induction ¨ Any 2nŜ2n defective chessboard can be covered with 1/3(2nŜ2n- 1) triominoes ¤ Basis step: n n=1 ¤ Inductive step: Divide-and-conquer 8 Triomino 2n+1 2n+1 2n 2n 2n 2n

9. IRIS H.-R. JIANG Proof vs. Algorithm ¨ Based on the defective chessboard, we can see Divide-and-conquer 9 Algorithm Proof IRIS H.-R. JIANG Outline ¨ Content: ¤ A first recurrence: the mergesort algorithm ¤ Counting inversions ¤ Finding the closest pair of points ¨ Reading: ¤ Chapter 5 Divide-and-conquer 10 IRIS H.-R. JIANG Warm Up: Searching ¨ Problem: Searching ¨ Given ¤ A sorted list of n distinct integers ¤ integer x ¨ Find ¤ j if x equals some integer of index j ¨ Solution: ¤ Naïve idea: compare one by one n Correct but slow: O(n) ¤ Better idea? n Hint: input is sorted Divide-and-conquer 11 Use known information to improve your solution IRIS H.-R. JIANG Binary Search ¨ Divide: Break the input into several parts of the same type. ¨ Conquer: Solve the problem in each part recursively. ¨ Combine: Combine the solutions to sub-problems into an overall solution ¨ Check the middle element ¨ Search the subarray recursively Divide-and-conquer paradigm Binary search on a sorted array 12 Divide-and-conquer 98 81 72 68 55 41 22 19 13 5 0 55 98 81 72 68 55 55 68 55 55 < 41 > 72 = 55 ¨ Divide: check the middle element ¨ Conquer: search the subarray recursively ¨ Combine: trivial

13. John von Neumann, 1945 Merge sort Mergesort 13 Divide-and-conquer http://en.wikipedia.org/wiki/File:Merge_sort_animation2.gif value index IRIS H.-R. JIANG Divide and Conquer ¨ Divide-and-conquer ¤ Divide: Break the input into several parts of the same type. ¤ Conquer: Solve the problem in each part recursively. ¤ Combine: Combine the solutions to sub-problems into an overall solution. ¨ Complexity: recurrence relation ¤ A divide and conquer algorithm is naturally implemented by a recursive procedure. ¤ The running time of a divide and conquer algorithm is generally represented by a recurrence relation that bounds the running time recursively in terms of the running time on smaller instances. ¨ Correctness: mathematical induction ¤ The basic idea is mathematical induction! Divide-and-conquer 14 IRIS H.-R. JIANG A Divide-and-Conquer Template ¨ A divide-and-conquer template ¤ Divide: divide the input into two pieces of equal size ¤ Conquer: solve the two subproblems on these pieces separately by recursion ¤ Combine: combine the two results into an overall solution ¤ Spend only linear time for the initial division and final recombining Divide-and-conquer 15 IRIS H.-R. JIANG Mergesort (1/2) ¨ Problem: Sorting ¨ Given ¤ A set of n numbers ¨ Find ¤ Sorted list in ascending order ¨ Solution: many! ¨ Mergesort fits the divide-and-conquer template ¤ Divide the input into two halves. ¤ Sort each half recursively. n Need base case ¤ Merge two halves into one. Divide-and-conquer 16 A L G O R I T H M S Stop recursion I T H M S A L G O R H I M S T A G L O R A G H I L M O R S T

17. IRIS H.-R. JIANG Mergesort (2/2) ¨ The base case: single element (trivially sorted) ¨ Running time: ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) ¤ Conquer: lines 3-4, 2T(n/2) ¤ Combine: line 5, C(n) ¤ T(n) = 2T(n/2) + D(n) + C(n) Divide-and-conquer 17 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) A L G O R I T H M S I T H M S A L G O R A G H I L M O R S T I T H M S A L G O R A L G I T H I T H M S A L G O R I T H A L G H I T M S A G L O R H I M S T A G L O R IRIS H.-R. JIANG Implementation: Division and Merging ¨ Running time: T(n) ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) n O(1) for array ¤ Combine: line 5, C(n) ¤ Q: Linear time O(n)? How? ¨ Efficient merging ¤ See the demonstration of Merge Divide-and-conquer 18 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) IRIS H.-R. JIANG Implementation: Division and Merging ¨ Running time: T(n) ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) n O(1) for array ¤ Combine: line 5, C(n) ¤ Q: Linear time O(n)? How? ¨ Efficient merging ¤ Linear number of comparisons ¤ Use an auxiliary array ¤ O(n) ¨ Merge sort is often the best choice for sorting a linked list ¤ Q: Why? How efficient on running time and storage? Divide-and-conquer 19 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) A G H I L H I M S T A G L O R IRIS H.-R. JIANG Recurrence Relation ¨ Running time: T(n) 1. Base case: for n £ 2, T(n) £ c 2. T(n) = 2T(n/2) + D(n) + C(n) n T(n) = 2T(n/2) + O(1) + O(n) n T(n) £ 2T(n/2) + cn ¤ A recursion corresponds to a recurrence relation n Recursion is a function defined by itself ¤ Q: Why not T(n) £ T(ën/2û) + T(én/2ù) + cn? ¤ A: The asymptotic bounds are not affected by ignoring ëû & éù. Divide-and-conquer 20 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r)

21. IRIS H.-R. JIANG Solving Recurrences ¨ Two basic ways to solve a recurrence ¤ Unrolling the recurrence (recursion tree) ¤ Substituting a guess ¨ Initially, we assume n is a power of 2 and replace £ with =. ¤ T(n) = 2T(n/2) + cn ¤ Simplify the problem by omitting floors and ceilings ¤ Solve the worst case Divide-and-conquer 21 IRIS H.-R. JIANG Unrolling – Recursion Tree ¨ Procedure 1. Analyzing the first few levels 2. Identifying a pattern 3. Summing over all levels Divide-and-conquer 22 T(n) T(n) cn T(n/2) T(n/2) cn/2 T(n/4) T(n/4) cn/2 T(n/4) T(n/4) cn/4 cn/4 cn/4 cn/4 Lvl 0: cn Lvl 1: cn Lvl 2: cn cn/2j … Lvl j: cn … Lvl k: cn T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) … k=log2n O(n log2n) T(n) = 2T(n/2) + cn T(2) = c IRIS H.-R. JIANG Substituting ¨ Any function T(.) satisfying this recurrence T(n) £ 2T(n/2) + cn when n > 2, and T(2) £ c is bounded by O(n log2 n), when n > 1. ¨ Pf: Guess and proof by induction ¨ Suppose we believe that T(n) £ cn log2 n for all n ³ 2 1. Base case: n = 2, T(2) £ c £ 2c. Indeed true. 2. Inductive step: n Inductive hypothesis: T(m) £ cm log2 m for all m < n. n T(n/2) £ c(n/2) log2(n/2); log2(n/2) = (log2 n) – 1 n T(n) £ 2T(n/2) + cn £ 2c(n/2) log2(n/2) + cn = cn [(log2 n) – 1] + cn = (cnlog2 n) – cn + cn = cnlog2 n. Divide-and-conquer 23 assume n is a power of 2 Counting Inversions 24 Divide-and-conquer

25. IRIS H.-R. JIANG Counting Inversions ¨ Music site tries to match your song preferences with others. ¤ You rank n songs. ¤ Music site consults database to find people with similar tastes. ¨ Similarity metric: number of inversions between two rankings. ¤ My rank: 1, 2, …, n. ¤ Your rank: a1, a2, …, an. ¤ Songs i and j inverted if i < j, but ai > aj. ¨ Brute force: check all Q(n2) pairs i and j. Divide-and-conquer 25 Me You A B C D E 1 2 3 4 5 2 4 1 3 5 1 2 3 4 5 2 4 1 3 5 Me You inversion = crossing (2, 1), (4, 1), (4, 3) IRIS H.-R. JIANG Divide and Conquer ¨ Counting inversions ¤ Divide: separate the list into two pieces. ¤ Conquer: recursively count inversions in each half. ¤ Combine: count inversions where ai and aj are in different halves, and return sum of three quantities. Divide-and-conquer 26 1 5 4 8 10 2 6 9 12 3 7 11 1 5 4 8 10 2 6 9 12 3 7 11 5 red-red inversions (5,4), (5,2), (4,2), (8,2), (10,2) 8 green-green inversions (6,3), (9,3), (9,7), (12,11), (12,3), (12,7), (11,3), (11,7) 9 red-green inversions (5,3), (4,3), (8,6), (8,3), (8,7), (10,6), (10,9), (10,3), (10,7) Combine: ?? Conquer: 2T(n/2) Divide: O(1) Total: 5 + 8 + 9 = 22 inversions IRIS H.-R. JIANG Combine? ¨ Inversions: inter and intra ¤ Intra: inversions within each half – done by conquer ¤ Inter: inversions between two halves – done by combine n The “combine” in Mergesort is done in O(n); goal: O(n log2 n) n Assume each half is sorted. n Count inversions where ai and aj are in different halves. n Merge two sorted halves into sorted whole. Divide-and-conquer 27 to maintain sorted invariant 1 2 4 5 8 10 3 6 7 11 12 9 (4,3), (5,3), (8,6), (10,3), (8,6), (10,6), (8,7), (10,7), (10,9) 9 red-green inversions Merge: O(n) Count: O(n) 4 2 2 0 0 1 1 2 3 4 5 6 7 8 9 11 12 10 Combine: O(n) Total: O(n log2 n) IRIS H.-R. JIANG Implementation: Counting Inversions ¨ Similar to Mergesort, extra effort on counting inter-inversions Divide-and-conquer 28 Sort-and-Count(L, p, q) // L[p..q]: initially unsorted 1. if (p = q) then return 0 2. else 3. m = ë(p+q)/2û 4. rp = Sort-and-Count(L, p, m) 5. rq = Sort-and-Count(L, m+1, q) 6. rm = Merge-and-Count(L, p, m, q) 7. return r = rp + rq + rm

29. M. I. Shamos and D. Hoey, 1975 Closest Pair of Points 29 Divide-and-conquer IRIS H.-R. JIANG Closest Pair of Points ¨ The closest pair of points problem ¨ Given ¤ A set of n points on a plane, pi is located at (xi, yi). ¨ Find ¤ A pair with the smallest Euclidean distance between them n Euclidean distance between pi and pj = [(xi - xj)2 + (yi - yj)2]1/2 Divide-and-conquer 30 IRIS H.-R. JIANG Closest Pair of Points: First Attempt ¨ Q: How? ¨ A: Brute-force? Q(n2) comparisons. ¨ Q: What if 1-D version? ¨ A: If points are all on a line, easy! O(n log n) for sorting. ¨ Q: What if 2-D version? ¨ A: Non-trivial. ¨ Assumption: No two points have same x-coordinate. Divide-and-conquer 31 to make presentation cleaner IRIS H.-R. JIANG L Divide-and-Conquer ¨ Divide: draw vertical line L so that half points on each side. ¨ Conquer: find closest pair in each side recursively. ¨ Combine: find closest pair with one point in each side; return best of 3 solutions. ¤ Q: How to “combine” in O(n)? Divide-and-conquer 32 M. I. Shamos and D. Hoey. "Closest-point problems." In Proc. 16th Annual IEEE Symposium on Foundations of Computer Science (FOCS), pp. 151—162, 1975 8 21 12

33. IRIS H.-R. JIANG Combining the Solutions (1/4) ¨ Find closest pair with one point in each side. ¤ L = {(x, y): x = x* = x-coordinate of the rightmost point in Q}. ¤ d = the smaller one of these two pairs. ¨ Observation: only need to consider points within d of line L. ¤ Q: Why? Divide-and-conquer 33 d = min(12, 21) L 21 12 d IRIS H.-R. JIANG Combining the Solutions (2/4) ¨ If $ qÎQ and rÎR for which d(q, r) < d, then each of q and r lies within a distance d of L. ¨ Pf: Suppose such q and r exist; let q = (qx, qy) and r = (rx, ry). ¤ By definition, qx £ x* £ rx. ¤ x* - qx £ rx - qx £ d(q, r) < d; rx - x* £ rx - qx £ d(q, r) < d. Divide-and-conquer 34 d = min(d(q*0, q*1), d(r*0, r*1)) L d(r*0, r*1) d(q*0, q*1) d Q R q r L: x = x* IRIS H.-R. JIANG Combining the Solutions (3/4) ¨ Observation: only need to consider points within d of line L. ¤ Sort points in 2d-strip by their y-coordinate. ¤ Only check distances of those within 15 positions in sorted list of y-coordinates! n Q: Why? Divide-and-conquer 35 d = min(12, 21) L 21 12 d 6 5 4 3 2 1 IRIS H.-R. JIANG Combining the Solutions (4/4) ¨ If s, s’ÎS are of d(s, s’) < d, then s and s’ are within 15 positions of each other in the sorted list Sy of y-coordinates of S. ¨ Pf: S contains all points within d of line L, we partition S into boxes, each box contains at most one point. ¤ Partition the region into boxes, each of area d/2*d/2; we claim 1. s and s’ lies in different boxes n Suppose s and s’ lies in the same box n These two points both belong either to Q or to R n d(s, s’) £ 0.5*d*(2)1/2 < d ®¬ 2. s and s’ are within 15 positions n Suppose s, s’ÎS of d(s, s’) < d and they are at least 16 positions apart in Sy. n Assume WLOG s has the smaller y-coordinate; since at most one point per box, at least 3 rows of S lying between s and s’. n d(s, s’) ³ 3d/2 > d ®¬ Divide-and-conquer 36 L d S d d/2 d/2

37. IRIS H.-R. JIANG Closest Pair Algorithm Divide-and-conquer 37 Closest-Pair-Rec(Px, Py) 1. if |P| £ 3 then return closest pair measured by all pair-wise distances 2. x* = (én/2ù)-th smallest x-coordinate in Px 3. construct Qx, Qy, Rx, Ry // O(n) 4. (q*0, q*1) = Closest-Pair-Rec(Qx, Qy) // T(n/2) 5. (r*0, r*1) = Closest-Pair-Rec(Rx, Ry) // T(n/2) 6. d = min(d(q*0, q*1), d(r*0, r*1)) 7. L = {(x, y): x = x*}; S = {points in P within distance d of L} 8. construct Sy // O(n) 9. for each sÎS do 10. compute distance from s to each of next 15 points in Sy 11.d(s, s’) = min distance of all computed distances // O(n) 12.if d(s, s’) < d then return (s, s’) 13.else if d(q*0, q*1) < d(r*0, r*1) then return (q*0, q*1) 14.else return (r*0, r*1) Closest-Pair(P) 1. construct Px and Py // O(n log n) 2. (p*0, p*1) = Closest-Pair-Rec(Px, Py) // T(n) Closest-Pair-Rec(Px, Py) 1. if |P| £ 3 then return closest pair measured by all pair-wise distances 2. x* = (én/2ù)-th smallest x-coordinate in Px 3. construct Qx, Qy, Rx, Ry 4. (q*0, q*1) = Closest-Pair-Rec(Qx, Qy) 5. (r*0, r*1) = Closest-Pair-Rec(Rx, Ry) 6. d = min(d(q*0, q*1), d(r*0, r*1)) 7. L = {(x, y): x = x*}; S = {points in P within distance d of L} 8. construct Sy 9. for each sÎS do 10. compute distance from s to each of next 15 points in Sy 11.d(s, s’) = min distance of all computed distances 12.if d(s, s’) < d then return (s, s’) 13.else if d(q*0, q*1) < d(r*0, r*1) then return (q*0, q*1) 14.else return (r*0, r*1) Closest-Pair(P) 1. construct Px and Py 2. (p*0, p*1) = Closest-Pair-Rec(Px, Py)

algorithm_5divide_and_conquer.pdf

Recommended

More Related Content

Similar to algorithm_5divide_and_conquer.pdf (20)

More from HsuChi Chen (14)

Recently uploaded (20)

algorithm_5divide_and_conquer.pdf