Abstract image of a woman sitting on books and studying on a laptop

algorithm_5divide_and_conquer.pdf

HsuChi Chen, profile picture
HsuChi Chen

The document discusses the principles of mathematical induction, specifically weak and strong induction, as proofs for propositions p(n). It includes examples, such as covering a defective chessboard with triominoes, and elaborates on the divide-and-conquer paradigm, particularly in relation to algorithms like binary search and mergesort. The author, Iris H.-R. Jiang, emphasizes the importance of mathematical induction in establishing the correctness of divide-and-conquer algorithms.

Software
Related topics:
Computer Science Basics
1 of 37
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
Iris Hui-Ru Jiang Fall 2014 CHAPTER 5 DIVIDE AND CONQUER Preliminaries: Mathematical Induction 2 Divide-and-conquer IRIS H.-R. JIANG Weak Induction ¨ Given the propositional P(n) where nÎ À, a proof by mathematical induction is of the form: ¤ Basis step: The proposition P(0) is shown to be true ¤ Inductive step: The implication P(k) ®P(k + 1) is shown to be true for every kÎ À n In the inductive step, the statement P(k) is called the inductive hypothesis Divide-and-conquer 3 IRIS H.-R. JIANG Strong Induction ¨ Given the propositional P(n) where nÎ À, a proof by second principle of mathematical induction (or strong induction) is of the form: ¤ Basis step: The proposition P(0) is shown to be true ¤ Inductive step: The implication P(0) ÙP(1) Ù … Ù P(k) ®P(k + 1) is shown to be true for every kÎ À Divide-and-conquer 4
Iris Hui-Ru Jiang Fall 2014 CHAPTER 5 DIVIDE AND CONQUER Preliminaries: Mathematical Induction 2 Divide-and-conquer IRIS H.-R. JIANG Weak Induction ¨ Given the propositional P(n) where nÎ À, a proof by mathematical induction is of the form: ¤ Basis step: The proposition P(0) is shown to be true ¤ Inductive step: The implication P(k) ®P(k + 1) is shown to be true for every kÎ À n In the inductive step, the statement P(k) is called the inductive hypothesis Divide-and-conquer 3 IRIS H.-R. JIANG Strong Induction ¨ Given the propositional P(n) where nÎ À, a proof by second principle of mathematical induction (or strong induction) is of the form: ¤ Basis step: The proposition P(0) is shown to be true ¤ Inductive step: The implication P(0) ÙP(1) Ù … Ù P(k) ®P(k + 1) is shown to be true for every kÎ À Divide-and-conquer 4
Iris Hui-Ru Jiang Fall 2014 CHAPTER 5 DIVIDE AND CONQUER Preliminaries: Mathematical Induction 2 Divide-and-conquer IRIS H.-R. JIANG Weak Induction ¨ Given the propositional P(n) where nÎ À, a proof by mathematical induction is of the form: ¤ Basis step: The proposition P(0) is shown to be true ¤ Inductive step: The implication P(k) ®P(k + 1) is shown to be true for every kÎ À n In the inductive step, the statement P(k) is called the inductive hypothesis Divide-and-conquer 3 IRIS H.-R. JIANG Strong Induction ¨ Given the propositional P(n) where nÎ À, a proof by second principle of mathematical induction (or strong induction) is of the form: ¤ Basis step: The proposition P(0) is shown to be true ¤ Inductive step: The implication P(0) ÙP(1) Ù … Ù P(k) ®P(k + 1) is shown to be true for every kÎ À Divide-and-conquer 4
Iris Hui-Ru Jiang Fall 2014 CHAPTER 5 DIVIDE AND CONQUER Preliminaries: Mathematical Induction 2 Divide-and-conquer IRIS H.-R. JIANG Weak Induction ¨ Given the propositional P(n) where nÎ À, a proof by mathematical induction is of the form: ¤ Basis step: The proposition P(0) is shown to be true ¤ Inductive step: The implication P(k) ®P(k + 1) is shown to be true for every kÎ À n In the inductive step, the statement P(k) is called the inductive hypothesis Divide-and-conquer 3 IRIS H.-R. JIANG Strong Induction ¨ Given the propositional P(n) where nÎ À, a proof by second principle of mathematical induction (or strong induction) is of the form: ¤ Basis step: The proposition P(0) is shown to be true ¤ Inductive step: The implication P(0) ÙP(1) Ù … Ù P(k) ®P(k + 1) is shown to be true for every kÎ À Divide-and-conquer 4
IRIS H.-R. JIANG Example: A Defective Chessboard ¨ Any 8Ŝ8 defective chessboard can be covered with twenty-one triominoes ¨ Q: How? Divide-and-conquer 5 Triomino IRIS H.-R. JIANG Example: A Defective Chessboard ¨ Any 8Ŝ8 defective chessboard can be covered with twenty-one triominoes ¨ Any 2nŜ2n defective chessboard can be covered with 1/3(2nŜ2n- 1) triominoes ¨ Prove by mathematical induction! Divide-and-conquer 6 IRIS H.-R. JIANG Mathematical Induction ¨ The first domino falls. ¨ If a domino falls, so will the next domino. ¨ All dominoes will fall! Divide-and-conquer 7 Courtesy of Prof. C.L. Liu 1 2 3 4 5 IRIS H.-R. JIANG Proof by Mathematical Induction ¨ Any 2nŜ2n defective chessboard can be covered with 1/3(2nŜ2n- 1) triominoes ¤ Basis step: n n=1 ¤ Inductive step: Divide-and-conquer 8 Triomino 2n+1 2n+1 2n 2n 2n 2n
IRIS H.-R. JIANG Example: A Defective Chessboard ¨ Any 8Ŝ8 defective chessboard can be covered with twenty-one triominoes ¨ Q: How? Divide-and-conquer 5 Triomino IRIS H.-R. JIANG Example: A Defective Chessboard ¨ Any 8Ŝ8 defective chessboard can be covered with twenty-one triominoes ¨ Any 2nŜ2n defective chessboard can be covered with 1/3(2nŜ2n- 1) triominoes ¨ Prove by mathematical induction! Divide-and-conquer 6 IRIS H.-R. JIANG Mathematical Induction ¨ The first domino falls. ¨ If a domino falls, so will the next domino. ¨ All dominoes will fall! Divide-and-conquer 7 Courtesy of Prof. C.L. Liu 1 2 3 4 5 IRIS H.-R. JIANG Proof by Mathematical Induction ¨ Any 2nŜ2n defective chessboard can be covered with 1/3(2nŜ2n- 1) triominoes ¤ Basis step: n n=1 ¤ Inductive step: Divide-and-conquer 8 Triomino 2n+1 2n+1 2n 2n 2n 2n
IRIS H.-R. JIANG Example: A Defective Chessboard ¨ Any 8Ŝ8 defective chessboard can be covered with twenty-one triominoes ¨ Q: How? Divide-and-conquer 5 Triomino IRIS H.-R. JIANG Example: A Defective Chessboard ¨ Any 8Ŝ8 defective chessboard can be covered with twenty-one triominoes ¨ Any 2nŜ2n defective chessboard can be covered with 1/3(2nŜ2n- 1) triominoes ¨ Prove by mathematical induction! Divide-and-conquer 6 IRIS H.-R. JIANG Mathematical Induction ¨ The first domino falls. ¨ If a domino falls, so will the next domino. ¨ All dominoes will fall! Divide-and-conquer 7 Courtesy of Prof. C.L. Liu 1 2 3 4 5 IRIS H.-R. JIANG Proof by Mathematical Induction ¨ Any 2nŜ2n defective chessboard can be covered with 1/3(2nŜ2n- 1) triominoes ¤ Basis step: n n=1 ¤ Inductive step: Divide-and-conquer 8 Triomino 2n+1 2n+1 2n 2n 2n 2n
IRIS H.-R. JIANG Example: A Defective Chessboard ¨ Any 8Ŝ8 defective chessboard can be covered with twenty-one triominoes ¨ Q: How? Divide-and-conquer 5 Triomino IRIS H.-R. JIANG Example: A Defective Chessboard ¨ Any 8Ŝ8 defective chessboard can be covered with twenty-one triominoes ¨ Any 2nŜ2n defective chessboard can be covered with 1/3(2nŜ2n- 1) triominoes ¨ Prove by mathematical induction! Divide-and-conquer 6 IRIS H.-R. JIANG Mathematical Induction ¨ The first domino falls. ¨ If a domino falls, so will the next domino. ¨ All dominoes will fall! Divide-and-conquer 7 Courtesy of Prof. C.L. Liu 1 2 3 4 5 IRIS H.-R. JIANG Proof by Mathematical Induction ¨ Any 2nŜ2n defective chessboard can be covered with 1/3(2nŜ2n- 1) triominoes ¤ Basis step: n n=1 ¤ Inductive step: Divide-and-conquer 8 Triomino 2n+1 2n+1 2n 2n 2n 2n
IRIS H.-R. JIANG Proof vs. Algorithm ¨ Based on the defective chessboard, we can see Divide-and-conquer 9 Algorithm Proof IRIS H.-R. JIANG Outline ¨ Content: ¤ A first recurrence: the mergesort algorithm ¤ Counting inversions ¤ Finding the closest pair of points ¨ Reading: ¤ Chapter 5 Divide-and-conquer 10 IRIS H.-R. JIANG Warm Up: Searching ¨ Problem: Searching ¨ Given ¤ A sorted list of n distinct integers ¤ integer x ¨ Find ¤ j if x equals some integer of index j ¨ Solution: ¤ Naïve idea: compare one by one n Correct but slow: O(n) ¤ Better idea? n Hint: input is sorted Divide-and-conquer 11 Use known information to improve your solution IRIS H.-R. JIANG Binary Search ¨ Divide: Break the input into several parts of the same type. ¨ Conquer: Solve the problem in each part recursively. ¨ Combine: Combine the solutions to sub-problems into an overall solution ¨ Check the middle element ¨ Search the subarray recursively Divide-and-conquer paradigm Binary search on a sorted array 12 Divide-and-conquer 98 81 72 68 55 41 22 19 13 5 0 55 98 81 72 68 55 55 68 55 55 < 41 > 72 = 55 ¨ Divide: check the middle element ¨ Conquer: search the subarray recursively ¨ Combine: trivial
IRIS H.-R. JIANG Proof vs. Algorithm ¨ Based on the defective chessboard, we can see Divide-and-conquer 9 Algorithm Proof IRIS H.-R. JIANG Outline ¨ Content: ¤ A first recurrence: the mergesort algorithm ¤ Counting inversions ¤ Finding the closest pair of points ¨ Reading: ¤ Chapter 5 Divide-and-conquer 10 IRIS H.-R. JIANG Warm Up: Searching ¨ Problem: Searching ¨ Given ¤ A sorted list of n distinct integers ¤ integer x ¨ Find ¤ j if x equals some integer of index j ¨ Solution: ¤ Naïve idea: compare one by one n Correct but slow: O(n) ¤ Better idea? n Hint: input is sorted Divide-and-conquer 11 Use known information to improve your solution IRIS H.-R. JIANG Binary Search ¨ Divide: Break the input into several parts of the same type. ¨ Conquer: Solve the problem in each part recursively. ¨ Combine: Combine the solutions to sub-problems into an overall solution ¨ Check the middle element ¨ Search the subarray recursively Divide-and-conquer paradigm Binary search on a sorted array 12 Divide-and-conquer 98 81 72 68 55 41 22 19 13 5 0 55 98 81 72 68 55 55 68 55 55 < 41 > 72 = 55 ¨ Divide: check the middle element ¨ Conquer: search the subarray recursively ¨ Combine: trivial
IRIS H.-R. JIANG Proof vs. Algorithm ¨ Based on the defective chessboard, we can see Divide-and-conquer 9 Algorithm Proof IRIS H.-R. JIANG Outline ¨ Content: ¤ A first recurrence: the mergesort algorithm ¤ Counting inversions ¤ Finding the closest pair of points ¨ Reading: ¤ Chapter 5 Divide-and-conquer 10 IRIS H.-R. JIANG Warm Up: Searching ¨ Problem: Searching ¨ Given ¤ A sorted list of n distinct integers ¤ integer x ¨ Find ¤ j if x equals some integer of index j ¨ Solution: ¤ Naïve idea: compare one by one n Correct but slow: O(n) ¤ Better idea? n Hint: input is sorted Divide-and-conquer 11 Use known information to improve your solution IRIS H.-R. JIANG Binary Search ¨ Divide: Break the input into several parts of the same type. ¨ Conquer: Solve the problem in each part recursively. ¨ Combine: Combine the solutions to sub-problems into an overall solution ¨ Check the middle element ¨ Search the subarray recursively Divide-and-conquer paradigm Binary search on a sorted array 12 Divide-and-conquer 98 81 72 68 55 41 22 19 13 5 0 55 98 81 72 68 55 55 68 55 55 < 41 > 72 = 55 ¨ Divide: check the middle element ¨ Conquer: search the subarray recursively ¨ Combine: trivial
IRIS H.-R. JIANG Proof vs. Algorithm ¨ Based on the defective chessboard, we can see Divide-and-conquer 9 Algorithm Proof IRIS H.-R. JIANG Outline ¨ Content: ¤ A first recurrence: the mergesort algorithm ¤ Counting inversions ¤ Finding the closest pair of points ¨ Reading: ¤ Chapter 5 Divide-and-conquer 10 IRIS H.-R. JIANG Warm Up: Searching ¨ Problem: Searching ¨ Given ¤ A sorted list of n distinct integers ¤ integer x ¨ Find ¤ j if x equals some integer of index j ¨ Solution: ¤ Naïve idea: compare one by one n Correct but slow: O(n) ¤ Better idea? n Hint: input is sorted Divide-and-conquer 11 Use known information to improve your solution IRIS H.-R. JIANG Binary Search ¨ Divide: Break the input into several parts of the same type. ¨ Conquer: Solve the problem in each part recursively. ¨ Combine: Combine the solutions to sub-problems into an overall solution ¨ Check the middle element ¨ Search the subarray recursively Divide-and-conquer paradigm Binary search on a sorted array 12 Divide-and-conquer 98 81 72 68 55 41 22 19 13 5 0 55 98 81 72 68 55 55 68 55 55 < 41 > 72 = 55 ¨ Divide: check the middle element ¨ Conquer: search the subarray recursively ¨ Combine: trivial
John von Neumann, 1945 Merge sort Mergesort 13 Divide-and-conquer http://en.wikipedia.org/wiki/File:Merge_sort_animation2.gif value index IRIS H.-R. JIANG Divide and Conquer ¨ Divide-and-conquer ¤ Divide: Break the input into several parts of the same type. ¤ Conquer: Solve the problem in each part recursively. ¤ Combine: Combine the solutions to sub-problems into an overall solution. ¨ Complexity: recurrence relation ¤ A divide and conquer algorithm is naturally implemented by a recursive procedure. ¤ The running time of a divide and conquer algorithm is generally represented by a recurrence relation that bounds the running time recursively in terms of the running time on smaller instances. ¨ Correctness: mathematical induction ¤ The basic idea is mathematical induction! Divide-and-conquer 14 IRIS H.-R. JIANG A Divide-and-Conquer Template ¨ A divide-and-conquer template ¤ Divide: divide the input into two pieces of equal size ¤ Conquer: solve the two subproblems on these pieces separately by recursion ¤ Combine: combine the two results into an overall solution ¤ Spend only linear time for the initial division and final recombining Divide-and-conquer 15 IRIS H.-R. JIANG Mergesort (1/2) ¨ Problem: Sorting ¨ Given ¤ A set of n numbers ¨ Find ¤ Sorted list in ascending order ¨ Solution: many! ¨ Mergesort fits the divide-and-conquer template ¤ Divide the input into two halves. ¤ Sort each half recursively. n Need base case ¤ Merge two halves into one. Divide-and-conquer 16 A L G O R I T H M S Stop recursion I T H M S A L G O R H I M S T A G L O R A G H I L M O R S T
John von Neumann, 1945 Merge sort Mergesort 13 Divide-and-conquer http://en.wikipedia.org/wiki/File:Merge_sort_animation2.gif value index IRIS H.-R. JIANG Divide and Conquer ¨ Divide-and-conquer ¤ Divide: Break the input into several parts of the same type. ¤ Conquer: Solve the problem in each part recursively. ¤ Combine: Combine the solutions to sub-problems into an overall solution. ¨ Complexity: recurrence relation ¤ A divide and conquer algorithm is naturally implemented by a recursive procedure. ¤ The running time of a divide and conquer algorithm is generally represented by a recurrence relation that bounds the running time recursively in terms of the running time on smaller instances. ¨ Correctness: mathematical induction ¤ The basic idea is mathematical induction! Divide-and-conquer 14 IRIS H.-R. JIANG A Divide-and-Conquer Template ¨ A divide-and-conquer template ¤ Divide: divide the input into two pieces of equal size ¤ Conquer: solve the two subproblems on these pieces separately by recursion ¤ Combine: combine the two results into an overall solution ¤ Spend only linear time for the initial division and final recombining Divide-and-conquer 15 IRIS H.-R. JIANG Mergesort (1/2) ¨ Problem: Sorting ¨ Given ¤ A set of n numbers ¨ Find ¤ Sorted list in ascending order ¨ Solution: many! ¨ Mergesort fits the divide-and-conquer template ¤ Divide the input into two halves. ¤ Sort each half recursively. n Need base case ¤ Merge two halves into one. Divide-and-conquer 16 A L G O R I T H M S Stop recursion I T H M S A L G O R H I M S T A G L O R A G H I L M O R S T
John von Neumann, 1945 Merge sort Mergesort 13 Divide-and-conquer http://en.wikipedia.org/wiki/File:Merge_sort_animation2.gif value index IRIS H.-R. JIANG Divide and Conquer ¨ Divide-and-conquer ¤ Divide: Break the input into several parts of the same type. ¤ Conquer: Solve the problem in each part recursively. ¤ Combine: Combine the solutions to sub-problems into an overall solution. ¨ Complexity: recurrence relation ¤ A divide and conquer algorithm is naturally implemented by a recursive procedure. ¤ The running time of a divide and conquer algorithm is generally represented by a recurrence relation that bounds the running time recursively in terms of the running time on smaller instances. ¨ Correctness: mathematical induction ¤ The basic idea is mathematical induction! Divide-and-conquer 14 IRIS H.-R. JIANG A Divide-and-Conquer Template ¨ A divide-and-conquer template ¤ Divide: divide the input into two pieces of equal size ¤ Conquer: solve the two subproblems on these pieces separately by recursion ¤ Combine: combine the two results into an overall solution ¤ Spend only linear time for the initial division and final recombining Divide-and-conquer 15 IRIS H.-R. JIANG Mergesort (1/2) ¨ Problem: Sorting ¨ Given ¤ A set of n numbers ¨ Find ¤ Sorted list in ascending order ¨ Solution: many! ¨ Mergesort fits the divide-and-conquer template ¤ Divide the input into two halves. ¤ Sort each half recursively. n Need base case ¤ Merge two halves into one. Divide-and-conquer 16 A L G O R I T H M S Stop recursion I T H M S A L G O R H I M S T A G L O R A G H I L M O R S T
John von Neumann, 1945 Merge sort Mergesort 13 Divide-and-conquer http://en.wikipedia.org/wiki/File:Merge_sort_animation2.gif value index IRIS H.-R. JIANG Divide and Conquer ¨ Divide-and-conquer ¤ Divide: Break the input into several parts of the same type. ¤ Conquer: Solve the problem in each part recursively. ¤ Combine: Combine the solutions to sub-problems into an overall solution. ¨ Complexity: recurrence relation ¤ A divide and conquer algorithm is naturally implemented by a recursive procedure. ¤ The running time of a divide and conquer algorithm is generally represented by a recurrence relation that bounds the running time recursively in terms of the running time on smaller instances. ¨ Correctness: mathematical induction ¤ The basic idea is mathematical induction! Divide-and-conquer 14 IRIS H.-R. JIANG A Divide-and-Conquer Template ¨ A divide-and-conquer template ¤ Divide: divide the input into two pieces of equal size ¤ Conquer: solve the two subproblems on these pieces separately by recursion ¤ Combine: combine the two results into an overall solution ¤ Spend only linear time for the initial division and final recombining Divide-and-conquer 15 IRIS H.-R. JIANG Mergesort (1/2) ¨ Problem: Sorting ¨ Given ¤ A set of n numbers ¨ Find ¤ Sorted list in ascending order ¨ Solution: many! ¨ Mergesort fits the divide-and-conquer template ¤ Divide the input into two halves. ¤ Sort each half recursively. n Need base case ¤ Merge two halves into one. Divide-and-conquer 16 A L G O R I T H M S Stop recursion I T H M S A L G O R H I M S T A G L O R A G H I L M O R S T
IRIS H.-R. JIANG Mergesort (2/2) ¨ The base case: single element (trivially sorted) ¨ Running time: ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) ¤ Conquer: lines 3-4, 2T(n/2) ¤ Combine: line 5, C(n) ¤ T(n) = 2T(n/2) + D(n) + C(n) Divide-and-conquer 17 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) A L G O R I T H M S I T H M S A L G O R A G H I L M O R S T I T H M S A L G O R A L G I T H I T H M S A L G O R I T H A L G H I T M S A G L O R H I M S T A G L O R IRIS H.-R. JIANG Implementation: Division and Merging ¨ Running time: T(n) ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) n O(1) for array ¤ Combine: line 5, C(n) ¤ Q: Linear time O(n)? How? ¨ Efficient merging ¤ See the demonstration of Merge Divide-and-conquer 18 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) IRIS H.-R. JIANG Implementation: Division and Merging ¨ Running time: T(n) ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) n O(1) for array ¤ Combine: line 5, C(n) ¤ Q: Linear time O(n)? How? ¨ Efficient merging ¤ Linear number of comparisons ¤ Use an auxiliary array ¤ O(n) ¨ Merge sort is often the best choice for sorting a linked list ¤ Q: Why? How efficient on running time and storage? Divide-and-conquer 19 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) A G H I L H I M S T A G L O R IRIS H.-R. JIANG Recurrence Relation ¨ Running time: T(n) 1. Base case: for n £ 2, T(n) £ c 2. T(n) = 2T(n/2) + D(n) + C(n) n T(n) = 2T(n/2) + O(1) + O(n) n T(n) £ 2T(n/2) + cn ¤ A recursion corresponds to a recurrence relation n Recursion is a function defined by itself ¤ Q: Why not T(n) £ T(ën/2û) + T(én/2ù) + cn? ¤ A: The asymptotic bounds are not affected by ignoring ëû & éù. Divide-and-conquer 20 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r)
IRIS H.-R. JIANG Mergesort (2/2) ¨ The base case: single element (trivially sorted) ¨ Running time: ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) ¤ Conquer: lines 3-4, 2T(n/2) ¤ Combine: line 5, C(n) ¤ T(n) = 2T(n/2) + D(n) + C(n) Divide-and-conquer 17 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) A L G O R I T H M S I T H M S A L G O R A G H I L M O R S T I T H M S A L G O R A L G I T H I T H M S A L G O R I T H A L G H I T M S A G L O R H I M S T A G L O R IRIS H.-R. JIANG Implementation: Division and Merging ¨ Running time: T(n) ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) n O(1) for array ¤ Combine: line 5, C(n) ¤ Q: Linear time O(n)? How? ¨ Efficient merging ¤ See the demonstration of Merge Divide-and-conquer 18 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) IRIS H.-R. JIANG Implementation: Division and Merging ¨ Running time: T(n) ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) n O(1) for array ¤ Combine: line 5, C(n) ¤ Q: Linear time O(n)? How? ¨ Efficient merging ¤ Linear number of comparisons ¤ Use an auxiliary array ¤ O(n) ¨ Merge sort is often the best choice for sorting a linked list ¤ Q: Why? How efficient on running time and storage? Divide-and-conquer 19 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) A G H I L H I M S T A G L O R IRIS H.-R. JIANG Recurrence Relation ¨ Running time: T(n) 1. Base case: for n £ 2, T(n) £ c 2. T(n) = 2T(n/2) + D(n) + C(n) n T(n) = 2T(n/2) + O(1) + O(n) n T(n) £ 2T(n/2) + cn ¤ A recursion corresponds to a recurrence relation n Recursion is a function defined by itself ¤ Q: Why not T(n) £ T(ën/2û) + T(én/2ù) + cn? ¤ A: The asymptotic bounds are not affected by ignoring ëû & éù. Divide-and-conquer 20 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r)
IRIS H.-R. JIANG Mergesort (2/2) ¨ The base case: single element (trivially sorted) ¨ Running time: ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) ¤ Conquer: lines 3-4, 2T(n/2) ¤ Combine: line 5, C(n) ¤ T(n) = 2T(n/2) + D(n) + C(n) Divide-and-conquer 17 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) A L G O R I T H M S I T H M S A L G O R A G H I L M O R S T I T H M S A L G O R A L G I T H I T H M S A L G O R I T H A L G H I T M S A G L O R H I M S T A G L O R IRIS H.-R. JIANG Implementation: Division and Merging ¨ Running time: T(n) ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) n O(1) for array ¤ Combine: line 5, C(n) ¤ Q: Linear time O(n)? How? ¨ Efficient merging ¤ See the demonstration of Merge Divide-and-conquer 18 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) IRIS H.-R. JIANG Implementation: Division and Merging ¨ Running time: T(n) ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) n O(1) for array ¤ Combine: line 5, C(n) ¤ Q: Linear time O(n)? How? ¨ Efficient merging ¤ Linear number of comparisons ¤ Use an auxiliary array ¤ O(n) ¨ Merge sort is often the best choice for sorting a linked list ¤ Q: Why? How efficient on running time and storage? Divide-and-conquer 19 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) A G H I L H I M S T A G L O R IRIS H.-R. JIANG Recurrence Relation ¨ Running time: T(n) 1. Base case: for n £ 2, T(n) £ c 2. T(n) = 2T(n/2) + D(n) + C(n) n T(n) = 2T(n/2) + O(1) + O(n) n T(n) £ 2T(n/2) + cn ¤ A recursion corresponds to a recurrence relation n Recursion is a function defined by itself ¤ Q: Why not T(n) £ T(ën/2û) + T(én/2ù) + cn? ¤ A: The asymptotic bounds are not affected by ignoring ëû & éù. Divide-and-conquer 20 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r)
IRIS H.-R. JIANG Mergesort (2/2) ¨ The base case: single element (trivially sorted) ¨ Running time: ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) ¤ Conquer: lines 3-4, 2T(n/2) ¤ Combine: line 5, C(n) ¤ T(n) = 2T(n/2) + D(n) + C(n) Divide-and-conquer 17 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) A L G O R I T H M S I T H M S A L G O R A G H I L M O R S T I T H M S A L G O R A L G I T H I T H M S A L G O R I T H A L G H I T M S A G L O R H I M S T A G L O R IRIS H.-R. JIANG Implementation: Division and Merging ¨ Running time: T(n) ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) n O(1) for array ¤ Combine: line 5, C(n) ¤ Q: Linear time O(n)? How? ¨ Efficient merging ¤ See the demonstration of Merge Divide-and-conquer 18 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) IRIS H.-R. JIANG Implementation: Division and Merging ¨ Running time: T(n) ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) n O(1) for array ¤ Combine: line 5, C(n) ¤ Q: Linear time O(n)? How? ¨ Efficient merging ¤ Linear number of comparisons ¤ Use an auxiliary array ¤ O(n) ¨ Merge sort is often the best choice for sorting a linked list ¤ Q: Why? How efficient on running time and storage? Divide-and-conquer 19 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) A G H I L H I M S T A G L O R IRIS H.-R. JIANG Recurrence Relation ¨ Running time: T(n) 1. Base case: for n £ 2, T(n) £ c 2. T(n) = 2T(n/2) + D(n) + C(n) n T(n) = 2T(n/2) + O(1) + O(n) n T(n) £ 2T(n/2) + cn ¤ A recursion corresponds to a recurrence relation n Recursion is a function defined by itself ¤ Q: Why not T(n) £ T(ën/2û) + T(én/2ù) + cn? ¤ A: The asymptotic bounds are not affected by ignoring ëû & éù. Divide-and-conquer 20 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r)
IRIS H.-R. JIANG Solving Recurrences ¨ Two basic ways to solve a recurrence ¤ Unrolling the recurrence (recursion tree) ¤ Substituting a guess ¨ Initially, we assume n is a power of 2 and replace £ with =. ¤ T(n) = 2T(n/2) + cn ¤ Simplify the problem by omitting floors and ceilings ¤ Solve the worst case Divide-and-conquer 21 IRIS H.-R. JIANG Unrolling – Recursion Tree ¨ Procedure 1. Analyzing the first few levels 2. Identifying a pattern 3. Summing over all levels Divide-and-conquer 22 T(n) T(n) cn T(n/2) T(n/2) cn/2 T(n/4) T(n/4) cn/2 T(n/4) T(n/4) cn/4 cn/4 cn/4 cn/4 Lvl 0: cn Lvl 1: cn Lvl 2: cn cn/2j … Lvl j: cn … Lvl k: cn T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) … k=log2n O(n log2n) T(n) = 2T(n/2) + cn T(2) = c IRIS H.-R. JIANG Substituting ¨ Any function T(.) satisfying this recurrence T(n) £ 2T(n/2) + cn when n > 2, and T(2) £ c is bounded by O(n log2 n), when n > 1. ¨ Pf: Guess and proof by induction ¨ Suppose we believe that T(n) £ cn log2 n for all n ³ 2 1. Base case: n = 2, T(2) £ c £ 2c. Indeed true. 2. Inductive step: n Inductive hypothesis: T(m) £ cm log2 m for all m < n. n T(n/2) £ c(n/2) log2(n/2); log2(n/2) = (log2 n) – 1 n T(n) £ 2T(n/2) + cn £ 2c(n/2) log2(n/2) + cn = cn [(log2 n) – 1] + cn = (cnlog2 n) – cn + cn = cnlog2 n. Divide-and-conquer 23 assume n is a power of 2 Counting Inversions 24 Divide-and-conquer
IRIS H.-R. JIANG Solving Recurrences ¨ Two basic ways to solve a recurrence ¤ Unrolling the recurrence (recursion tree) ¤ Substituting a guess ¨ Initially, we assume n is a power of 2 and replace £ with =. ¤ T(n) = 2T(n/2) + cn ¤ Simplify the problem by omitting floors and ceilings ¤ Solve the worst case Divide-and-conquer 21 IRIS H.-R. JIANG Unrolling – Recursion Tree ¨ Procedure 1. Analyzing the first few levels 2. Identifying a pattern 3. Summing over all levels Divide-and-conquer 22 T(n) T(n) cn T(n/2) T(n/2) cn/2 T(n/4) T(n/4) cn/2 T(n/4) T(n/4) cn/4 cn/4 cn/4 cn/4 Lvl 0: cn Lvl 1: cn Lvl 2: cn cn/2j … Lvl j: cn … Lvl k: cn T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) … k=log2n O(n log2n) T(n) = 2T(n/2) + cn T(2) = c IRIS H.-R. JIANG Substituting ¨ Any function T(.) satisfying this recurrence T(n) £ 2T(n/2) + cn when n > 2, and T(2) £ c is bounded by O(n log2 n), when n > 1. ¨ Pf: Guess and proof by induction ¨ Suppose we believe that T(n) £ cn log2 n for all n ³ 2 1. Base case: n = 2, T(2) £ c £ 2c. Indeed true. 2. Inductive step: n Inductive hypothesis: T(m) £ cm log2 m for all m < n. n T(n/2) £ c(n/2) log2(n/2); log2(n/2) = (log2 n) – 1 n T(n) £ 2T(n/2) + cn £ 2c(n/2) log2(n/2) + cn = cn [(log2 n) – 1] + cn = (cnlog2 n) – cn + cn = cnlog2 n. Divide-and-conquer 23 assume n is a power of 2 Counting Inversions 24 Divide-and-conquer
IRIS H.-R. JIANG Solving Recurrences ¨ Two basic ways to solve a recurrence ¤ Unrolling the recurrence (recursion tree) ¤ Substituting a guess ¨ Initially, we assume n is a power of 2 and replace £ with =. ¤ T(n) = 2T(n/2) + cn ¤ Simplify the problem by omitting floors and ceilings ¤ Solve the worst case Divide-and-conquer 21 IRIS H.-R. JIANG Unrolling – Recursion Tree ¨ Procedure 1. Analyzing the first few levels 2. Identifying a pattern 3. Summing over all levels Divide-and-conquer 22 T(n) T(n) cn T(n/2) T(n/2) cn/2 T(n/4) T(n/4) cn/2 T(n/4) T(n/4) cn/4 cn/4 cn/4 cn/4 Lvl 0: cn Lvl 1: cn Lvl 2: cn cn/2j … Lvl j: cn … Lvl k: cn T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) … k=log2n O(n log2n) T(n) = 2T(n/2) + cn T(2) = c IRIS H.-R. JIANG Substituting ¨ Any function T(.) satisfying this recurrence T(n) £ 2T(n/2) + cn when n > 2, and T(2) £ c is bounded by O(n log2 n), when n > 1. ¨ Pf: Guess and proof by induction ¨ Suppose we believe that T(n) £ cn log2 n for all n ³ 2 1. Base case: n = 2, T(2) £ c £ 2c. Indeed true. 2. Inductive step: n Inductive hypothesis: T(m) £ cm log2 m for all m < n. n T(n/2) £ c(n/2) log2(n/2); log2(n/2) = (log2 n) – 1 n T(n) £ 2T(n/2) + cn £ 2c(n/2) log2(n/2) + cn = cn [(log2 n) – 1] + cn = (cnlog2 n) – cn + cn = cnlog2 n. Divide-and-conquer 23 assume n is a power of 2 Counting Inversions 24 Divide-and-conquer
IRIS H.-R. JIANG Solving Recurrences ¨ Two basic ways to solve a recurrence ¤ Unrolling the recurrence (recursion tree) ¤ Substituting a guess ¨ Initially, we assume n is a power of 2 and replace £ with =. ¤ T(n) = 2T(n/2) + cn ¤ Simplify the problem by omitting floors and ceilings ¤ Solve the worst case Divide-and-conquer 21 IRIS H.-R. JIANG Unrolling – Recursion Tree ¨ Procedure 1. Analyzing the first few levels 2. Identifying a pattern 3. Summing over all levels Divide-and-conquer 22 T(n) T(n) cn T(n/2) T(n/2) cn/2 T(n/4) T(n/4) cn/2 T(n/4) T(n/4) cn/4 cn/4 cn/4 cn/4 Lvl 0: cn Lvl 1: cn Lvl 2: cn cn/2j … Lvl j: cn … Lvl k: cn T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) … k=log2n O(n log2n) T(n) = 2T(n/2) + cn T(2) = c IRIS H.-R. JIANG Substituting ¨ Any function T(.) satisfying this recurrence T(n) £ 2T(n/2) + cn when n > 2, and T(2) £ c is bounded by O(n log2 n), when n > 1. ¨ Pf: Guess and proof by induction ¨ Suppose we believe that T(n) £ cn log2 n for all n ³ 2 1. Base case: n = 2, T(2) £ c £ 2c. Indeed true. 2. Inductive step: n Inductive hypothesis: T(m) £ cm log2 m for all m < n. n T(n/2) £ c(n/2) log2(n/2); log2(n/2) = (log2 n) – 1 n T(n) £ 2T(n/2) + cn £ 2c(n/2) log2(n/2) + cn = cn [(log2 n) – 1] + cn = (cnlog2 n) – cn + cn = cnlog2 n. Divide-and-conquer 23 assume n is a power of 2 Counting Inversions 24 Divide-and-conquer
IRIS H.-R. JIANG Counting Inversions ¨ Music site tries to match your song preferences with others. ¤ You rank n songs. ¤ Music site consults database to find people with similar tastes. ¨ Similarity metric: number of inversions between two rankings. ¤ My rank: 1, 2, …, n. ¤ Your rank: a1, a2, …, an. ¤ Songs i and j inverted if i < j, but ai > aj. ¨ Brute force: check all Q(n2) pairs i and j. Divide-and-conquer 25 Me You A B C D E 1 2 3 4 5 2 4 1 3 5 1 2 3 4 5 2 4 1 3 5 Me You inversion = crossing (2, 1), (4, 1), (4, 3) IRIS H.-R. JIANG Divide and Conquer ¨ Counting inversions ¤ Divide: separate the list into two pieces. ¤ Conquer: recursively count inversions in each half. ¤ Combine: count inversions where ai and aj are in different halves, and return sum of three quantities. Divide-and-conquer 26 1 5 4 8 10 2 6 9 12 3 7 11 1 5 4 8 10 2 6 9 12 3 7 11 5 red-red inversions (5,4), (5,2), (4,2), (8,2), (10,2) 8 green-green inversions (6,3), (9,3), (9,7), (12,11), (12,3), (12,7), (11,3), (11,7) 9 red-green inversions (5,3), (4,3), (8,6), (8,3), (8,7), (10,6), (10,9), (10,3), (10,7) Combine: ?? Conquer: 2T(n/2) Divide: O(1) Total: 5 + 8 + 9 = 22 inversions IRIS H.-R. JIANG Combine? ¨ Inversions: inter and intra ¤ Intra: inversions within each half – done by conquer ¤ Inter: inversions between two halves – done by combine n The “combine” in Mergesort is done in O(n); goal: O(n log2 n) n Assume each half is sorted. n Count inversions where ai and aj are in different halves. n Merge two sorted halves into sorted whole. Divide-and-conquer 27 to maintain sorted invariant 1 2 4 5 8 10 3 6 7 11 12 9 (4,3), (5,3), (8,6), (10,3), (8,6), (10,6), (8,7), (10,7), (10,9) 9 red-green inversions Merge: O(n) Count: O(n) 4 2 2 0 0 1 1 2 3 4 5 6 7 8 9 11 12 10 Combine: O(n) Total: O(n log2 n) IRIS H.-R. JIANG Implementation: Counting Inversions ¨ Similar to Mergesort, extra effort on counting inter-inversions Divide-and-conquer 28 Sort-and-Count(L, p, q) // L[p..q]: initially unsorted 1. if (p = q) then return 0 2. else 3. m = ë(p+q)/2û 4. rp = Sort-and-Count(L, p, m) 5. rq = Sort-and-Count(L, m+1, q) 6. rm = Merge-and-Count(L, p, m, q) 7. return r = rp + rq + rm
IRIS H.-R. JIANG Counting Inversions ¨ Music site tries to match your song preferences with others. ¤ You rank n songs. ¤ Music site consults database to find people with similar tastes. ¨ Similarity metric: number of inversions between two rankings. ¤ My rank: 1, 2, …, n. ¤ Your rank: a1, a2, …, an. ¤ Songs i and j inverted if i < j, but ai > aj. ¨ Brute force: check all Q(n2) pairs i and j. Divide-and-conquer 25 Me You A B C D E 1 2 3 4 5 2 4 1 3 5 1 2 3 4 5 2 4 1 3 5 Me You inversion = crossing (2, 1), (4, 1), (4, 3) IRIS H.-R. JIANG Divide and Conquer ¨ Counting inversions ¤ Divide: separate the list into two pieces. ¤ Conquer: recursively count inversions in each half. ¤ Combine: count inversions where ai and aj are in different halves, and return sum of three quantities. Divide-and-conquer 26 1 5 4 8 10 2 6 9 12 3 7 11 1 5 4 8 10 2 6 9 12 3 7 11 5 red-red inversions (5,4), (5,2), (4,2), (8,2), (10,2) 8 green-green inversions (6,3), (9,3), (9,7), (12,11), (12,3), (12,7), (11,3), (11,7) 9 red-green inversions (5,3), (4,3), (8,6), (8,3), (8,7), (10,6), (10,9), (10,3), (10,7) Combine: ?? Conquer: 2T(n/2) Divide: O(1) Total: 5 + 8 + 9 = 22 inversions IRIS H.-R. JIANG Combine? ¨ Inversions: inter and intra ¤ Intra: inversions within each half – done by conquer ¤ Inter: inversions between two halves – done by combine n The “combine” in Mergesort is done in O(n); goal: O(n log2 n) n Assume each half is sorted. n Count inversions where ai and aj are in different halves. n Merge two sorted halves into sorted whole. Divide-and-conquer 27 to maintain sorted invariant 1 2 4 5 8 10 3 6 7 11 12 9 (4,3), (5,3), (8,6), (10,3), (8,6), (10,6), (8,7), (10,7), (10,9) 9 red-green inversions Merge: O(n) Count: O(n) 4 2 2 0 0 1 1 2 3 4 5 6 7 8 9 11 12 10 Combine: O(n) Total: O(n log2 n) IRIS H.-R. JIANG Implementation: Counting Inversions ¨ Similar to Mergesort, extra effort on counting inter-inversions Divide-and-conquer 28 Sort-and-Count(L, p, q) // L[p..q]: initially unsorted 1. if (p = q) then return 0 2. else 3. m = ë(p+q)/2û 4. rp = Sort-and-Count(L, p, m) 5. rq = Sort-and-Count(L, m+1, q) 6. rm = Merge-and-Count(L, p, m, q) 7. return r = rp + rq + rm
IRIS H.-R. JIANG Counting Inversions ¨ Music site tries to match your song preferences with others. ¤ You rank n songs. ¤ Music site consults database to find people with similar tastes. ¨ Similarity metric: number of inversions between two rankings. ¤ My rank: 1, 2, …, n. ¤ Your rank: a1, a2, …, an. ¤ Songs i and j inverted if i < j, but ai > aj. ¨ Brute force: check all Q(n2) pairs i and j. Divide-and-conquer 25 Me You A B C D E 1 2 3 4 5 2 4 1 3 5 1 2 3 4 5 2 4 1 3 5 Me You inversion = crossing (2, 1), (4, 1), (4, 3) IRIS H.-R. JIANG Divide and Conquer ¨ Counting inversions ¤ Divide: separate the list into two pieces. ¤ Conquer: recursively count inversions in each half. ¤ Combine: count inversions where ai and aj are in different halves, and return sum of three quantities. Divide-and-conquer 26 1 5 4 8 10 2 6 9 12 3 7 11 1 5 4 8 10 2 6 9 12 3 7 11 5 red-red inversions (5,4), (5,2), (4,2), (8,2), (10,2) 8 green-green inversions (6,3), (9,3), (9,7), (12,11), (12,3), (12,7), (11,3), (11,7) 9 red-green inversions (5,3), (4,3), (8,6), (8,3), (8,7), (10,6), (10,9), (10,3), (10,7) Combine: ?? Conquer: 2T(n/2) Divide: O(1) Total: 5 + 8 + 9 = 22 inversions IRIS H.-R. JIANG Combine? ¨ Inversions: inter and intra ¤ Intra: inversions within each half – done by conquer ¤ Inter: inversions between two halves – done by combine n The “combine” in Mergesort is done in O(n); goal: O(n log2 n) n Assume each half is sorted. n Count inversions where ai and aj are in different halves. n Merge two sorted halves into sorted whole. Divide-and-conquer 27 to maintain sorted invariant 1 2 4 5 8 10 3 6 7 11 12 9 (4,3), (5,3), (8,6), (10,3), (8,6), (10,6), (8,7), (10,7), (10,9) 9 red-green inversions Merge: O(n) Count: O(n) 4 2 2 0 0 1 1 2 3 4 5 6 7 8 9 11 12 10 Combine: O(n) Total: O(n log2 n) IRIS H.-R. JIANG Implementation: Counting Inversions ¨ Similar to Mergesort, extra effort on counting inter-inversions Divide-and-conquer 28 Sort-and-Count(L, p, q) // L[p..q]: initially unsorted 1. if (p = q) then return 0 2. else 3. m = ë(p+q)/2û 4. rp = Sort-and-Count(L, p, m) 5. rq = Sort-and-Count(L, m+1, q) 6. rm = Merge-and-Count(L, p, m, q) 7. return r = rp + rq + rm
IRIS H.-R. JIANG Counting Inversions ¨ Music site tries to match your song preferences with others. ¤ You rank n songs. ¤ Music site consults database to find people with similar tastes. ¨ Similarity metric: number of inversions between two rankings. ¤ My rank: 1, 2, …, n. ¤ Your rank: a1, a2, …, an. ¤ Songs i and j inverted if i < j, but ai > aj. ¨ Brute force: check all Q(n2) pairs i and j. Divide-and-conquer 25 Me You A B C D E 1 2 3 4 5 2 4 1 3 5 1 2 3 4 5 2 4 1 3 5 Me You inversion = crossing (2, 1), (4, 1), (4, 3) IRIS H.-R. JIANG Divide and Conquer ¨ Counting inversions ¤ Divide: separate the list into two pieces. ¤ Conquer: recursively count inversions in each half. ¤ Combine: count inversions where ai and aj are in different halves, and return sum of three quantities. Divide-and-conquer 26 1 5 4 8 10 2 6 9 12 3 7 11 1 5 4 8 10 2 6 9 12 3 7 11 5 red-red inversions (5,4), (5,2), (4,2), (8,2), (10,2) 8 green-green inversions (6,3), (9,3), (9,7), (12,11), (12,3), (12,7), (11,3), (11,7) 9 red-green inversions (5,3), (4,3), (8,6), (8,3), (8,7), (10,6), (10,9), (10,3), (10,7) Combine: ?? Conquer: 2T(n/2) Divide: O(1) Total: 5 + 8 + 9 = 22 inversions IRIS H.-R. JIANG Combine? ¨ Inversions: inter and intra ¤ Intra: inversions within each half – done by conquer ¤ Inter: inversions between two halves – done by combine n The “combine” in Mergesort is done in O(n); goal: O(n log2 n) n Assume each half is sorted. n Count inversions where ai and aj are in different halves. n Merge two sorted halves into sorted whole. Divide-and-conquer 27 to maintain sorted invariant 1 2 4 5 8 10 3 6 7 11 12 9 (4,3), (5,3), (8,6), (10,3), (8,6), (10,6), (8,7), (10,7), (10,9) 9 red-green inversions Merge: O(n) Count: O(n) 4 2 2 0 0 1 1 2 3 4 5 6 7 8 9 11 12 10 Combine: O(n) Total: O(n log2 n) IRIS H.-R. JIANG Implementation: Counting Inversions ¨ Similar to Mergesort, extra effort on counting inter-inversions Divide-and-conquer 28 Sort-and-Count(L, p, q) // L[p..q]: initially unsorted 1. if (p = q) then return 0 2. else 3. m = ë(p+q)/2û 4. rp = Sort-and-Count(L, p, m) 5. rq = Sort-and-Count(L, m+1, q) 6. rm = Merge-and-Count(L, p, m, q) 7. return r = rp + rq + rm
M. I. Shamos and D. Hoey, 1975 Closest Pair of Points 29 Divide-and-conquer IRIS H.-R. JIANG Closest Pair of Points ¨ The closest pair of points problem ¨ Given ¤ A set of n points on a plane, pi is located at (xi, yi). ¨ Find ¤ A pair with the smallest Euclidean distance between them n Euclidean distance between pi and pj = [(xi - xj)2 + (yi - yj)2]1/2 Divide-and-conquer 30 IRIS H.-R. JIANG Closest Pair of Points: First Attempt ¨ Q: How? ¨ A: Brute-force? Q(n2) comparisons. ¨ Q: What if 1-D version? ¨ A: If points are all on a line, easy! O(n log n) for sorting. ¨ Q: What if 2-D version? ¨ A: Non-trivial. ¨ Assumption: No two points have same x-coordinate. Divide-and-conquer 31 to make presentation cleaner IRIS H.-R. JIANG L Divide-and-Conquer ¨ Divide: draw vertical line L so that half points on each side. ¨ Conquer: find closest pair in each side recursively. ¨ Combine: find closest pair with one point in each side; return best of 3 solutions. ¤ Q: How to “combine” in O(n)? Divide-and-conquer 32 M. I. Shamos and D. Hoey. "Closest-point problems." In Proc. 16th Annual IEEE Symposium on Foundations of Computer Science (FOCS), pp. 151—162, 1975 8 21 12
M. I. Shamos and D. Hoey, 1975 Closest Pair of Points 29 Divide-and-conquer IRIS H.-R. JIANG Closest Pair of Points ¨ The closest pair of points problem ¨ Given ¤ A set of n points on a plane, pi is located at (xi, yi). ¨ Find ¤ A pair with the smallest Euclidean distance between them n Euclidean distance between pi and pj = [(xi - xj)2 + (yi - yj)2]1/2 Divide-and-conquer 30 IRIS H.-R. JIANG Closest Pair of Points: First Attempt ¨ Q: How? ¨ A: Brute-force? Q(n2) comparisons. ¨ Q: What if 1-D version? ¨ A: If points are all on a line, easy! O(n log n) for sorting. ¨ Q: What if 2-D version? ¨ A: Non-trivial. ¨ Assumption: No two points have same x-coordinate. Divide-and-conquer 31 to make presentation cleaner IRIS H.-R. JIANG L Divide-and-Conquer ¨ Divide: draw vertical line L so that half points on each side. ¨ Conquer: find closest pair in each side recursively. ¨ Combine: find closest pair with one point in each side; return best of 3 solutions. ¤ Q: How to “combine” in O(n)? Divide-and-conquer 32 M. I. Shamos and D. Hoey. "Closest-point problems." In Proc. 16th Annual IEEE Symposium on Foundations of Computer Science (FOCS), pp. 151—162, 1975 8 21 12
M. I. Shamos and D. Hoey, 1975 Closest Pair of Points 29 Divide-and-conquer IRIS H.-R. JIANG Closest Pair of Points ¨ The closest pair of points problem ¨ Given ¤ A set of n points on a plane, pi is located at (xi, yi). ¨ Find ¤ A pair with the smallest Euclidean distance between them n Euclidean distance between pi and pj = [(xi - xj)2 + (yi - yj)2]1/2 Divide-and-conquer 30 IRIS H.-R. JIANG Closest Pair of Points: First Attempt ¨ Q: How? ¨ A: Brute-force? Q(n2) comparisons. ¨ Q: What if 1-D version? ¨ A: If points are all on a line, easy! O(n log n) for sorting. ¨ Q: What if 2-D version? ¨ A: Non-trivial. ¨ Assumption: No two points have same x-coordinate. Divide-and-conquer 31 to make presentation cleaner IRIS H.-R. JIANG L Divide-and-Conquer ¨ Divide: draw vertical line L so that half points on each side. ¨ Conquer: find closest pair in each side recursively. ¨ Combine: find closest pair with one point in each side; return best of 3 solutions. ¤ Q: How to “combine” in O(n)? Divide-and-conquer 32 M. I. Shamos and D. Hoey. "Closest-point problems." In Proc. 16th Annual IEEE Symposium on Foundations of Computer Science (FOCS), pp. 151—162, 1975 8 21 12
M. I. Shamos and D. Hoey, 1975 Closest Pair of Points 29 Divide-and-conquer IRIS H.-R. JIANG Closest Pair of Points ¨ The closest pair of points problem ¨ Given ¤ A set of n points on a plane, pi is located at (xi, yi). ¨ Find ¤ A pair with the smallest Euclidean distance between them n Euclidean distance between pi and pj = [(xi - xj)2 + (yi - yj)2]1/2 Divide-and-conquer 30 IRIS H.-R. JIANG Closest Pair of Points: First Attempt ¨ Q: How? ¨ A: Brute-force? Q(n2) comparisons. ¨ Q: What if 1-D version? ¨ A: If points are all on a line, easy! O(n log n) for sorting. ¨ Q: What if 2-D version? ¨ A: Non-trivial. ¨ Assumption: No two points have same x-coordinate. Divide-and-conquer 31 to make presentation cleaner IRIS H.-R. JIANG L Divide-and-Conquer ¨ Divide: draw vertical line L so that half points on each side. ¨ Conquer: find closest pair in each side recursively. ¨ Combine: find closest pair with one point in each side; return best of 3 solutions. ¤ Q: How to “combine” in O(n)? Divide-and-conquer 32 M. I. Shamos and D. Hoey. "Closest-point problems." In Proc. 16th Annual IEEE Symposium on Foundations of Computer Science (FOCS), pp. 151—162, 1975 8 21 12
IRIS H.-R. JIANG Combining the Solutions (1/4) ¨ Find closest pair with one point in each side. ¤ L = {(x, y): x = x* = x-coordinate of the rightmost point in Q}. ¤ d = the smaller one of these two pairs. ¨ Observation: only need to consider points within d of line L. ¤ Q: Why? Divide-and-conquer 33 d = min(12, 21) L 21 12 d IRIS H.-R. JIANG Combining the Solutions (2/4) ¨ If $ qÎQ and rÎR for which d(q, r) < d, then each of q and r lies within a distance d of L. ¨ Pf: Suppose such q and r exist; let q = (qx, qy) and r = (rx, ry). ¤ By definition, qx £ x* £ rx. ¤ x* - qx £ rx - qx £ d(q, r) < d; rx - x* £ rx - qx £ d(q, r) < d. Divide-and-conquer 34 d = min(d(q*0, q*1), d(r*0, r*1)) L d(r*0, r*1) d(q*0, q*1) d Q R q r L: x = x* IRIS H.-R. JIANG Combining the Solutions (3/4) ¨ Observation: only need to consider points within d of line L. ¤ Sort points in 2d-strip by their y-coordinate. ¤ Only check distances of those within 15 positions in sorted list of y-coordinates! n Q: Why? Divide-and-conquer 35 d = min(12, 21) L 21 12 d 6 5 4 3 2 1 IRIS H.-R. JIANG Combining the Solutions (4/4) ¨ If s, s’ÎS are of d(s, s’) < d, then s and s’ are within 15 positions of each other in the sorted list Sy of y-coordinates of S. ¨ Pf: S contains all points within d of line L, we partition S into boxes, each box contains at most one point. ¤ Partition the region into boxes, each of area d/2*d/2; we claim 1. s and s’ lies in different boxes n Suppose s and s’ lies in the same box n These two points both belong either to Q or to R n d(s, s’) £ 0.5*d*(2)1/2 < d ®¬ 2. s and s’ are within 15 positions n Suppose s, s’ÎS of d(s, s’) < d and they are at least 16 positions apart in Sy. n Assume WLOG s has the smaller y-coordinate; since at most one point per box, at least 3 rows of S lying between s and s’. n d(s, s’) ³ 3d/2 > d ®¬ Divide-and-conquer 36 L d S d d/2 d/2
IRIS H.-R. JIANG Combining the Solutions (1/4) ¨ Find closest pair with one point in each side. ¤ L = {(x, y): x = x* = x-coordinate of the rightmost point in Q}. ¤ d = the smaller one of these two pairs. ¨ Observation: only need to consider points within d of line L. ¤ Q: Why? Divide-and-conquer 33 d = min(12, 21) L 21 12 d IRIS H.-R. JIANG Combining the Solutions (2/4) ¨ If $ qÎQ and rÎR for which d(q, r) < d, then each of q and r lies within a distance d of L. ¨ Pf: Suppose such q and r exist; let q = (qx, qy) and r = (rx, ry). ¤ By definition, qx £ x* £ rx. ¤ x* - qx £ rx - qx £ d(q, r) < d; rx - x* £ rx - qx £ d(q, r) < d. Divide-and-conquer 34 d = min(d(q*0, q*1), d(r*0, r*1)) L d(r*0, r*1) d(q*0, q*1) d Q R q r L: x = x* IRIS H.-R. JIANG Combining the Solutions (3/4) ¨ Observation: only need to consider points within d of line L. ¤ Sort points in 2d-strip by their y-coordinate. ¤ Only check distances of those within 15 positions in sorted list of y-coordinates! n Q: Why? Divide-and-conquer 35 d = min(12, 21) L 21 12 d 6 5 4 3 2 1 IRIS H.-R. JIANG Combining the Solutions (4/4) ¨ If s, s’ÎS are of d(s, s’) < d, then s and s’ are within 15 positions of each other in the sorted list Sy of y-coordinates of S. ¨ Pf: S contains all points within d of line L, we partition S into boxes, each box contains at most one point. ¤ Partition the region into boxes, each of area d/2*d/2; we claim 1. s and s’ lies in different boxes n Suppose s and s’ lies in the same box n These two points both belong either to Q or to R n d(s, s’) £ 0.5*d*(2)1/2 < d ®¬ 2. s and s’ are within 15 positions n Suppose s, s’ÎS of d(s, s’) < d and they are at least 16 positions apart in Sy. n Assume WLOG s has the smaller y-coordinate; since at most one point per box, at least 3 rows of S lying between s and s’. n d(s, s’) ³ 3d/2 > d ®¬ Divide-and-conquer 36 L d S d d/2 d/2
IRIS H.-R. JIANG Combining the Solutions (1/4) ¨ Find closest pair with one point in each side. ¤ L = {(x, y): x = x* = x-coordinate of the rightmost point in Q}. ¤ d = the smaller one of these two pairs. ¨ Observation: only need to consider points within d of line L. ¤ Q: Why? Divide-and-conquer 33 d = min(12, 21) L 21 12 d IRIS H.-R. JIANG Combining the Solutions (2/4) ¨ If $ qÎQ and rÎR for which d(q, r) < d, then each of q and r lies within a distance d of L. ¨ Pf: Suppose such q and r exist; let q = (qx, qy) and r = (rx, ry). ¤ By definition, qx £ x* £ rx. ¤ x* - qx £ rx - qx £ d(q, r) < d; rx - x* £ rx - qx £ d(q, r) < d. Divide-and-conquer 34 d = min(d(q*0, q*1), d(r*0, r*1)) L d(r*0, r*1) d(q*0, q*1) d Q R q r L: x = x* IRIS H.-R. JIANG Combining the Solutions (3/4) ¨ Observation: only need to consider points within d of line L. ¤ Sort points in 2d-strip by their y-coordinate. ¤ Only check distances of those within 15 positions in sorted list of y-coordinates! n Q: Why? Divide-and-conquer 35 d = min(12, 21) L 21 12 d 6 5 4 3 2 1 IRIS H.-R. JIANG Combining the Solutions (4/4) ¨ If s, s’ÎS are of d(s, s’) < d, then s and s’ are within 15 positions of each other in the sorted list Sy of y-coordinates of S. ¨ Pf: S contains all points within d of line L, we partition S into boxes, each box contains at most one point. ¤ Partition the region into boxes, each of area d/2*d/2; we claim 1. s and s’ lies in different boxes n Suppose s and s’ lies in the same box n These two points both belong either to Q or to R n d(s, s’) £ 0.5*d*(2)1/2 < d ®¬ 2. s and s’ are within 15 positions n Suppose s, s’ÎS of d(s, s’) < d and they are at least 16 positions apart in Sy. n Assume WLOG s has the smaller y-coordinate; since at most one point per box, at least 3 rows of S lying between s and s’. n d(s, s’) ³ 3d/2 > d ®¬ Divide-and-conquer 36 L d S d d/2 d/2
IRIS H.-R. JIANG Combining the Solutions (1/4) ¨ Find closest pair with one point in each side. ¤ L = {(x, y): x = x* = x-coordinate of the rightmost point in Q}. ¤ d = the smaller one of these two pairs. ¨ Observation: only need to consider points within d of line L. ¤ Q: Why? Divide-and-conquer 33 d = min(12, 21) L 21 12 d IRIS H.-R. JIANG Combining the Solutions (2/4) ¨ If $ qÎQ and rÎR for which d(q, r) < d, then each of q and r lies within a distance d of L. ¨ Pf: Suppose such q and r exist; let q = (qx, qy) and r = (rx, ry). ¤ By definition, qx £ x* £ rx. ¤ x* - qx £ rx - qx £ d(q, r) < d; rx - x* £ rx - qx £ d(q, r) < d. Divide-and-conquer 34 d = min(d(q*0, q*1), d(r*0, r*1)) L d(r*0, r*1) d(q*0, q*1) d Q R q r L: x = x* IRIS H.-R. JIANG Combining the Solutions (3/4) ¨ Observation: only need to consider points within d of line L. ¤ Sort points in 2d-strip by their y-coordinate. ¤ Only check distances of those within 15 positions in sorted list of y-coordinates! n Q: Why? Divide-and-conquer 35 d = min(12, 21) L 21 12 d 6 5 4 3 2 1 IRIS H.-R. JIANG Combining the Solutions (4/4) ¨ If s, s’ÎS are of d(s, s’) < d, then s and s’ are within 15 positions of each other in the sorted list Sy of y-coordinates of S. ¨ Pf: S contains all points within d of line L, we partition S into boxes, each box contains at most one point. ¤ Partition the region into boxes, each of area d/2*d/2; we claim 1. s and s’ lies in different boxes n Suppose s and s’ lies in the same box n These two points both belong either to Q or to R n d(s, s’) £ 0.5*d*(2)1/2 < d ®¬ 2. s and s’ are within 15 positions n Suppose s, s’ÎS of d(s, s’) < d and they are at least 16 positions apart in Sy. n Assume WLOG s has the smaller y-coordinate; since at most one point per box, at least 3 rows of S lying between s and s’. n d(s, s’) ³ 3d/2 > d ®¬ Divide-and-conquer 36 L d S d d/2 d/2
IRIS H.-R. JIANG Closest Pair Algorithm Divide-and-conquer 37 Closest-Pair-Rec(Px, Py) 1. if |P| £ 3 then return closest pair measured by all pair-wise distances 2. x* = (én/2ù)-th smallest x-coordinate in Px 3. construct Qx, Qy, Rx, Ry // O(n) 4. (q*0, q*1) = Closest-Pair-Rec(Qx, Qy) // T(n/2) 5. (r*0, r*1) = Closest-Pair-Rec(Rx, Ry) // T(n/2) 6. d = min(d(q*0, q*1), d(r*0, r*1)) 7. L = {(x, y): x = x*}; S = {points in P within distance d of L} 8. construct Sy // O(n) 9. for each sÎS do 10. compute distance from s to each of next 15 points in Sy 11.d(s, s’) = min distance of all computed distances // O(n) 12.if d(s, s’) < d then return (s, s’) 13.else if d(q*0, q*1) < d(r*0, r*1) then return (q*0, q*1) 14.else return (r*0, r*1) Closest-Pair(P) 1. construct Px and Py // O(n log n) 2. (p*0, p*1) = Closest-Pair-Rec(Px, Py) // T(n) Closest-Pair-Rec(Px, Py) 1. if |P| £ 3 then return closest pair measured by all pair-wise distances 2. x* = (én/2ù)-th smallest x-coordinate in Px 3. construct Qx, Qy, Rx, Ry 4. (q*0, q*1) = Closest-Pair-Rec(Qx, Qy) 5. (r*0, r*1) = Closest-Pair-Rec(Rx, Ry) 6. d = min(d(q*0, q*1), d(r*0, r*1)) 7. L = {(x, y): x = x*}; S = {points in P within distance d of L} 8. construct Sy 9. for each sÎS do 10. compute distance from s to each of next 15 points in Sy 11.d(s, s’) = min distance of all computed distances 12.if d(s, s’) < d then return (s, s’) 13.else if d(q*0, q*1) < d(r*0, r*1) then return (q*0, q*1) 14.else return (r*0, r*1) Closest-Pair(P) 1. construct Px and Py 2. (p*0, p*1) = Closest-Pair-Rec(Px, Py)

More Related Content

PPTX
Lecture 3_Methods of Proofs (1) Ghana Communication Technology
christianaboagye1
 
PDF
Induction.pdf
pubggaming58982
 
PPTX
Chapter5.pptx
sneha510051
 
PPTX
Week 4 Mathematical Induction Mathematical Induction
YuniartaBasani1
 
PPTX
11-Induction CIIT.pptx
jaffarbikat
 
PPT
L05 Mathematical Induction in Discrete Mathmatics.ppt
gbikorno
 
PPTX
Ch-5 Mathematical Induction_Discrete_Structure.pptx
junaidssc
 
PDF
Chapter 1_Sets. Operation sets, Principle of inclusion and exclusion etc.
joelyndagohoy
 
Lecture 3_Methods of Proofs (1) Ghana Communication Technology
christianaboagye1
 
Induction.pdf
pubggaming58982
 
Chapter5.pptx
sneha510051
 
Week 4 Mathematical Induction Mathematical Induction
YuniartaBasani1
 
11-Induction CIIT.pptx
jaffarbikat
 
L05 Mathematical Induction in Discrete Mathmatics.ppt
gbikorno
 
Ch-5 Mathematical Induction_Discrete_Structure.pptx
junaidssc
 
Chapter 1_Sets. Operation sets, Principle of inclusion and exclusion etc.
joelyndagohoy
 

Similar to algorithm_5divide_and_conquer.pdf (20)

PPTX
1-Theory of Computation_formal proofs.pptx
FathimaAmana1
 
PDF
Mathematical Induction A Powerful And Elegant Method Of Proof Titu Andreescu
jhmyisca
 
PPTX
LEC 2 ORal patholgy chemistry by nhy.pptx
SamanArshad11
 
PPT
Tma2033 chap1.1&1.2handout
shiqinrino
 
PDF
Understanding Mathematical Proof
Sasanko Sekhar Gantayat
 
PDF
Taylor john garnier_ rowan-understanding mathematical proof-taylor &amp; fr...
Vidi Al Imami
 
PPT
countablesets Sequences summations proof by induction lecture.ppt
Zaina Zaki
 
PPT
Methods_of_Proof In Discrete Mathematics
ssuserb399fd
 
PPTX
Discrete_Mathematics 8 math induction.pptx
RadwaEssam21
 
PDF
Math induction principle (slides)
IIUM
 
PPTX
UNIT III.pptx
UmeshReddy49
 
PPTX
Discrete Mathematics With Calculus and Derivation
DrAsifKhan10
 
PPTX
Properties-of-divisibility-2-LYNE-CHE-MANDIGAL.pptx
SasmerAsejo
 
PPT
04-Induction and Recursion.ppt ppt bai tap
15LThThuHuyn
 
PPTX
Proving techniques for discrete mathematics
ankurgupta857016
 
PPT
MATHEMATICAL INDUCTION.ppt
Marjorie Malveda
 
PPT
ppt02.ppt
MuhammadAbubakar114879
 
PPTX
Intro.pptx boolean algebra and logic gates
AhmedSamow
 
PPTX
Introtodiscteremath123456789qwertyu.pptx
jayarao21
 
PPTX
Divide and Conquer - Part 1
Amrinder Arora
 
1-Theory of Computation_formal proofs.pptx
FathimaAmana1
 
Mathematical Induction A Powerful And Elegant Method Of Proof Titu Andreescu
jhmyisca
 
LEC 2 ORal patholgy chemistry by nhy.pptx
SamanArshad11
 
Tma2033 chap1.1&1.2handout
shiqinrino
 
Understanding Mathematical Proof
Sasanko Sekhar Gantayat
 
Taylor john garnier_ rowan-understanding mathematical proof-taylor &amp; fr...
Vidi Al Imami
 
countablesets Sequences summations proof by induction lecture.ppt
Zaina Zaki
 
Methods_of_Proof In Discrete Mathematics
ssuserb399fd
 
Discrete_Mathematics 8 math induction.pptx
RadwaEssam21
 
Math induction principle (slides)
IIUM
 
UNIT III.pptx
UmeshReddy49
 
Discrete Mathematics With Calculus and Derivation
DrAsifKhan10
 
Properties-of-divisibility-2-LYNE-CHE-MANDIGAL.pptx
SasmerAsejo
 
04-Induction and Recursion.ppt ppt bai tap
15LThThuHuyn
 
Proving techniques for discrete mathematics
ankurgupta857016
 
MATHEMATICAL INDUCTION.ppt
Marjorie Malveda
 
ppt02.ppt
MuhammadAbubakar114879
 
Intro.pptx boolean algebra and logic gates
AhmedSamow
 
Introtodiscteremath123456789qwertyu.pptx
jayarao21
 
Divide and Conquer - Part 1
Amrinder Arora
 
Ad

More from HsuChi Chen (14)

PDF
algorithm_6dynamic_programming.pdf
HsuChi Chen
 
PDF
algorithm_1introdunction.pdf
HsuChi Chen
 
PDF
algorithm_9linear_programming.pdf
HsuChi Chen
 
PDF
algorithm_4greedy_algorithms.pdf
HsuChi Chen
 
PDF
algorithm_3graphs.pdf
HsuChi Chen
 
PDF
algorithm_7network_flow.pdf
HsuChi Chen
 
PDF
algorithm_2algorithm_analysis.pdf
HsuChi Chen
 
PDF
algorithm_8beyond_polynomial_running_times.pdf
HsuChi Chen
 
PDF
algorithm_0introdunction.pdf
HsuChi Chen
 
PDF
期末專題報告書
HsuChi Chen
 
PPTX
單晶片期末專題-報告二
HsuChi Chen
 
PPTX
單晶片期末專題-報告一
HsuChi Chen
 
PPTX
模組化課程生醫微製程期末報告
HsuChi Chen
 
PPTX
單晶片期末專題-初步想法
HsuChi Chen
 
algorithm_6dynamic_programming.pdf
HsuChi Chen
 
algorithm_1introdunction.pdf
HsuChi Chen
 
algorithm_9linear_programming.pdf
HsuChi Chen
 
algorithm_4greedy_algorithms.pdf
HsuChi Chen
 
algorithm_3graphs.pdf
HsuChi Chen
 
algorithm_7network_flow.pdf
HsuChi Chen
 
algorithm_2algorithm_analysis.pdf
HsuChi Chen
 
algorithm_8beyond_polynomial_running_times.pdf
HsuChi Chen
 
algorithm_0introdunction.pdf
HsuChi Chen
 
期末專題報告書
HsuChi Chen
 
單晶片期末專題-報告二
HsuChi Chen
 
單晶片期末專題-報告一
HsuChi Chen
 
模組化課程生醫微製程期末報告
HsuChi Chen
 
單晶片期末專題-初步想法
HsuChi Chen
 
Ad

Recently uploaded (20)

PDF
The Dynamic Duo Transforming Financial Accounting Systems Through Modern Expe...
Triforce Global
 
PDF
E-Commerce Website Development Companyin india
Digital India
 
PPTX
GSA Content Generator Crack (2025 Latest)
halimaanam8
 
PDF
AI/ML Infra Meetup | Beyond S3's Basics: Architecting for AI-Native Data Access
Alluxio, Inc.
 
PPTX
Plex Media Server 1.28.2.6151 With Crac5 2022 Free .
cariaelif911
 
PDF
AI-Powered Fuzz Testing: The Future of QA
Shubham Joshi
 
PDF
EaseUS PDF Editor Pro 6.2.0.2 Crack with License Key 2025
halimaanam8
 
PPTX
Full-Stack Developer Courses That Actually Land You Jobs
Ask the C Expert
 
PDF
Topaz Photo AI Crack New Download (Latest 2025)
hashmi66g66
 
PDF
Top 10 Software Development Trends to Watch in 2025 🚀.pdf
KodekX
 
PPTX
Python is a high-level, interpreted programming language
kesavansa2002
 
PPTX
WiFi Honeypot Detecscfddssdffsedfseztor.pptx
singlesrihari
 
PPTX
Matchmaking for JVMs: How to Pick the Perfect GC Partner
Tier1 app
 
PDF
Wondershare Recoverit Full Crack New Version (Latest 2025)
hashmi66g66
 
PPTX
Download Adobe Photoshop Crack 2025 Free
kokilasc86
 
PDF
Practical Indispensable Project Management Tips for Delivering Successful Exp...
Kandarp Bhatt
 
PPTX
Lecture 5 Software Requirement Engineering
Huda Nasir
 
DOCX
How to Use SharePoint as an ISO-Compliant Document Management System
Sharepoint Designs
 
PPTX
Introduction to Windows Operating System
ssuser1028f8
 
PDF
PDF-XChange Editor Plus 10.7.0.398.0 Crack Free Download Latest 2025
imang66g
 
The Dynamic Duo Transforming Financial Accounting Systems Through Modern Expe...
Triforce Global
 
E-Commerce Website Development Companyin india
Digital India
 
GSA Content Generator Crack (2025 Latest)
halimaanam8
 
AI/ML Infra Meetup | Beyond S3's Basics: Architecting for AI-Native Data Access
Alluxio, Inc.
 
Plex Media Server 1.28.2.6151 With Crac5 2022 Free .
cariaelif911
 
AI-Powered Fuzz Testing: The Future of QA
Shubham Joshi
 
EaseUS PDF Editor Pro 6.2.0.2 Crack with License Key 2025
halimaanam8
 
Full-Stack Developer Courses That Actually Land You Jobs
Ask the C Expert
 
Topaz Photo AI Crack New Download (Latest 2025)
hashmi66g66
 
Top 10 Software Development Trends to Watch in 2025 🚀.pdf
KodekX
 
Python is a high-level, interpreted programming language
kesavansa2002
 
WiFi Honeypot Detecscfddssdffsedfseztor.pptx
singlesrihari
 
Matchmaking for JVMs: How to Pick the Perfect GC Partner
Tier1 app
 
Wondershare Recoverit Full Crack New Version (Latest 2025)
hashmi66g66
 
Download Adobe Photoshop Crack 2025 Free
kokilasc86
 
Practical Indispensable Project Management Tips for Delivering Successful Exp...
Kandarp Bhatt
 
Lecture 5 Software Requirement Engineering
Huda Nasir
 
How to Use SharePoint as an ISO-Compliant Document Management System
Sharepoint Designs
 
Introduction to Windows Operating System
ssuser1028f8
 
PDF-XChange Editor Plus 10.7.0.398.0 Crack Free Download Latest 2025
imang66g
 

algorithm_5divide_and_conquer.pdf

  • 1. Iris Hui-Ru Jiang Fall 2014 CHAPTER 5 DIVIDE AND CONQUER Preliminaries: Mathematical Induction 2 Divide-and-conquer IRIS H.-R. JIANG Weak Induction ¨ Given the propositional P(n) where nÎ À, a proof by mathematical induction is of the form: ¤ Basis step: The proposition P(0) is shown to be true ¤ Inductive step: The implication P(k) ®P(k + 1) is shown to be true for every kÎ À n In the inductive step, the statement P(k) is called the inductive hypothesis Divide-and-conquer 3 IRIS H.-R. JIANG Strong Induction ¨ Given the propositional P(n) where nÎ À, a proof by second principle of mathematical induction (or strong induction) is of the form: ¤ Basis step: The proposition P(0) is shown to be true ¤ Inductive step: The implication P(0) ÙP(1) Ù … Ù P(k) ®P(k + 1) is shown to be true for every kÎ À Divide-and-conquer 4
  • 2. Iris Hui-Ru Jiang Fall 2014 CHAPTER 5 DIVIDE AND CONQUER Preliminaries: Mathematical Induction 2 Divide-and-conquer IRIS H.-R. JIANG Weak Induction ¨ Given the propositional P(n) where nÎ À, a proof by mathematical induction is of the form: ¤ Basis step: The proposition P(0) is shown to be true ¤ Inductive step: The implication P(k) ®P(k + 1) is shown to be true for every kÎ À n In the inductive step, the statement P(k) is called the inductive hypothesis Divide-and-conquer 3 IRIS H.-R. JIANG Strong Induction ¨ Given the propositional P(n) where nÎ À, a proof by second principle of mathematical induction (or strong induction) is of the form: ¤ Basis step: The proposition P(0) is shown to be true ¤ Inductive step: The implication P(0) ÙP(1) Ù … Ù P(k) ®P(k + 1) is shown to be true for every kÎ À Divide-and-conquer 4
  • 3. Iris Hui-Ru Jiang Fall 2014 CHAPTER 5 DIVIDE AND CONQUER Preliminaries: Mathematical Induction 2 Divide-and-conquer IRIS H.-R. JIANG Weak Induction ¨ Given the propositional P(n) where nÎ À, a proof by mathematical induction is of the form: ¤ Basis step: The proposition P(0) is shown to be true ¤ Inductive step: The implication P(k) ®P(k + 1) is shown to be true for every kÎ À n In the inductive step, the statement P(k) is called the inductive hypothesis Divide-and-conquer 3 IRIS H.-R. JIANG Strong Induction ¨ Given the propositional P(n) where nÎ À, a proof by second principle of mathematical induction (or strong induction) is of the form: ¤ Basis step: The proposition P(0) is shown to be true ¤ Inductive step: The implication P(0) ÙP(1) Ù … Ù P(k) ®P(k + 1) is shown to be true for every kÎ À Divide-and-conquer 4
  • 4. Iris Hui-Ru Jiang Fall 2014 CHAPTER 5 DIVIDE AND CONQUER Preliminaries: Mathematical Induction 2 Divide-and-conquer IRIS H.-R. JIANG Weak Induction ¨ Given the propositional P(n) where nÎ À, a proof by mathematical induction is of the form: ¤ Basis step: The proposition P(0) is shown to be true ¤ Inductive step: The implication P(k) ®P(k + 1) is shown to be true for every kÎ À n In the inductive step, the statement P(k) is called the inductive hypothesis Divide-and-conquer 3 IRIS H.-R. JIANG Strong Induction ¨ Given the propositional P(n) where nÎ À, a proof by second principle of mathematical induction (or strong induction) is of the form: ¤ Basis step: The proposition P(0) is shown to be true ¤ Inductive step: The implication P(0) ÙP(1) Ù … Ù P(k) ®P(k + 1) is shown to be true for every kÎ À Divide-and-conquer 4
  • 5. IRIS H.-R. JIANG Example: A Defective Chessboard ¨ Any 8Ŝ8 defective chessboard can be covered with twenty-one triominoes ¨ Q: How? Divide-and-conquer 5 Triomino IRIS H.-R. JIANG Example: A Defective Chessboard ¨ Any 8Ŝ8 defective chessboard can be covered with twenty-one triominoes ¨ Any 2nŜ2n defective chessboard can be covered with 1/3(2nŜ2n- 1) triominoes ¨ Prove by mathematical induction! Divide-and-conquer 6 IRIS H.-R. JIANG Mathematical Induction ¨ The first domino falls. ¨ If a domino falls, so will the next domino. ¨ All dominoes will fall! Divide-and-conquer 7 Courtesy of Prof. C.L. Liu 1 2 3 4 5 IRIS H.-R. JIANG Proof by Mathematical Induction ¨ Any 2nŜ2n defective chessboard can be covered with 1/3(2nŜ2n- 1) triominoes ¤ Basis step: n n=1 ¤ Inductive step: Divide-and-conquer 8 Triomino 2n+1 2n+1 2n 2n 2n 2n
  • 6. IRIS H.-R. JIANG Example: A Defective Chessboard ¨ Any 8Ŝ8 defective chessboard can be covered with twenty-one triominoes ¨ Q: How? Divide-and-conquer 5 Triomino IRIS H.-R. JIANG Example: A Defective Chessboard ¨ Any 8Ŝ8 defective chessboard can be covered with twenty-one triominoes ¨ Any 2nŜ2n defective chessboard can be covered with 1/3(2nŜ2n- 1) triominoes ¨ Prove by mathematical induction! Divide-and-conquer 6 IRIS H.-R. JIANG Mathematical Induction ¨ The first domino falls. ¨ If a domino falls, so will the next domino. ¨ All dominoes will fall! Divide-and-conquer 7 Courtesy of Prof. C.L. Liu 1 2 3 4 5 IRIS H.-R. JIANG Proof by Mathematical Induction ¨ Any 2nŜ2n defective chessboard can be covered with 1/3(2nŜ2n- 1) triominoes ¤ Basis step: n n=1 ¤ Inductive step: Divide-and-conquer 8 Triomino 2n+1 2n+1 2n 2n 2n 2n
  • 7. IRIS H.-R. JIANG Example: A Defective Chessboard ¨ Any 8Ŝ8 defective chessboard can be covered with twenty-one triominoes ¨ Q: How? Divide-and-conquer 5 Triomino IRIS H.-R. JIANG Example: A Defective Chessboard ¨ Any 8Ŝ8 defective chessboard can be covered with twenty-one triominoes ¨ Any 2nŜ2n defective chessboard can be covered with 1/3(2nŜ2n- 1) triominoes ¨ Prove by mathematical induction! Divide-and-conquer 6 IRIS H.-R. JIANG Mathematical Induction ¨ The first domino falls. ¨ If a domino falls, so will the next domino. ¨ All dominoes will fall! Divide-and-conquer 7 Courtesy of Prof. C.L. Liu 1 2 3 4 5 IRIS H.-R. JIANG Proof by Mathematical Induction ¨ Any 2nŜ2n defective chessboard can be covered with 1/3(2nŜ2n- 1) triominoes ¤ Basis step: n n=1 ¤ Inductive step: Divide-and-conquer 8 Triomino 2n+1 2n+1 2n 2n 2n 2n
  • 8. IRIS H.-R. JIANG Example: A Defective Chessboard ¨ Any 8Ŝ8 defective chessboard can be covered with twenty-one triominoes ¨ Q: How? Divide-and-conquer 5 Triomino IRIS H.-R. JIANG Example: A Defective Chessboard ¨ Any 8Ŝ8 defective chessboard can be covered with twenty-one triominoes ¨ Any 2nŜ2n defective chessboard can be covered with 1/3(2nŜ2n- 1) triominoes ¨ Prove by mathematical induction! Divide-and-conquer 6 IRIS H.-R. JIANG Mathematical Induction ¨ The first domino falls. ¨ If a domino falls, so will the next domino. ¨ All dominoes will fall! Divide-and-conquer 7 Courtesy of Prof. C.L. Liu 1 2 3 4 5 IRIS H.-R. JIANG Proof by Mathematical Induction ¨ Any 2nŜ2n defective chessboard can be covered with 1/3(2nŜ2n- 1) triominoes ¤ Basis step: n n=1 ¤ Inductive step: Divide-and-conquer 8 Triomino 2n+1 2n+1 2n 2n 2n 2n
  • 9. IRIS H.-R. JIANG Proof vs. Algorithm ¨ Based on the defective chessboard, we can see Divide-and-conquer 9 Algorithm Proof IRIS H.-R. JIANG Outline ¨ Content: ¤ A first recurrence: the mergesort algorithm ¤ Counting inversions ¤ Finding the closest pair of points ¨ Reading: ¤ Chapter 5 Divide-and-conquer 10 IRIS H.-R. JIANG Warm Up: Searching ¨ Problem: Searching ¨ Given ¤ A sorted list of n distinct integers ¤ integer x ¨ Find ¤ j if x equals some integer of index j ¨ Solution: ¤ Naïve idea: compare one by one n Correct but slow: O(n) ¤ Better idea? n Hint: input is sorted Divide-and-conquer 11 Use known information to improve your solution IRIS H.-R. JIANG Binary Search ¨ Divide: Break the input into several parts of the same type. ¨ Conquer: Solve the problem in each part recursively. ¨ Combine: Combine the solutions to sub-problems into an overall solution ¨ Check the middle element ¨ Search the subarray recursively Divide-and-conquer paradigm Binary search on a sorted array 12 Divide-and-conquer 98 81 72 68 55 41 22 19 13 5 0 55 98 81 72 68 55 55 68 55 55 < 41 > 72 = 55 ¨ Divide: check the middle element ¨ Conquer: search the subarray recursively ¨ Combine: trivial
  • 10. IRIS H.-R. JIANG Proof vs. Algorithm ¨ Based on the defective chessboard, we can see Divide-and-conquer 9 Algorithm Proof IRIS H.-R. JIANG Outline ¨ Content: ¤ A first recurrence: the mergesort algorithm ¤ Counting inversions ¤ Finding the closest pair of points ¨ Reading: ¤ Chapter 5 Divide-and-conquer 10 IRIS H.-R. JIANG Warm Up: Searching ¨ Problem: Searching ¨ Given ¤ A sorted list of n distinct integers ¤ integer x ¨ Find ¤ j if x equals some integer of index j ¨ Solution: ¤ Naïve idea: compare one by one n Correct but slow: O(n) ¤ Better idea? n Hint: input is sorted Divide-and-conquer 11 Use known information to improve your solution IRIS H.-R. JIANG Binary Search ¨ Divide: Break the input into several parts of the same type. ¨ Conquer: Solve the problem in each part recursively. ¨ Combine: Combine the solutions to sub-problems into an overall solution ¨ Check the middle element ¨ Search the subarray recursively Divide-and-conquer paradigm Binary search on a sorted array 12 Divide-and-conquer 98 81 72 68 55 41 22 19 13 5 0 55 98 81 72 68 55 55 68 55 55 < 41 > 72 = 55 ¨ Divide: check the middle element ¨ Conquer: search the subarray recursively ¨ Combine: trivial
  • 11. IRIS H.-R. JIANG Proof vs. Algorithm ¨ Based on the defective chessboard, we can see Divide-and-conquer 9 Algorithm Proof IRIS H.-R. JIANG Outline ¨ Content: ¤ A first recurrence: the mergesort algorithm ¤ Counting inversions ¤ Finding the closest pair of points ¨ Reading: ¤ Chapter 5 Divide-and-conquer 10 IRIS H.-R. JIANG Warm Up: Searching ¨ Problem: Searching ¨ Given ¤ A sorted list of n distinct integers ¤ integer x ¨ Find ¤ j if x equals some integer of index j ¨ Solution: ¤ Naïve idea: compare one by one n Correct but slow: O(n) ¤ Better idea? n Hint: input is sorted Divide-and-conquer 11 Use known information to improve your solution IRIS H.-R. JIANG Binary Search ¨ Divide: Break the input into several parts of the same type. ¨ Conquer: Solve the problem in each part recursively. ¨ Combine: Combine the solutions to sub-problems into an overall solution ¨ Check the middle element ¨ Search the subarray recursively Divide-and-conquer paradigm Binary search on a sorted array 12 Divide-and-conquer 98 81 72 68 55 41 22 19 13 5 0 55 98 81 72 68 55 55 68 55 55 < 41 > 72 = 55 ¨ Divide: check the middle element ¨ Conquer: search the subarray recursively ¨ Combine: trivial
  • 12. IRIS H.-R. JIANG Proof vs. Algorithm ¨ Based on the defective chessboard, we can see Divide-and-conquer 9 Algorithm Proof IRIS H.-R. JIANG Outline ¨ Content: ¤ A first recurrence: the mergesort algorithm ¤ Counting inversions ¤ Finding the closest pair of points ¨ Reading: ¤ Chapter 5 Divide-and-conquer 10 IRIS H.-R. JIANG Warm Up: Searching ¨ Problem: Searching ¨ Given ¤ A sorted list of n distinct integers ¤ integer x ¨ Find ¤ j if x equals some integer of index j ¨ Solution: ¤ Naïve idea: compare one by one n Correct but slow: O(n) ¤ Better idea? n Hint: input is sorted Divide-and-conquer 11 Use known information to improve your solution IRIS H.-R. JIANG Binary Search ¨ Divide: Break the input into several parts of the same type. ¨ Conquer: Solve the problem in each part recursively. ¨ Combine: Combine the solutions to sub-problems into an overall solution ¨ Check the middle element ¨ Search the subarray recursively Divide-and-conquer paradigm Binary search on a sorted array 12 Divide-and-conquer 98 81 72 68 55 41 22 19 13 5 0 55 98 81 72 68 55 55 68 55 55 < 41 > 72 = 55 ¨ Divide: check the middle element ¨ Conquer: search the subarray recursively ¨ Combine: trivial
  • 13. John von Neumann, 1945 Merge sort Mergesort 13 Divide-and-conquer http://en.wikipedia.org/wiki/File:Merge_sort_animation2.gif value index IRIS H.-R. JIANG Divide and Conquer ¨ Divide-and-conquer ¤ Divide: Break the input into several parts of the same type. ¤ Conquer: Solve the problem in each part recursively. ¤ Combine: Combine the solutions to sub-problems into an overall solution. ¨ Complexity: recurrence relation ¤ A divide and conquer algorithm is naturally implemented by a recursive procedure. ¤ The running time of a divide and conquer algorithm is generally represented by a recurrence relation that bounds the running time recursively in terms of the running time on smaller instances. ¨ Correctness: mathematical induction ¤ The basic idea is mathematical induction! Divide-and-conquer 14 IRIS H.-R. JIANG A Divide-and-Conquer Template ¨ A divide-and-conquer template ¤ Divide: divide the input into two pieces of equal size ¤ Conquer: solve the two subproblems on these pieces separately by recursion ¤ Combine: combine the two results into an overall solution ¤ Spend only linear time for the initial division and final recombining Divide-and-conquer 15 IRIS H.-R. JIANG Mergesort (1/2) ¨ Problem: Sorting ¨ Given ¤ A set of n numbers ¨ Find ¤ Sorted list in ascending order ¨ Solution: many! ¨ Mergesort fits the divide-and-conquer template ¤ Divide the input into two halves. ¤ Sort each half recursively. n Need base case ¤ Merge two halves into one. Divide-and-conquer 16 A L G O R I T H M S Stop recursion I T H M S A L G O R H I M S T A G L O R A G H I L M O R S T
  • 14. John von Neumann, 1945 Merge sort Mergesort 13 Divide-and-conquer http://en.wikipedia.org/wiki/File:Merge_sort_animation2.gif value index IRIS H.-R. JIANG Divide and Conquer ¨ Divide-and-conquer ¤ Divide: Break the input into several parts of the same type. ¤ Conquer: Solve the problem in each part recursively. ¤ Combine: Combine the solutions to sub-problems into an overall solution. ¨ Complexity: recurrence relation ¤ A divide and conquer algorithm is naturally implemented by a recursive procedure. ¤ The running time of a divide and conquer algorithm is generally represented by a recurrence relation that bounds the running time recursively in terms of the running time on smaller instances. ¨ Correctness: mathematical induction ¤ The basic idea is mathematical induction! Divide-and-conquer 14 IRIS H.-R. JIANG A Divide-and-Conquer Template ¨ A divide-and-conquer template ¤ Divide: divide the input into two pieces of equal size ¤ Conquer: solve the two subproblems on these pieces separately by recursion ¤ Combine: combine the two results into an overall solution ¤ Spend only linear time for the initial division and final recombining Divide-and-conquer 15 IRIS H.-R. JIANG Mergesort (1/2) ¨ Problem: Sorting ¨ Given ¤ A set of n numbers ¨ Find ¤ Sorted list in ascending order ¨ Solution: many! ¨ Mergesort fits the divide-and-conquer template ¤ Divide the input into two halves. ¤ Sort each half recursively. n Need base case ¤ Merge two halves into one. Divide-and-conquer 16 A L G O R I T H M S Stop recursion I T H M S A L G O R H I M S T A G L O R A G H I L M O R S T
  • 15. John von Neumann, 1945 Merge sort Mergesort 13 Divide-and-conquer http://en.wikipedia.org/wiki/File:Merge_sort_animation2.gif value index IRIS H.-R. JIANG Divide and Conquer ¨ Divide-and-conquer ¤ Divide: Break the input into several parts of the same type. ¤ Conquer: Solve the problem in each part recursively. ¤ Combine: Combine the solutions to sub-problems into an overall solution. ¨ Complexity: recurrence relation ¤ A divide and conquer algorithm is naturally implemented by a recursive procedure. ¤ The running time of a divide and conquer algorithm is generally represented by a recurrence relation that bounds the running time recursively in terms of the running time on smaller instances. ¨ Correctness: mathematical induction ¤ The basic idea is mathematical induction! Divide-and-conquer 14 IRIS H.-R. JIANG A Divide-and-Conquer Template ¨ A divide-and-conquer template ¤ Divide: divide the input into two pieces of equal size ¤ Conquer: solve the two subproblems on these pieces separately by recursion ¤ Combine: combine the two results into an overall solution ¤ Spend only linear time for the initial division and final recombining Divide-and-conquer 15 IRIS H.-R. JIANG Mergesort (1/2) ¨ Problem: Sorting ¨ Given ¤ A set of n numbers ¨ Find ¤ Sorted list in ascending order ¨ Solution: many! ¨ Mergesort fits the divide-and-conquer template ¤ Divide the input into two halves. ¤ Sort each half recursively. n Need base case ¤ Merge two halves into one. Divide-and-conquer 16 A L G O R I T H M S Stop recursion I T H M S A L G O R H I M S T A G L O R A G H I L M O R S T
  • 16. John von Neumann, 1945 Merge sort Mergesort 13 Divide-and-conquer http://en.wikipedia.org/wiki/File:Merge_sort_animation2.gif value index IRIS H.-R. JIANG Divide and Conquer ¨ Divide-and-conquer ¤ Divide: Break the input into several parts of the same type. ¤ Conquer: Solve the problem in each part recursively. ¤ Combine: Combine the solutions to sub-problems into an overall solution. ¨ Complexity: recurrence relation ¤ A divide and conquer algorithm is naturally implemented by a recursive procedure. ¤ The running time of a divide and conquer algorithm is generally represented by a recurrence relation that bounds the running time recursively in terms of the running time on smaller instances. ¨ Correctness: mathematical induction ¤ The basic idea is mathematical induction! Divide-and-conquer 14 IRIS H.-R. JIANG A Divide-and-Conquer Template ¨ A divide-and-conquer template ¤ Divide: divide the input into two pieces of equal size ¤ Conquer: solve the two subproblems on these pieces separately by recursion ¤ Combine: combine the two results into an overall solution ¤ Spend only linear time for the initial division and final recombining Divide-and-conquer 15 IRIS H.-R. JIANG Mergesort (1/2) ¨ Problem: Sorting ¨ Given ¤ A set of n numbers ¨ Find ¤ Sorted list in ascending order ¨ Solution: many! ¨ Mergesort fits the divide-and-conquer template ¤ Divide the input into two halves. ¤ Sort each half recursively. n Need base case ¤ Merge two halves into one. Divide-and-conquer 16 A L G O R I T H M S Stop recursion I T H M S A L G O R H I M S T A G L O R A G H I L M O R S T
  • 17. IRIS H.-R. JIANG Mergesort (2/2) ¨ The base case: single element (trivially sorted) ¨ Running time: ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) ¤ Conquer: lines 3-4, 2T(n/2) ¤ Combine: line 5, C(n) ¤ T(n) = 2T(n/2) + D(n) + C(n) Divide-and-conquer 17 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) A L G O R I T H M S I T H M S A L G O R A G H I L M O R S T I T H M S A L G O R A L G I T H I T H M S A L G O R I T H A L G H I T M S A G L O R H I M S T A G L O R IRIS H.-R. JIANG Implementation: Division and Merging ¨ Running time: T(n) ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) n O(1) for array ¤ Combine: line 5, C(n) ¤ Q: Linear time O(n)? How? ¨ Efficient merging ¤ See the demonstration of Merge Divide-and-conquer 18 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) IRIS H.-R. JIANG Implementation: Division and Merging ¨ Running time: T(n) ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) n O(1) for array ¤ Combine: line 5, C(n) ¤ Q: Linear time O(n)? How? ¨ Efficient merging ¤ Linear number of comparisons ¤ Use an auxiliary array ¤ O(n) ¨ Merge sort is often the best choice for sorting a linked list ¤ Q: Why? How efficient on running time and storage? Divide-and-conquer 19 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) A G H I L H I M S T A G L O R IRIS H.-R. JIANG Recurrence Relation ¨ Running time: T(n) 1. Base case: for n £ 2, T(n) £ c 2. T(n) = 2T(n/2) + D(n) + C(n) n T(n) = 2T(n/2) + O(1) + O(n) n T(n) £ 2T(n/2) + cn ¤ A recursion corresponds to a recurrence relation n Recursion is a function defined by itself ¤ Q: Why not T(n) £ T(ën/2û) + T(én/2ù) + cn? ¤ A: The asymptotic bounds are not affected by ignoring ëû & éù. Divide-and-conquer 20 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r)
  • 18. IRIS H.-R. JIANG Mergesort (2/2) ¨ The base case: single element (trivially sorted) ¨ Running time: ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) ¤ Conquer: lines 3-4, 2T(n/2) ¤ Combine: line 5, C(n) ¤ T(n) = 2T(n/2) + D(n) + C(n) Divide-and-conquer 17 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) A L G O R I T H M S I T H M S A L G O R A G H I L M O R S T I T H M S A L G O R A L G I T H I T H M S A L G O R I T H A L G H I T M S A G L O R H I M S T A G L O R IRIS H.-R. JIANG Implementation: Division and Merging ¨ Running time: T(n) ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) n O(1) for array ¤ Combine: line 5, C(n) ¤ Q: Linear time O(n)? How? ¨ Efficient merging ¤ See the demonstration of Merge Divide-and-conquer 18 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) IRIS H.-R. JIANG Implementation: Division and Merging ¨ Running time: T(n) ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) n O(1) for array ¤ Combine: line 5, C(n) ¤ Q: Linear time O(n)? How? ¨ Efficient merging ¤ Linear number of comparisons ¤ Use an auxiliary array ¤ O(n) ¨ Merge sort is often the best choice for sorting a linked list ¤ Q: Why? How efficient on running time and storage? Divide-and-conquer 19 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) A G H I L H I M S T A G L O R IRIS H.-R. JIANG Recurrence Relation ¨ Running time: T(n) 1. Base case: for n £ 2, T(n) £ c 2. T(n) = 2T(n/2) + D(n) + C(n) n T(n) = 2T(n/2) + O(1) + O(n) n T(n) £ 2T(n/2) + cn ¤ A recursion corresponds to a recurrence relation n Recursion is a function defined by itself ¤ Q: Why not T(n) £ T(ën/2û) + T(én/2ù) + cn? ¤ A: The asymptotic bounds are not affected by ignoring ëû & éù. Divide-and-conquer 20 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r)
  • 19. IRIS H.-R. JIANG Mergesort (2/2) ¨ The base case: single element (trivially sorted) ¨ Running time: ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) ¤ Conquer: lines 3-4, 2T(n/2) ¤ Combine: line 5, C(n) ¤ T(n) = 2T(n/2) + D(n) + C(n) Divide-and-conquer 17 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) A L G O R I T H M S I T H M S A L G O R A G H I L M O R S T I T H M S A L G O R A L G I T H I T H M S A L G O R I T H A L G H I T M S A G L O R H I M S T A G L O R IRIS H.-R. JIANG Implementation: Division and Merging ¨ Running time: T(n) ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) n O(1) for array ¤ Combine: line 5, C(n) ¤ Q: Linear time O(n)? How? ¨ Efficient merging ¤ See the demonstration of Merge Divide-and-conquer 18 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) IRIS H.-R. JIANG Implementation: Division and Merging ¨ Running time: T(n) ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) n O(1) for array ¤ Combine: line 5, C(n) ¤ Q: Linear time O(n)? How? ¨ Efficient merging ¤ Linear number of comparisons ¤ Use an auxiliary array ¤ O(n) ¨ Merge sort is often the best choice for sorting a linked list ¤ Q: Why? How efficient on running time and storage? Divide-and-conquer 19 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) A G H I L H I M S T A G L O R IRIS H.-R. JIANG Recurrence Relation ¨ Running time: T(n) 1. Base case: for n £ 2, T(n) £ c 2. T(n) = 2T(n/2) + D(n) + C(n) n T(n) = 2T(n/2) + O(1) + O(n) n T(n) £ 2T(n/2) + cn ¤ A recursion corresponds to a recurrence relation n Recursion is a function defined by itself ¤ Q: Why not T(n) £ T(ën/2û) + T(én/2ù) + cn? ¤ A: The asymptotic bounds are not affected by ignoring ëû & éù. Divide-and-conquer 20 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r)
  • 20. IRIS H.-R. JIANG Mergesort (2/2) ¨ The base case: single element (trivially sorted) ¨ Running time: ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) ¤ Conquer: lines 3-4, 2T(n/2) ¤ Combine: line 5, C(n) ¤ T(n) = 2T(n/2) + D(n) + C(n) Divide-and-conquer 17 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) A L G O R I T H M S I T H M S A L G O R A G H I L M O R S T I T H M S A L G O R A L G I T H I T H M S A L G O R I T H A L G H I T M S A G L O R H I M S T A G L O R IRIS H.-R. JIANG Implementation: Division and Merging ¨ Running time: T(n) ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) n O(1) for array ¤ Combine: line 5, C(n) ¤ Q: Linear time O(n)? How? ¨ Efficient merging ¤ See the demonstration of Merge Divide-and-conquer 18 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) IRIS H.-R. JIANG Implementation: Division and Merging ¨ Running time: T(n) ¤ T(n) for input size n ¤ Divide: lines 1-2, D(n) n O(1) for array ¤ Combine: line 5, C(n) ¤ Q: Linear time O(n)? How? ¨ Efficient merging ¤ Linear number of comparisons ¤ Use an auxiliary array ¤ O(n) ¨ Merge sort is often the best choice for sorting a linked list ¤ Q: Why? How efficient on running time and storage? Divide-and-conquer 19 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r) A G H I L H I M S T A G L O R IRIS H.-R. JIANG Recurrence Relation ¨ Running time: T(n) 1. Base case: for n £ 2, T(n) £ c 2. T(n) = 2T(n/2) + D(n) + C(n) n T(n) = 2T(n/2) + O(1) + O(n) n T(n) £ 2T(n/2) + cn ¤ A recursion corresponds to a recurrence relation n Recursion is a function defined by itself ¤ Q: Why not T(n) £ T(ën/2û) + T(én/2ù) + cn? ¤ A: The asymptotic bounds are not affected by ignoring ëû & éù. Divide-and-conquer 20 Mergesort(A, p, r) // A[p..r]: initially unsorted 1. if (p < r) then 2. q = ë(p+r)/2û 3. Mergesort(A, p, q) 4. Mergesort(A, q+1, r) 5. Merge(A, p, q, r)
  • 21. IRIS H.-R. JIANG Solving Recurrences ¨ Two basic ways to solve a recurrence ¤ Unrolling the recurrence (recursion tree) ¤ Substituting a guess ¨ Initially, we assume n is a power of 2 and replace £ with =. ¤ T(n) = 2T(n/2) + cn ¤ Simplify the problem by omitting floors and ceilings ¤ Solve the worst case Divide-and-conquer 21 IRIS H.-R. JIANG Unrolling – Recursion Tree ¨ Procedure 1. Analyzing the first few levels 2. Identifying a pattern 3. Summing over all levels Divide-and-conquer 22 T(n) T(n) cn T(n/2) T(n/2) cn/2 T(n/4) T(n/4) cn/2 T(n/4) T(n/4) cn/4 cn/4 cn/4 cn/4 Lvl 0: cn Lvl 1: cn Lvl 2: cn cn/2j … Lvl j: cn … Lvl k: cn T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) … k=log2n O(n log2n) T(n) = 2T(n/2) + cn T(2) = c IRIS H.-R. JIANG Substituting ¨ Any function T(.) satisfying this recurrence T(n) £ 2T(n/2) + cn when n > 2, and T(2) £ c is bounded by O(n log2 n), when n > 1. ¨ Pf: Guess and proof by induction ¨ Suppose we believe that T(n) £ cn log2 n for all n ³ 2 1. Base case: n = 2, T(2) £ c £ 2c. Indeed true. 2. Inductive step: n Inductive hypothesis: T(m) £ cm log2 m for all m < n. n T(n/2) £ c(n/2) log2(n/2); log2(n/2) = (log2 n) – 1 n T(n) £ 2T(n/2) + cn £ 2c(n/2) log2(n/2) + cn = cn [(log2 n) – 1] + cn = (cnlog2 n) – cn + cn = cnlog2 n. Divide-and-conquer 23 assume n is a power of 2 Counting Inversions 24 Divide-and-conquer
  • 22. IRIS H.-R. JIANG Solving Recurrences ¨ Two basic ways to solve a recurrence ¤ Unrolling the recurrence (recursion tree) ¤ Substituting a guess ¨ Initially, we assume n is a power of 2 and replace £ with =. ¤ T(n) = 2T(n/2) + cn ¤ Simplify the problem by omitting floors and ceilings ¤ Solve the worst case Divide-and-conquer 21 IRIS H.-R. JIANG Unrolling – Recursion Tree ¨ Procedure 1. Analyzing the first few levels 2. Identifying a pattern 3. Summing over all levels Divide-and-conquer 22 T(n) T(n) cn T(n/2) T(n/2) cn/2 T(n/4) T(n/4) cn/2 T(n/4) T(n/4) cn/4 cn/4 cn/4 cn/4 Lvl 0: cn Lvl 1: cn Lvl 2: cn cn/2j … Lvl j: cn … Lvl k: cn T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) … k=log2n O(n log2n) T(n) = 2T(n/2) + cn T(2) = c IRIS H.-R. JIANG Substituting ¨ Any function T(.) satisfying this recurrence T(n) £ 2T(n/2) + cn when n > 2, and T(2) £ c is bounded by O(n log2 n), when n > 1. ¨ Pf: Guess and proof by induction ¨ Suppose we believe that T(n) £ cn log2 n for all n ³ 2 1. Base case: n = 2, T(2) £ c £ 2c. Indeed true. 2. Inductive step: n Inductive hypothesis: T(m) £ cm log2 m for all m < n. n T(n/2) £ c(n/2) log2(n/2); log2(n/2) = (log2 n) – 1 n T(n) £ 2T(n/2) + cn £ 2c(n/2) log2(n/2) + cn = cn [(log2 n) – 1] + cn = (cnlog2 n) – cn + cn = cnlog2 n. Divide-and-conquer 23 assume n is a power of 2 Counting Inversions 24 Divide-and-conquer
  • 23. IRIS H.-R. JIANG Solving Recurrences ¨ Two basic ways to solve a recurrence ¤ Unrolling the recurrence (recursion tree) ¤ Substituting a guess ¨ Initially, we assume n is a power of 2 and replace £ with =. ¤ T(n) = 2T(n/2) + cn ¤ Simplify the problem by omitting floors and ceilings ¤ Solve the worst case Divide-and-conquer 21 IRIS H.-R. JIANG Unrolling – Recursion Tree ¨ Procedure 1. Analyzing the first few levels 2. Identifying a pattern 3. Summing over all levels Divide-and-conquer 22 T(n) T(n) cn T(n/2) T(n/2) cn/2 T(n/4) T(n/4) cn/2 T(n/4) T(n/4) cn/4 cn/4 cn/4 cn/4 Lvl 0: cn Lvl 1: cn Lvl 2: cn cn/2j … Lvl j: cn … Lvl k: cn T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) … k=log2n O(n log2n) T(n) = 2T(n/2) + cn T(2) = c IRIS H.-R. JIANG Substituting ¨ Any function T(.) satisfying this recurrence T(n) £ 2T(n/2) + cn when n > 2, and T(2) £ c is bounded by O(n log2 n), when n > 1. ¨ Pf: Guess and proof by induction ¨ Suppose we believe that T(n) £ cn log2 n for all n ³ 2 1. Base case: n = 2, T(2) £ c £ 2c. Indeed true. 2. Inductive step: n Inductive hypothesis: T(m) £ cm log2 m for all m < n. n T(n/2) £ c(n/2) log2(n/2); log2(n/2) = (log2 n) – 1 n T(n) £ 2T(n/2) + cn £ 2c(n/2) log2(n/2) + cn = cn [(log2 n) – 1] + cn = (cnlog2 n) – cn + cn = cnlog2 n. Divide-and-conquer 23 assume n is a power of 2 Counting Inversions 24 Divide-and-conquer
  • 24. IRIS H.-R. JIANG Solving Recurrences ¨ Two basic ways to solve a recurrence ¤ Unrolling the recurrence (recursion tree) ¤ Substituting a guess ¨ Initially, we assume n is a power of 2 and replace £ with =. ¤ T(n) = 2T(n/2) + cn ¤ Simplify the problem by omitting floors and ceilings ¤ Solve the worst case Divide-and-conquer 21 IRIS H.-R. JIANG Unrolling – Recursion Tree ¨ Procedure 1. Analyzing the first few levels 2. Identifying a pattern 3. Summing over all levels Divide-and-conquer 22 T(n) T(n) cn T(n/2) T(n/2) cn/2 T(n/4) T(n/4) cn/2 T(n/4) T(n/4) cn/4 cn/4 cn/4 cn/4 Lvl 0: cn Lvl 1: cn Lvl 2: cn cn/2j … Lvl j: cn … Lvl k: cn T(2) T(2) T(2) T(2) T(2) T(2) T(2) T(2) … k=log2n O(n log2n) T(n) = 2T(n/2) + cn T(2) = c IRIS H.-R. JIANG Substituting ¨ Any function T(.) satisfying this recurrence T(n) £ 2T(n/2) + cn when n > 2, and T(2) £ c is bounded by O(n log2 n), when n > 1. ¨ Pf: Guess and proof by induction ¨ Suppose we believe that T(n) £ cn log2 n for all n ³ 2 1. Base case: n = 2, T(2) £ c £ 2c. Indeed true. 2. Inductive step: n Inductive hypothesis: T(m) £ cm log2 m for all m < n. n T(n/2) £ c(n/2) log2(n/2); log2(n/2) = (log2 n) – 1 n T(n) £ 2T(n/2) + cn £ 2c(n/2) log2(n/2) + cn = cn [(log2 n) – 1] + cn = (cnlog2 n) – cn + cn = cnlog2 n. Divide-and-conquer 23 assume n is a power of 2 Counting Inversions 24 Divide-and-conquer
  • 25. IRIS H.-R. JIANG Counting Inversions ¨ Music site tries to match your song preferences with others. ¤ You rank n songs. ¤ Music site consults database to find people with similar tastes. ¨ Similarity metric: number of inversions between two rankings. ¤ My rank: 1, 2, …, n. ¤ Your rank: a1, a2, …, an. ¤ Songs i and j inverted if i < j, but ai > aj. ¨ Brute force: check all Q(n2) pairs i and j. Divide-and-conquer 25 Me You A B C D E 1 2 3 4 5 2 4 1 3 5 1 2 3 4 5 2 4 1 3 5 Me You inversion = crossing (2, 1), (4, 1), (4, 3) IRIS H.-R. JIANG Divide and Conquer ¨ Counting inversions ¤ Divide: separate the list into two pieces. ¤ Conquer: recursively count inversions in each half. ¤ Combine: count inversions where ai and aj are in different halves, and return sum of three quantities. Divide-and-conquer 26 1 5 4 8 10 2 6 9 12 3 7 11 1 5 4 8 10 2 6 9 12 3 7 11 5 red-red inversions (5,4), (5,2), (4,2), (8,2), (10,2) 8 green-green inversions (6,3), (9,3), (9,7), (12,11), (12,3), (12,7), (11,3), (11,7) 9 red-green inversions (5,3), (4,3), (8,6), (8,3), (8,7), (10,6), (10,9), (10,3), (10,7) Combine: ?? Conquer: 2T(n/2) Divide: O(1) Total: 5 + 8 + 9 = 22 inversions IRIS H.-R. JIANG Combine? ¨ Inversions: inter and intra ¤ Intra: inversions within each half – done by conquer ¤ Inter: inversions between two halves – done by combine n The “combine” in Mergesort is done in O(n); goal: O(n log2 n) n Assume each half is sorted. n Count inversions where ai and aj are in different halves. n Merge two sorted halves into sorted whole. Divide-and-conquer 27 to maintain sorted invariant 1 2 4 5 8 10 3 6 7 11 12 9 (4,3), (5,3), (8,6), (10,3), (8,6), (10,6), (8,7), (10,7), (10,9) 9 red-green inversions Merge: O(n) Count: O(n) 4 2 2 0 0 1 1 2 3 4 5 6 7 8 9 11 12 10 Combine: O(n) Total: O(n log2 n) IRIS H.-R. JIANG Implementation: Counting Inversions ¨ Similar to Mergesort, extra effort on counting inter-inversions Divide-and-conquer 28 Sort-and-Count(L, p, q) // L[p..q]: initially unsorted 1. if (p = q) then return 0 2. else 3. m = ë(p+q)/2û 4. rp = Sort-and-Count(L, p, m) 5. rq = Sort-and-Count(L, m+1, q) 6. rm = Merge-and-Count(L, p, m, q) 7. return r = rp + rq + rm
  • 26. IRIS H.-R. JIANG Counting Inversions ¨ Music site tries to match your song preferences with others. ¤ You rank n songs. ¤ Music site consults database to find people with similar tastes. ¨ Similarity metric: number of inversions between two rankings. ¤ My rank: 1, 2, …, n. ¤ Your rank: a1, a2, …, an. ¤ Songs i and j inverted if i < j, but ai > aj. ¨ Brute force: check all Q(n2) pairs i and j. Divide-and-conquer 25 Me You A B C D E 1 2 3 4 5 2 4 1 3 5 1 2 3 4 5 2 4 1 3 5 Me You inversion = crossing (2, 1), (4, 1), (4, 3) IRIS H.-R. JIANG Divide and Conquer ¨ Counting inversions ¤ Divide: separate the list into two pieces. ¤ Conquer: recursively count inversions in each half. ¤ Combine: count inversions where ai and aj are in different halves, and return sum of three quantities. Divide-and-conquer 26 1 5 4 8 10 2 6 9 12 3 7 11 1 5 4 8 10 2 6 9 12 3 7 11 5 red-red inversions (5,4), (5,2), (4,2), (8,2), (10,2) 8 green-green inversions (6,3), (9,3), (9,7), (12,11), (12,3), (12,7), (11,3), (11,7) 9 red-green inversions (5,3), (4,3), (8,6), (8,3), (8,7), (10,6), (10,9), (10,3), (10,7) Combine: ?? Conquer: 2T(n/2) Divide: O(1) Total: 5 + 8 + 9 = 22 inversions IRIS H.-R. JIANG Combine? ¨ Inversions: inter and intra ¤ Intra: inversions within each half – done by conquer ¤ Inter: inversions between two halves – done by combine n The “combine” in Mergesort is done in O(n); goal: O(n log2 n) n Assume each half is sorted. n Count inversions where ai and aj are in different halves. n Merge two sorted halves into sorted whole. Divide-and-conquer 27 to maintain sorted invariant 1 2 4 5 8 10 3 6 7 11 12 9 (4,3), (5,3), (8,6), (10,3), (8,6), (10,6), (8,7), (10,7), (10,9) 9 red-green inversions Merge: O(n) Count: O(n) 4 2 2 0 0 1 1 2 3 4 5 6 7 8 9 11 12 10 Combine: O(n) Total: O(n log2 n) IRIS H.-R. JIANG Implementation: Counting Inversions ¨ Similar to Mergesort, extra effort on counting inter-inversions Divide-and-conquer 28 Sort-and-Count(L, p, q) // L[p..q]: initially unsorted 1. if (p = q) then return 0 2. else 3. m = ë(p+q)/2û 4. rp = Sort-and-Count(L, p, m) 5. rq = Sort-and-Count(L, m+1, q) 6. rm = Merge-and-Count(L, p, m, q) 7. return r = rp + rq + rm
  • 27. IRIS H.-R. JIANG Counting Inversions ¨ Music site tries to match your song preferences with others. ¤ You rank n songs. ¤ Music site consults database to find people with similar tastes. ¨ Similarity metric: number of inversions between two rankings. ¤ My rank: 1, 2, …, n. ¤ Your rank: a1, a2, …, an. ¤ Songs i and j inverted if i < j, but ai > aj. ¨ Brute force: check all Q(n2) pairs i and j. Divide-and-conquer 25 Me You A B C D E 1 2 3 4 5 2 4 1 3 5 1 2 3 4 5 2 4 1 3 5 Me You inversion = crossing (2, 1), (4, 1), (4, 3) IRIS H.-R. JIANG Divide and Conquer ¨ Counting inversions ¤ Divide: separate the list into two pieces. ¤ Conquer: recursively count inversions in each half. ¤ Combine: count inversions where ai and aj are in different halves, and return sum of three quantities. Divide-and-conquer 26 1 5 4 8 10 2 6 9 12 3 7 11 1 5 4 8 10 2 6 9 12 3 7 11 5 red-red inversions (5,4), (5,2), (4,2), (8,2), (10,2) 8 green-green inversions (6,3), (9,3), (9,7), (12,11), (12,3), (12,7), (11,3), (11,7) 9 red-green inversions (5,3), (4,3), (8,6), (8,3), (8,7), (10,6), (10,9), (10,3), (10,7) Combine: ?? Conquer: 2T(n/2) Divide: O(1) Total: 5 + 8 + 9 = 22 inversions IRIS H.-R. JIANG Combine? ¨ Inversions: inter and intra ¤ Intra: inversions within each half – done by conquer ¤ Inter: inversions between two halves – done by combine n The “combine” in Mergesort is done in O(n); goal: O(n log2 n) n Assume each half is sorted. n Count inversions where ai and aj are in different halves. n Merge two sorted halves into sorted whole. Divide-and-conquer 27 to maintain sorted invariant 1 2 4 5 8 10 3 6 7 11 12 9 (4,3), (5,3), (8,6), (10,3), (8,6), (10,6), (8,7), (10,7), (10,9) 9 red-green inversions Merge: O(n) Count: O(n) 4 2 2 0 0 1 1 2 3 4 5 6 7 8 9 11 12 10 Combine: O(n) Total: O(n log2 n) IRIS H.-R. JIANG Implementation: Counting Inversions ¨ Similar to Mergesort, extra effort on counting inter-inversions Divide-and-conquer 28 Sort-and-Count(L, p, q) // L[p..q]: initially unsorted 1. if (p = q) then return 0 2. else 3. m = ë(p+q)/2û 4. rp = Sort-and-Count(L, p, m) 5. rq = Sort-and-Count(L, m+1, q) 6. rm = Merge-and-Count(L, p, m, q) 7. return r = rp + rq + rm
  • 28. IRIS H.-R. JIANG Counting Inversions ¨ Music site tries to match your song preferences with others. ¤ You rank n songs. ¤ Music site consults database to find people with similar tastes. ¨ Similarity metric: number of inversions between two rankings. ¤ My rank: 1, 2, …, n. ¤ Your rank: a1, a2, …, an. ¤ Songs i and j inverted if i < j, but ai > aj. ¨ Brute force: check all Q(n2) pairs i and j. Divide-and-conquer 25 Me You A B C D E 1 2 3 4 5 2 4 1 3 5 1 2 3 4 5 2 4 1 3 5 Me You inversion = crossing (2, 1), (4, 1), (4, 3) IRIS H.-R. JIANG Divide and Conquer ¨ Counting inversions ¤ Divide: separate the list into two pieces. ¤ Conquer: recursively count inversions in each half. ¤ Combine: count inversions where ai and aj are in different halves, and return sum of three quantities. Divide-and-conquer 26 1 5 4 8 10 2 6 9 12 3 7 11 1 5 4 8 10 2 6 9 12 3 7 11 5 red-red inversions (5,4), (5,2), (4,2), (8,2), (10,2) 8 green-green inversions (6,3), (9,3), (9,7), (12,11), (12,3), (12,7), (11,3), (11,7) 9 red-green inversions (5,3), (4,3), (8,6), (8,3), (8,7), (10,6), (10,9), (10,3), (10,7) Combine: ?? Conquer: 2T(n/2) Divide: O(1) Total: 5 + 8 + 9 = 22 inversions IRIS H.-R. JIANG Combine? ¨ Inversions: inter and intra ¤ Intra: inversions within each half – done by conquer ¤ Inter: inversions between two halves – done by combine n The “combine” in Mergesort is done in O(n); goal: O(n log2 n) n Assume each half is sorted. n Count inversions where ai and aj are in different halves. n Merge two sorted halves into sorted whole. Divide-and-conquer 27 to maintain sorted invariant 1 2 4 5 8 10 3 6 7 11 12 9 (4,3), (5,3), (8,6), (10,3), (8,6), (10,6), (8,7), (10,7), (10,9) 9 red-green inversions Merge: O(n) Count: O(n) 4 2 2 0 0 1 1 2 3 4 5 6 7 8 9 11 12 10 Combine: O(n) Total: O(n log2 n) IRIS H.-R. JIANG Implementation: Counting Inversions ¨ Similar to Mergesort, extra effort on counting inter-inversions Divide-and-conquer 28 Sort-and-Count(L, p, q) // L[p..q]: initially unsorted 1. if (p = q) then return 0 2. else 3. m = ë(p+q)/2û 4. rp = Sort-and-Count(L, p, m) 5. rq = Sort-and-Count(L, m+1, q) 6. rm = Merge-and-Count(L, p, m, q) 7. return r = rp + rq + rm
  • 29. M. I. Shamos and D. Hoey, 1975 Closest Pair of Points 29 Divide-and-conquer IRIS H.-R. JIANG Closest Pair of Points ¨ The closest pair of points problem ¨ Given ¤ A set of n points on a plane, pi is located at (xi, yi). ¨ Find ¤ A pair with the smallest Euclidean distance between them n Euclidean distance between pi and pj = [(xi - xj)2 + (yi - yj)2]1/2 Divide-and-conquer 30 IRIS H.-R. JIANG Closest Pair of Points: First Attempt ¨ Q: How? ¨ A: Brute-force? Q(n2) comparisons. ¨ Q: What if 1-D version? ¨ A: If points are all on a line, easy! O(n log n) for sorting. ¨ Q: What if 2-D version? ¨ A: Non-trivial. ¨ Assumption: No two points have same x-coordinate. Divide-and-conquer 31 to make presentation cleaner IRIS H.-R. JIANG L Divide-and-Conquer ¨ Divide: draw vertical line L so that half points on each side. ¨ Conquer: find closest pair in each side recursively. ¨ Combine: find closest pair with one point in each side; return best of 3 solutions. ¤ Q: How to “combine” in O(n)? Divide-and-conquer 32 M. I. Shamos and D. Hoey. "Closest-point problems." In Proc. 16th Annual IEEE Symposium on Foundations of Computer Science (FOCS), pp. 151—162, 1975 8 21 12
  • 30. M. I. Shamos and D. Hoey, 1975 Closest Pair of Points 29 Divide-and-conquer IRIS H.-R. JIANG Closest Pair of Points ¨ The closest pair of points problem ¨ Given ¤ A set of n points on a plane, pi is located at (xi, yi). ¨ Find ¤ A pair with the smallest Euclidean distance between them n Euclidean distance between pi and pj = [(xi - xj)2 + (yi - yj)2]1/2 Divide-and-conquer 30 IRIS H.-R. JIANG Closest Pair of Points: First Attempt ¨ Q: How? ¨ A: Brute-force? Q(n2) comparisons. ¨ Q: What if 1-D version? ¨ A: If points are all on a line, easy! O(n log n) for sorting. ¨ Q: What if 2-D version? ¨ A: Non-trivial. ¨ Assumption: No two points have same x-coordinate. Divide-and-conquer 31 to make presentation cleaner IRIS H.-R. JIANG L Divide-and-Conquer ¨ Divide: draw vertical line L so that half points on each side. ¨ Conquer: find closest pair in each side recursively. ¨ Combine: find closest pair with one point in each side; return best of 3 solutions. ¤ Q: How to “combine” in O(n)? Divide-and-conquer 32 M. I. Shamos and D. Hoey. "Closest-point problems." In Proc. 16th Annual IEEE Symposium on Foundations of Computer Science (FOCS), pp. 151—162, 1975 8 21 12
  • 31. M. I. Shamos and D. Hoey, 1975 Closest Pair of Points 29 Divide-and-conquer IRIS H.-R. JIANG Closest Pair of Points ¨ The closest pair of points problem ¨ Given ¤ A set of n points on a plane, pi is located at (xi, yi). ¨ Find ¤ A pair with the smallest Euclidean distance between them n Euclidean distance between pi and pj = [(xi - xj)2 + (yi - yj)2]1/2 Divide-and-conquer 30 IRIS H.-R. JIANG Closest Pair of Points: First Attempt ¨ Q: How? ¨ A: Brute-force? Q(n2) comparisons. ¨ Q: What if 1-D version? ¨ A: If points are all on a line, easy! O(n log n) for sorting. ¨ Q: What if 2-D version? ¨ A: Non-trivial. ¨ Assumption: No two points have same x-coordinate. Divide-and-conquer 31 to make presentation cleaner IRIS H.-R. JIANG L Divide-and-Conquer ¨ Divide: draw vertical line L so that half points on each side. ¨ Conquer: find closest pair in each side recursively. ¨ Combine: find closest pair with one point in each side; return best of 3 solutions. ¤ Q: How to “combine” in O(n)? Divide-and-conquer 32 M. I. Shamos and D. Hoey. "Closest-point problems." In Proc. 16th Annual IEEE Symposium on Foundations of Computer Science (FOCS), pp. 151—162, 1975 8 21 12
  • 32. M. I. Shamos and D. Hoey, 1975 Closest Pair of Points 29 Divide-and-conquer IRIS H.-R. JIANG Closest Pair of Points ¨ The closest pair of points problem ¨ Given ¤ A set of n points on a plane, pi is located at (xi, yi). ¨ Find ¤ A pair with the smallest Euclidean distance between them n Euclidean distance between pi and pj = [(xi - xj)2 + (yi - yj)2]1/2 Divide-and-conquer 30 IRIS H.-R. JIANG Closest Pair of Points: First Attempt ¨ Q: How? ¨ A: Brute-force? Q(n2) comparisons. ¨ Q: What if 1-D version? ¨ A: If points are all on a line, easy! O(n log n) for sorting. ¨ Q: What if 2-D version? ¨ A: Non-trivial. ¨ Assumption: No two points have same x-coordinate. Divide-and-conquer 31 to make presentation cleaner IRIS H.-R. JIANG L Divide-and-Conquer ¨ Divide: draw vertical line L so that half points on each side. ¨ Conquer: find closest pair in each side recursively. ¨ Combine: find closest pair with one point in each side; return best of 3 solutions. ¤ Q: How to “combine” in O(n)? Divide-and-conquer 32 M. I. Shamos and D. Hoey. "Closest-point problems." In Proc. 16th Annual IEEE Symposium on Foundations of Computer Science (FOCS), pp. 151—162, 1975 8 21 12
  • 33. IRIS H.-R. JIANG Combining the Solutions (1/4) ¨ Find closest pair with one point in each side. ¤ L = {(x, y): x = x* = x-coordinate of the rightmost point in Q}. ¤ d = the smaller one of these two pairs. ¨ Observation: only need to consider points within d of line L. ¤ Q: Why? Divide-and-conquer 33 d = min(12, 21) L 21 12 d IRIS H.-R. JIANG Combining the Solutions (2/4) ¨ If $ qÎQ and rÎR for which d(q, r) < d, then each of q and r lies within a distance d of L. ¨ Pf: Suppose such q and r exist; let q = (qx, qy) and r = (rx, ry). ¤ By definition, qx £ x* £ rx. ¤ x* - qx £ rx - qx £ d(q, r) < d; rx - x* £ rx - qx £ d(q, r) < d. Divide-and-conquer 34 d = min(d(q*0, q*1), d(r*0, r*1)) L d(r*0, r*1) d(q*0, q*1) d Q R q r L: x = x* IRIS H.-R. JIANG Combining the Solutions (3/4) ¨ Observation: only need to consider points within d of line L. ¤ Sort points in 2d-strip by their y-coordinate. ¤ Only check distances of those within 15 positions in sorted list of y-coordinates! n Q: Why? Divide-and-conquer 35 d = min(12, 21) L 21 12 d 6 5 4 3 2 1 IRIS H.-R. JIANG Combining the Solutions (4/4) ¨ If s, s’ÎS are of d(s, s’) < d, then s and s’ are within 15 positions of each other in the sorted list Sy of y-coordinates of S. ¨ Pf: S contains all points within d of line L, we partition S into boxes, each box contains at most one point. ¤ Partition the region into boxes, each of area d/2*d/2; we claim 1. s and s’ lies in different boxes n Suppose s and s’ lies in the same box n These two points both belong either to Q or to R n d(s, s’) £ 0.5*d*(2)1/2 < d ®¬ 2. s and s’ are within 15 positions n Suppose s, s’ÎS of d(s, s’) < d and they are at least 16 positions apart in Sy. n Assume WLOG s has the smaller y-coordinate; since at most one point per box, at least 3 rows of S lying between s and s’. n d(s, s’) ³ 3d/2 > d ®¬ Divide-and-conquer 36 L d S d d/2 d/2
  • 34. IRIS H.-R. JIANG Combining the Solutions (1/4) ¨ Find closest pair with one point in each side. ¤ L = {(x, y): x = x* = x-coordinate of the rightmost point in Q}. ¤ d = the smaller one of these two pairs. ¨ Observation: only need to consider points within d of line L. ¤ Q: Why? Divide-and-conquer 33 d = min(12, 21) L 21 12 d IRIS H.-R. JIANG Combining the Solutions (2/4) ¨ If $ qÎQ and rÎR for which d(q, r) < d, then each of q and r lies within a distance d of L. ¨ Pf: Suppose such q and r exist; let q = (qx, qy) and r = (rx, ry). ¤ By definition, qx £ x* £ rx. ¤ x* - qx £ rx - qx £ d(q, r) < d; rx - x* £ rx - qx £ d(q, r) < d. Divide-and-conquer 34 d = min(d(q*0, q*1), d(r*0, r*1)) L d(r*0, r*1) d(q*0, q*1) d Q R q r L: x = x* IRIS H.-R. JIANG Combining the Solutions (3/4) ¨ Observation: only need to consider points within d of line L. ¤ Sort points in 2d-strip by their y-coordinate. ¤ Only check distances of those within 15 positions in sorted list of y-coordinates! n Q: Why? Divide-and-conquer 35 d = min(12, 21) L 21 12 d 6 5 4 3 2 1 IRIS H.-R. JIANG Combining the Solutions (4/4) ¨ If s, s’ÎS are of d(s, s’) < d, then s and s’ are within 15 positions of each other in the sorted list Sy of y-coordinates of S. ¨ Pf: S contains all points within d of line L, we partition S into boxes, each box contains at most one point. ¤ Partition the region into boxes, each of area d/2*d/2; we claim 1. s and s’ lies in different boxes n Suppose s and s’ lies in the same box n These two points both belong either to Q or to R n d(s, s’) £ 0.5*d*(2)1/2 < d ®¬ 2. s and s’ are within 15 positions n Suppose s, s’ÎS of d(s, s’) < d and they are at least 16 positions apart in Sy. n Assume WLOG s has the smaller y-coordinate; since at most one point per box, at least 3 rows of S lying between s and s’. n d(s, s’) ³ 3d/2 > d ®¬ Divide-and-conquer 36 L d S d d/2 d/2
  • 35. IRIS H.-R. JIANG Combining the Solutions (1/4) ¨ Find closest pair with one point in each side. ¤ L = {(x, y): x = x* = x-coordinate of the rightmost point in Q}. ¤ d = the smaller one of these two pairs. ¨ Observation: only need to consider points within d of line L. ¤ Q: Why? Divide-and-conquer 33 d = min(12, 21) L 21 12 d IRIS H.-R. JIANG Combining the Solutions (2/4) ¨ If $ qÎQ and rÎR for which d(q, r) < d, then each of q and r lies within a distance d of L. ¨ Pf: Suppose such q and r exist; let q = (qx, qy) and r = (rx, ry). ¤ By definition, qx £ x* £ rx. ¤ x* - qx £ rx - qx £ d(q, r) < d; rx - x* £ rx - qx £ d(q, r) < d. Divide-and-conquer 34 d = min(d(q*0, q*1), d(r*0, r*1)) L d(r*0, r*1) d(q*0, q*1) d Q R q r L: x = x* IRIS H.-R. JIANG Combining the Solutions (3/4) ¨ Observation: only need to consider points within d of line L. ¤ Sort points in 2d-strip by their y-coordinate. ¤ Only check distances of those within 15 positions in sorted list of y-coordinates! n Q: Why? Divide-and-conquer 35 d = min(12, 21) L 21 12 d 6 5 4 3 2 1 IRIS H.-R. JIANG Combining the Solutions (4/4) ¨ If s, s’ÎS are of d(s, s’) < d, then s and s’ are within 15 positions of each other in the sorted list Sy of y-coordinates of S. ¨ Pf: S contains all points within d of line L, we partition S into boxes, each box contains at most one point. ¤ Partition the region into boxes, each of area d/2*d/2; we claim 1. s and s’ lies in different boxes n Suppose s and s’ lies in the same box n These two points both belong either to Q or to R n d(s, s’) £ 0.5*d*(2)1/2 < d ®¬ 2. s and s’ are within 15 positions n Suppose s, s’ÎS of d(s, s’) < d and they are at least 16 positions apart in Sy. n Assume WLOG s has the smaller y-coordinate; since at most one point per box, at least 3 rows of S lying between s and s’. n d(s, s’) ³ 3d/2 > d ®¬ Divide-and-conquer 36 L d S d d/2 d/2
  • 36. IRIS H.-R. JIANG Combining the Solutions (1/4) ¨ Find closest pair with one point in each side. ¤ L = {(x, y): x = x* = x-coordinate of the rightmost point in Q}. ¤ d = the smaller one of these two pairs. ¨ Observation: only need to consider points within d of line L. ¤ Q: Why? Divide-and-conquer 33 d = min(12, 21) L 21 12 d IRIS H.-R. JIANG Combining the Solutions (2/4) ¨ If $ qÎQ and rÎR for which d(q, r) < d, then each of q and r lies within a distance d of L. ¨ Pf: Suppose such q and r exist; let q = (qx, qy) and r = (rx, ry). ¤ By definition, qx £ x* £ rx. ¤ x* - qx £ rx - qx £ d(q, r) < d; rx - x* £ rx - qx £ d(q, r) < d. Divide-and-conquer 34 d = min(d(q*0, q*1), d(r*0, r*1)) L d(r*0, r*1) d(q*0, q*1) d Q R q r L: x = x* IRIS H.-R. JIANG Combining the Solutions (3/4) ¨ Observation: only need to consider points within d of line L. ¤ Sort points in 2d-strip by their y-coordinate. ¤ Only check distances of those within 15 positions in sorted list of y-coordinates! n Q: Why? Divide-and-conquer 35 d = min(12, 21) L 21 12 d 6 5 4 3 2 1 IRIS H.-R. JIANG Combining the Solutions (4/4) ¨ If s, s’ÎS are of d(s, s’) < d, then s and s’ are within 15 positions of each other in the sorted list Sy of y-coordinates of S. ¨ Pf: S contains all points within d of line L, we partition S into boxes, each box contains at most one point. ¤ Partition the region into boxes, each of area d/2*d/2; we claim 1. s and s’ lies in different boxes n Suppose s and s’ lies in the same box n These two points both belong either to Q or to R n d(s, s’) £ 0.5*d*(2)1/2 < d ®¬ 2. s and s’ are within 15 positions n Suppose s, s’ÎS of d(s, s’) < d and they are at least 16 positions apart in Sy. n Assume WLOG s has the smaller y-coordinate; since at most one point per box, at least 3 rows of S lying between s and s’. n d(s, s’) ³ 3d/2 > d ®¬ Divide-and-conquer 36 L d S d d/2 d/2
  • 37. IRIS H.-R. JIANG Closest Pair Algorithm Divide-and-conquer 37 Closest-Pair-Rec(Px, Py) 1. if |P| £ 3 then return closest pair measured by all pair-wise distances 2. x* = (én/2ù)-th smallest x-coordinate in Px 3. construct Qx, Qy, Rx, Ry // O(n) 4. (q*0, q*1) = Closest-Pair-Rec(Qx, Qy) // T(n/2) 5. (r*0, r*1) = Closest-Pair-Rec(Rx, Ry) // T(n/2) 6. d = min(d(q*0, q*1), d(r*0, r*1)) 7. L = {(x, y): x = x*}; S = {points in P within distance d of L} 8. construct Sy // O(n) 9. for each sÎS do 10. compute distance from s to each of next 15 points in Sy 11.d(s, s’) = min distance of all computed distances // O(n) 12.if d(s, s’) < d then return (s, s’) 13.else if d(q*0, q*1) < d(r*0, r*1) then return (q*0, q*1) 14.else return (r*0, r*1) Closest-Pair(P) 1. construct Px and Py // O(n log n) 2. (p*0, p*1) = Closest-Pair-Rec(Px, Py) // T(n) Closest-Pair-Rec(Px, Py) 1. if |P| £ 3 then return closest pair measured by all pair-wise distances 2. x* = (én/2ù)-th smallest x-coordinate in Px 3. construct Qx, Qy, Rx, Ry 4. (q*0, q*1) = Closest-Pair-Rec(Qx, Qy) 5. (r*0, r*1) = Closest-Pair-Rec(Rx, Ry) 6. d = min(d(q*0, q*1), d(r*0, r*1)) 7. L = {(x, y): x = x*}; S = {points in P within distance d of L} 8. construct Sy 9. for each sÎS do 10. compute distance from s to each of next 15 points in Sy 11.d(s, s’) = min distance of all computed distances 12.if d(s, s’) < d then return (s, s’) 13.else if d(q*0, q*1) < d(r*0, r*1) then return (q*0, q*1) 14.else return (r*0, r*1) Closest-Pair(P) 1. construct Px and Py 2. (p*0, p*1) = Closest-Pair-Rec(Px, Py)