SlideShare a Scribd company logo

CS330-Lectures Statistics And Probability

B
bryan111472

Here are the key steps to multiply two large integers efficiently: 1. Use a divide-and-conquer approach called the Karatsuba algorithm. It recursively breaks the multiplication into smaller subproblems. 2. On inputs of size n, the Karatsuba algorithm uses 3 multiplications of size approximately n/2 instead of the 4 multiplications of the grade school method. 3. This reduction saves a multiplicative factor, resulting in an asymptotic running time of O(n^{log_2 3}) ≈ O(n^{1.585}), which is faster than the O(n^2) time of the grade school method. 4. The main idea is to express the products of

Software
1 of 46
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
Recursion tree method Solve 22 T(n) = 2 ⋅ T(n/2) + Θ(n) T (n) T (n / 2) T (n / 2) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 4) T (n / 4) T (n / 4) T (n / 4) ⋮ inputs of length n inputs of length n/2 inputs of length n/4 base case: input n=1 T(1) T(1) ……….
Recursion tree method — TopHat Question 2 Solve 23 T(n) = 2 ⋅ T(n/2) + Θ(n) T (n) T (n / 2) T (n / 2) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 4) T (n / 4) T (n / 4) T (n / 4) ⋮ Question. How many layers? A. constant B. log2 n C. log4 n D. n E. n2 T(1) T(1) ……….
Recursion tree method — TopHat Question 3 Solve 24 T(n) = 2 ⋅ T(n/2) + Θ(n) T (n) T (n / 2) T (n / 2) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 4) T (n / 4) T (n / 4) T (n / 4) ⋮ Question. How many leaves? A. constant B. log2 n C. log4 n D. n E. n2 T(1) T(1) ……….
Recursion tree method Solve 25 T(n) = 2 ⋅ T(n/2) + Θ(n) T (n) T (n / 2) T (n / 2) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 4) T (n / 4) T (n / 4) T (n / 4) ⋮ inputs of length n inputs of length n/2 inputs of length n/4 base case: input n=1 log 2 n #leaves = n T(1) T(1) ……….
Recursion tree method — analysis idea • charge each operation to the function call (i.e. tree node) at which it is executed • for Mergesort on a subarray of length k we do O(k) work to compute the split point and do the merge • work in further recursive calls is counted separately T (n) T (n / 2) T (n / 2) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 4) T (n / 4) T (n / 4) T (n / 4) ⋮ T(n) = 2 ⋅ T(n/2) + cn where c > 0 T(1) T(1) ………. use cn instead of Θ(n)
Recursion tree method — analysis idea • charge each operation to the function call (i.e. tree node) at which it is executed • for Mergesort on a subarray of length k we do O(k) work to compute the split point and do the merge • work in further recursive calls is counted separately T (n) T (n / 2) T (n / 2) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 4) T (n / 4) T (n / 4) T (n / 4) ⋮ T(n) = 2 ⋅ T(n/2) + cn where c > 0 cn cn/2 cn/2 T(1) T(1) ………. use cn instead of Θ(n)
Recursion tree method — analysis idea • charge each operation to the function call (i.e. tree node) at which it is executed • for Mergesort on a subarray of length k we do O(k) work to compute the split point and do the merge • work in further recursive calls is counted separately T (n) T (n / 2) T (n / 2) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 4) T (n / 4) T (n / 4) T (n / 4) T(n) = 2 ⋅ T(n/2) + cn where c > 0 cn cn/2 cn/2 cn/4 cn/4 cn/4 cn/4 cn/2k amount of work per level: c ⋅ n 2 ⋅ c ⋅ n/2 4 ⋅ c ⋅ n/4 2k ⋅ c ⋅ n/2k Θ(n) Θ(1) Total: Θ(n log n) use cn instead of Θ(n)
binary search 29 Input: sorted array A, integer x Output: the index in A where the int x is located or “x is not in A” 1.Divide: compare x to middle element 2.Conquer: recursively search one subarray 3.Combine: trivial 2 3 7 10 11 14 16 18 20 23 example: find 16
binary search - running time 30 Algorithm 1: BinarySearch( sorted array A, int p, int r, query ) /* find the index of query in A (if present) */ 1 q bp+r 2 c; 2 middle A[q]; 3 if query == middle then 4 return q; 5 if query < middle then 6 BinarySearch(A, p, q, query); 7 else 8 BinarySearch(A, p, q, query); 9 return q not in A;
binary search — running time 31 Algorithm 1: BinarySearch( sorted array A, int p, int r, query ) /* find the index of query in A (if present) */ 1 q bp+r 2 c; 2 middle A[q]; 3 if query == middle then 4 return q; 5 if query < middle then 6 BinarySearch(A, p, q, query); 7 else 8 BinarySearch(A, p, q, query); 9 return q not in A; O(c) recursive call on array half the size T(n) = 1 ⋅ T(n/2) + Θ(1) #of subproblems subproblem size work dividing and combining
find closed form by “telescoping” Reminder: our goal is to find a closed form formula for the recurrence method: • substitute the recursive formula until we get a good guess • use induction to prove the formula T(n) = T(n/2) + c = T(n/4) + 2c = … = T(n/23 ) + 3c = … = T(n/2log n ) + (log n)c = Θ(log n)
proof by “telescoping” claim. proof. by induction on n. • base case: for n = 2 (or any constant) it takes const comparisons and log n = const • suppose that the formula is true for every k < n • proof for n: substitute k = n/2 T(n) = Θ(log n) T(n) = T(n/2) + Θ(1) = Θ(log(n/2)) + Θ(1) = Θ(log n)
Proof by “telescoping” for mergesort Proposition. If T (n) satisfies the following recurrence, then T (n) = n log2 n. Pf. The last line corresponds to the base case. We know that the base case is T(1) = 0 by substituting log(n) for k we get 34 assuming n is a power of 2 Substitute the formula in to T(n/2) base case T(1) = T( n 2k ) =) k = log(n) T(n)  2 · T n 2 + cn  4 · T n 4 + 2cn   8 · T n 8 + 3cn  24 · T n 24 + 4cn  · · ·  2k T n 2k + kcn T(n)  2log(n) · T n 2log(n) + log(n) · n = n + O(n log n) T(n) = ( 1 n == 1 2T n 2 + cn otherwise
Master Theorem For recurrences defined as T(n) = aT(n/b) + f(n) in which a > 0, b ≥ 1 are constants Then the overall computational cost is given by T(n) = Θ(nlogb a ) ! "# $ leaves + logb n % k=1 ak−1 f(n/bk−1 ) ! "# $ internal nodes Such that T(n) =      Θ(nlogb a) if f(n) = O(nlogb a−!) Θ(nlogb a log n) if f(n) = Θ(nlogb a) Θ(f(n)) if f(n) = Ω(nlogb a+!) and af(n/b) ≤ cf(n) in which a ≥ 1, b > 1, c < 1 and ! > 0 are constants.
Solve the following recurrences 1. T(n) = 4T(n/2) + n 2. T(n) = 4T(n/2) + n2 3. T(n) = 4T(n/2) + n3
Asymptotic time to execute integer operations Basic operations: adding, subtracting, multiplying, dividing Complexity of mathematical operations: • Usually in this class we count them as constant time • Today: we look at algorithms for efficiently implementing these basic operations. • Today: in our analysis one computational step is counted as one bit operation. Unit of measurement will be bit operations. Input: two n-bit long integers a , b Output: arithmetic operation on the two integers 2
Integer addition Addition. Given two n-bit integers a and b, compute a + b. Subtraction. Given two n-bit integers a and b, compute a – b. Grade-school algorithm. Θ(n) bit operations. 3 1 1 0 1 0 1 0 1 + 0 1 1 1 1 1 0 1 2 1 3 + 1 2 5
Integer addition Addition. Given two n-bit integers a and b, compute a + b. Subtraction. Given two n-bit integers a and b, compute a – b. Grade-school algorithm. Θ(n) bit operations. 4 1 1 1 1 1 1 0 1 1 1 0 1 0 1 0 1 + 0 1 1 1 1 1 0 1 1 0 1 0 1 0 0 1 0 2 1 3 + 1 2 5 3 3 8
Integer addition Addition. Given two n-bit integers a and b, compute a + b. Subtraction. Given two n-bit integers a and b, compute a – b. Grade-school algorithm. Θ(n) bit operations. Remark. Grade-school addition and subtraction algorithms are asymptotically optimal. 5 an-1 an-2…a2 a1 a0 bits of a + b + bn-1 bn-2…b2 b1 b0
Integer multiplication Multiply 21310 x 12510 = 110101012 x 011111012 ・(subscript refers to decimal and binary representation) Grade-school algorithm: 6 2 1 3 x 1 2 5 +
Integer multiplication Multiply 21310 x 12510 = 110101012 x 011111012 ・(subscript refers to decimal and binary representation) Grade-school algorithm: 7 2 1 3 x 1 2 5 1 0 6 5 4 2 6 + 2 1 3 2 6 6 2 5
TopHat Multiplication. Given two n-bit integers a and b, compute a × b. Grade-school algorithm. 8 1 1 0 1 0 1 0 1 × 0 1 1 1 1 1 0 1 1 1 0 1 0 1 0 1 0 0 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 0 0 0 0 0 0 0 0 0 1 1 0 1 0 0 0 0 0 0 0 0 0 0 1 Question: What is the running time? A. O(1) B. O(log n) C. O(n) D. D. (n2)
Multiplication. Given two n-bit integers a and b, compute a × b. Grade-school algorithm. Θ(n2) bit operations. Integer multiplication 9 1 1 0 1 0 1 0 1 × 0 1 1 1 1 1 0 1 1 1 0 1 0 1 0 1 0 0 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 0 0 0 0 0 0 0 0 0 1 1 0 1 0 0 0 0 0 0 0 0 0 0 1 n i n t e r m e d i a t e products 2n additions n bits each total of O(n2) bits
Integer multiplication 10 n i n t e r m e d i a t e products 2n additions n bits each total of O(n2) bits an-1 an-2…a2 a1 a0 n bits + bn-1 bn-2…b2 b1 b0 a x b 2n bits n bits n bits … + Multiplication. Given two n-bit integers a and b, compute a × b. Grade-school algorithm. Θ(n2) bit operations. Conjecture. [Kolmogorov 1952] Grade-school algorithm is optimal. Theorem. [Karatsuba 1960] Conjecture is wrong. (his result ) O(nlog23 ) = O(n1.59... )
Multiplying large integers Input: n bit integers a and b Compute: ab computing ab = multiplying two n bit integers ~ O(n2) time Divide-and-conquer: ・subproblem: same problem on smaller input - input size here is the number of bits in the integers - goal: compute ab by multiplying n/2 bit integers and then combine them 11
Divide: n bits to n/2 12 355710 = 3500 + 57 = 35 x 102 + 57 355710 = 1101111001012 = 1101110000002 + 1001012 = 1101112 x 26 + 1001012 Input: n-bit integers a , b Compute: ab How to divide n-bit integers into n/2-bit ints? Note: in decimal multiplying by 10k shifts the number k bits to the left. Same in binary, multiplying by 2k shifts k bits to the left (glues k 0s at the end.)
TopHat Question. Write the representation of the base-3 number 1201203 as two parts of n/2 digit base-3 integers. A. B. C. D. 2 ⋅ 1203 3 ⋅ 1203 + 1203 1203 ⋅ 33 + 1203 1203 ⋅ 36 + 1203 ⋅ 33
Divide: n bits to n/2 Input: integers a ,b of length n bits How to “divide”? What are the subproblems? ・split a in to two substrings A1, A0 of length n/2 ・then a = A12n/2 + A0 ・same for b: b = B12n/2+B0 14 a = an 1an 2 . . . an 2 an 2 1 . . . a2a1a0 A1 = an−1an−2…an 2 A0 = an 2 −1…a1a0
Multiplying large integers using D&C Input: n bit integers a and b Compute: ab ・A1, A0, B1, B0 are n/2 bit integers ab consists of multiplying 2 n bit integers ~ O(n2) time a = A12n/2 + A0 b = B12n/2 + B0 Compute: ab = (A12n/2 + A0)(B12n/2 + B0)= 15
Multiplying large integers using D&C Input: n bit integers a and b Compute: ab ・A1, A0, B1, B0 are n/2 bit integers ab consists of multiplying 2 n bit integers ~ O(n2) time a = A12n/2 + A0 b = B12n/2 + B0 Compute ab = A1B12n + (A1B0 + A0B1)2n/2 + A0B0 ・this formula consists of 4 multiplications of n/2 bit numbers and 3 additions Divide and conquer approach: multiply n/2 bit integers recursively 16
17 Multiplying large integers using D&C Algorithm 1: Multiply(ints a, b, n) 1 if n == 1 then 2 return ab; 3 m bn 2 c; 4 A0 ba/2m c /* integer division */ 5 B0 bb/2m c /* integer division */ 6 A1 a mod 2m ; 7 B1 b mod 2m ; 8 x Multiply(A1, B1, m); 9 y Multiply(A1, B0, m); 10 z Multiply(A0, B1, m); 11 w Multiply(A0, B0, m); 12 return x22m + (y + z)2m + w; recursive call on input of size n/2 Note that lines 4-7 can be implemented by simple bit-shifts — no real division is taking place! Recurrence: (reminder: ab = A1B12n + (A1B0 + A0B1)2n/2 + A0B0)
18 Solve the recurrence T(n)  ( 1 n = 1 4T(n 2 ) + O(n) otherwise
19 Solve the recurrence T(n)  ( 1 n = 1 4T(n 2 ) + O(n) otherwise Use telescoping and solve T(n) = 4 ⋅ T( n 2 ) + cn T(n) = 4 ⋅ T( n 2 ) + cn = 4 ( 4 ⋅ T( n 4 ) + c n 2) + cn = (22 )2 ⋅ T( n 22 ) + 3cn = … = (22 )3 ⋅ T( n 23 ) + 5cn = … = (22 )k ⋅ T( n 2k ) + (k + 1)cn = … we want to substitute the base case T(1), which happens when k = log n = (22 )log n T(1) + (log n)cn = n2 + Θ(1)n log n = Θ(n2 ) Now we can prove by induction that T(n) = Θ(n2 )
Recurrence: General form: Discussed in detail: Kleinberg-Tardos pages 214 - 218 General formula for recurrences of specific form 20 T(n) = 4T ( n 2) + cn = cnlog24 = Θ(n2 ) T(n) = qT ( n 2 ) + cn = cnlog2 q for q > 2
21 Multiplying large integers using D&C Algorithm 1: Multiply(ints a, b, n) 1 if n == 1 then 2 return ab; 3 m bn 2 c; 4 A0 ba/2m c /* integer division */ 5 B0 bb/2m c /* integer division */ 6 A1 a mod 2m ; 7 B1 b mod 2m ; 8 x Multiply(A1, B1, m); 9 y Multiply(A1, B0, m); 10 z Multiply(A0, B1, m); 11 w Multiply(A0, B0, m); 12 return x22m + (y + z)2m + w; recursive call on input of size n/2 Note that lines 4-7 can be implemented by simple bit-shifts — no real division is taking place! Recurrence: T(n) = 4T(n 2 ) + cn T(n) = ⇥(n2 ) (reminder: ab = A1B12n + (A1B0 + A0B1)2n/2 + A0B0)
Karatsuba’s algorithm Input: n bit integers a and b Compute: ab a = A12n/2 + A0 b = B12n/2 + B0 Compute ab = A1B12n + (A1B0 + A0B1)2n/2 + A0B0 Karatsuba’s trick: (A1+A0)(B1+B0) = A1B1 + A0B0 + (A1B0 + A0B1) 22
Input: n bit integers a and b Compute: ab a = A12n/2 + A0 b = B12n/2 + B0 Compute ab = A1B12n + (A1B0 + A0B1)2n/2 + A0B0 Karatsuba’s trick: (A1+A0)(B1+B0) = A1B1 + A0B0 + (A1B0 + A0B1) Karatsuba’s algorithm 23
Input: n bit integers a and b Compute: ab a = A12n/2 + A0 b = B12n/2 + B0 Compute ab = A1B12n + (A1B0 + A0B1)2n/2 + A0B0 Karatsuba’s trick: (A1+A0)(B1+B0) = A1B1 + A0B0 + (A1B0 + A0B1) Thus we get x = A1B1 , y = A0B0, z = (A1+A0)(B1+B0) ab = A1B12n + (A1B0 + A0B1)2n/2 + A0B0 = x2n + (z - x - y)2n/2 + y The new formula contains only 3 multiplications of n/2 bits. Karatsuba’s algorithm 24 multiply n/2 bit integers n/2 bits!
25 Karatsuba’s algorithm Algorithm 1: Karatsuba(ints a, b, n) 1 if n == 1 then 2 return ab; 3 m bn 2 c; 4 A0 ba/2m c /* integer division */ 5 B0 bb/2m c /* integer division */ 6 A1 a mod 2m ; 7 B1 b mod 2m ; 8 x Karatsuba(A1, B1, m); 9 y Karatsuba(A0, B0, m); 10 z Karatsuba(A1 + A0, B1 + B0, m); 11 return x22m + (z x y)2m + y;
26 Karatsuba’s algorithm - TopHat Algorithm 1: Karatsuba(ints a, b, n) 1 if n == 1 then 2 return ab; 3 m bn 2 c; 4 A0 ba/2m c /* integer division */ 5 B0 bb/2m c /* integer division */ 6 A1 a mod 2m ; 7 B1 b mod 2m ; 8 x Karatsuba(A1, B1, m); 9 y Karatsuba(A0, B0, m); 10 z Karatsuba(A1 + A0, B1 + B0, m); 11 return x22m + (z x y)2m + y; Question: What is the recurrence for this algorithm? A. C. B. D. T(n) = Θ(n) T(n) = 2 ⋅ T( n 3 ) + Θ(n) T(n) = 3 ⋅ T( n 2 ) + Θ(n) T(n) = 3 ⋅ T( n 3 ) + Θ(1)
27 Karatsuba’s algorithm Algorithm 1: Karatsuba(ints a, b, n) 1 if n == 1 then 2 return ab; 3 m bn 2 c; 4 A0 ba/2m c /* integer division */ 5 B0 bb/2m c /* integer division */ 6 A1 a mod 2m ; 7 B1 b mod 2m ; 8 x Karatsuba(A1, B1, m); 9 y Karatsuba(A0, B0, m); 10 z Karatsuba(A1 + A0, B1 + B0, m); 11 return x22m + (z x y)2m + y; T(n) = 3T(n 2 ) + cn T(n) = ⇥(nlog2 3 ) = ⇥(n1.59... )
Solve where Recursion tree method 28 c 2 const cn cn 2 cn 2 cn 4 cn 4 cn 4 cn 4 + work per procedure T(n) = 3T(n 2 ) + cn cn 2 cn 4 cn 4 cn 3 2 cn 9 4 cn (3 2 )k cn
Solve where Recursion tree method - TopHat 29 c 2 const cn cn 2 cn 2 cn 4 cn 4 cn 4 cn 4 + work per procedure T(n) = 3T(n 2 ) + cn cn 2 cn 4 cn 4 cn 3 2 cn 9 4 cn (3 2 )k cn Question: what is the hight of the tree and number of nodes at layer k? (you may assume the root is at layer 0) A. B. C. D. log3 n & nk log2 n & 3k log2 n & n3 log3 n & k3
Solve where Recursion tree method 30 c 2 const cn cn 2 cn 2 cn 4 cn 4 cn 4 cn 4 + work per procedure T(n) = 3T(n 2 ) + cn cn 2 cn 4 cn 4 cn 3 2 cn 9 4 cn (3 2 )k cn (3 2 )log3 n cn = cnlog2 3 = cn1.59... h = log2 n (3 2 log3n ) = 1 n 3log2 n = = 3log3n ⇤ 3 1 log32 = n3log2 3
Recurrence for Karatsuba’s: General form: Discussed in detail: Kleinberg-Tardos pages 214 - 218 General formula for recurrences of specific form 31 T(n) = 3T(n 2 ) + cn = cnlog2 3 T(n) = qT(n 2 ) + cn = cnlog2 q for q > 2
History of asymptotic complexity of integer multiplication Remark. GNU Multiple Precision Library uses one of five different algorithms depending on size of operands. 32 year algorithm order of growth ? grade-school Θ(n2) 1962 Karatsuba–Ofman Θ(n1.585) 1963 Toom-3, Toom-4 Θ(n1.465), Θ(n1.404) 1966 Toom–Cook Θ(n1 + ε) 1971 Schönhage–Strassen Θ(n log n log log n) 2007 Fürer n log n 2 O(log*n) 2019 Harvey, van der Hoeven number of bit operations to multiply two n-bit integers ⇥(n log n) Faster than grade- school algorithm for about 320–640 bits.

More Related Content

DOC
pradeepbishtLecture13 div conq
Pradeep Bisht
 
PPTX
T2311 - Ch 4_Part1.pptx
GadaFarhan
 
PPT
Recurrences
Ala' Mohammad
 
PDF
Recurrence relation solutions
subhashchandra197
 
PPTX
Divide and Conquer_Binary_Search_Merge_Sort.pptx
ranjithkumar379904
 
PPTX
2.pptx
MohAlyasin1
 
PPT
recurrence relation is explained in this
vidhyapm2
 
PPTX
Divide and Conquer in DAA concept. For B Tech CSE
RUHULAMINHAZARIKA
 
pradeepbishtLecture13 div conq
Pradeep Bisht
 
T2311 - Ch 4_Part1.pptx
GadaFarhan
 
Recurrences
Ala' Mohammad
 
Recurrence relation solutions
subhashchandra197
 
Divide and Conquer_Binary_Search_Merge_Sort.pptx
ranjithkumar379904
 
2.pptx
MohAlyasin1
 
recurrence relation is explained in this
vidhyapm2
 
Divide and Conquer in DAA concept. For B Tech CSE
RUHULAMINHAZARIKA
 

Similar to CS330-Lectures Statistics And Probability (20)

PPTX
Solving recurrences
Megha V
 
PDF
Recurrences
DEVTYPE
 
PPT
L2
FALLEE31188
 
PDF
3.pdf
AlaaOdeh18
 
PDF
lec 03wweweweweweweeweweweewewewewee.pdf
Huma Ayub
 
PPTX
solving_Recurrence_relations_using_methods1.pptx
ssuserbc2081
 
PPT
lecture 15
sajinsc
 
PPTX
1_Asymptotic_Notation_pptx.pptx
pallavidhade2
 
PPT
Divide and conquer
Vikas Sharma
 
PPTX
3. D&C and Recurrence Relation.ppYtxVVVV
NetraBansal3
 
PDF
Skiena algorithm 2007 lecture09 linear sorting
zukun
 
PPT
recurrence relations in analysis of algorithm
vidhyapm2
 
PPT
3. Recursion and Recurrences.ppt detail about recursive learning
KashifNadeem52
 
PPT
Algorithm.ppt
Tareq Hasan
 
PPT
3-Chapter Three - Divide and Conquer.ppt
mershaabdisa
 
PPT
5.2 divide and conquer
Krish_ver2
 
PPT
03 dc
Hira Gul
 
PPT
5.2 divede and conquer 03
Krish_ver2
 
PPT
5.2 divede and conquer 03
Krish_ver2
 
PPT
l1.ppt
SonaliAjankar
 
Solving recurrences
Megha V
 
Recurrences
DEVTYPE
 
L2
FALLEE31188
 
3.pdf
AlaaOdeh18
 
lec 03wweweweweweweeweweweewewewewee.pdf
Huma Ayub
 
solving_Recurrence_relations_using_methods1.pptx
ssuserbc2081
 
lecture 15
sajinsc
 
1_Asymptotic_Notation_pptx.pptx
pallavidhade2
 
Divide and conquer
Vikas Sharma
 
3. D&C and Recurrence Relation.ppYtxVVVV
NetraBansal3
 
Skiena algorithm 2007 lecture09 linear sorting
zukun
 
recurrence relations in analysis of algorithm
vidhyapm2
 
3. Recursion and Recurrences.ppt detail about recursive learning
KashifNadeem52
 
Algorithm.ppt
Tareq Hasan
 
3-Chapter Three - Divide and Conquer.ppt
mershaabdisa
 
5.2 divide and conquer
Krish_ver2
 
03 dc
Hira Gul
 
5.2 divede and conquer 03
Krish_ver2
 
5.2 divede and conquer 03
Krish_ver2
 
l1.ppt
SonaliAjankar
 
Ad

Recently uploaded (20)

PDF
How to Choose the Right IT Partner for Your Business in Malaysia
Asian Business Services & Technologies
 
PDF
Design an Analysis of Algorithms I-SECS-1021-03
DilanEscobar1
 
PDF
Wondershare Filmora 15 Crack With Activation Key [2025
ghrom2211g
 
PDF
iTop VPN Free 5.6.0.5262 Crack latest version 2025
itskinga12
 
DOCX
Greta — No-Code AI for Building Full-Stack Web & Mobile Apps
niloyislam1187
 
PPTX
Oracle Fusion HCM Cloud Demo for Beginners
DharanOracleerp
 
PDF
AutoCAD Professional Crack 2025 With License Key
youngle786g
 
PPTX
Why Generative AI is the Future of Content, Code & Creativity?
Webuters Technologies Pvt. Ltd
 
PDF
Odoo Companies in India – Driving Business Transformation.pdf
azhuhardeen1968
 
PPTX
AMADEUS TRAVEL AGENT SOFTWARE | AMADEUS TICKETING SYSTEM
philipnathen82
 
PDF
Nekopoi APK 2025 free lastest update
umarali35kp
 
PDF
Tally Prime Crack Download New Version 5.1 [2025] (License Key Free
itskinga12
 
PDF
CCleaner Pro 6.38.11537 Crack Final Latest Version 2025
itskinga12
 
PDF
Download FL Studio Crack Latest version 2025 ?
ghrom2211g
 
PDF
How to Make Money in the Metaverse_ Top Strategies for Beginners.pdf
Herond Labs
 
PPTX
assetexplorer- product-overview - presentation
nonameist3434
 
PDF
T3DD25 TYPO3 Content Blocks - Deep Dive by André Kraus
Andre Kraus
 
PDF
Salesforce Agentforce AI Implementation.pdf
Robert Mark
 
PDF
Adobe Premiere Pro 2025 (v24.5.0.057) Crack free
ghrom2211g
 
PDF
Designing Intelligence for the Shop Floor.pdf
nikglvk
 
How to Choose the Right IT Partner for Your Business in Malaysia
Asian Business Services & Technologies
 
Design an Analysis of Algorithms I-SECS-1021-03
DilanEscobar1
 
Wondershare Filmora 15 Crack With Activation Key [2025
ghrom2211g
 
iTop VPN Free 5.6.0.5262 Crack latest version 2025
itskinga12
 
Greta — No-Code AI for Building Full-Stack Web & Mobile Apps
niloyislam1187
 
Oracle Fusion HCM Cloud Demo for Beginners
DharanOracleerp
 
AutoCAD Professional Crack 2025 With License Key
youngle786g
 
Why Generative AI is the Future of Content, Code & Creativity?
Webuters Technologies Pvt. Ltd
 
Odoo Companies in India – Driving Business Transformation.pdf
azhuhardeen1968
 
AMADEUS TRAVEL AGENT SOFTWARE | AMADEUS TICKETING SYSTEM
philipnathen82
 
Nekopoi APK 2025 free lastest update
umarali35kp
 
Tally Prime Crack Download New Version 5.1 [2025] (License Key Free
itskinga12
 
CCleaner Pro 6.38.11537 Crack Final Latest Version 2025
itskinga12
 
Download FL Studio Crack Latest version 2025 ?
ghrom2211g
 
How to Make Money in the Metaverse_ Top Strategies for Beginners.pdf
Herond Labs
 
assetexplorer- product-overview - presentation
nonameist3434
 
T3DD25 TYPO3 Content Blocks - Deep Dive by André Kraus
Andre Kraus
 
Salesforce Agentforce AI Implementation.pdf
Robert Mark
 
Adobe Premiere Pro 2025 (v24.5.0.057) Crack free
ghrom2211g
 
Designing Intelligence for the Shop Floor.pdf
nikglvk
 
Ad

CS330-Lectures Statistics And Probability

  • 1. Recursion tree method Solve 22 T(n) = 2 ⋅ T(n/2) + Θ(n) T (n) T (n / 2) T (n / 2) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 4) T (n / 4) T (n / 4) T (n / 4) ⋮ inputs of length n inputs of length n/2 inputs of length n/4 base case: input n=1 T(1) T(1) ……….
  • 2. Recursion tree method — TopHat Question 2 Solve 23 T(n) = 2 ⋅ T(n/2) + Θ(n) T (n) T (n / 2) T (n / 2) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 4) T (n / 4) T (n / 4) T (n / 4) ⋮ Question. How many layers? A. constant B. log2 n C. log4 n D. n E. n2 T(1) T(1) ……….
  • 3. Recursion tree method — TopHat Question 3 Solve 24 T(n) = 2 ⋅ T(n/2) + Θ(n) T (n) T (n / 2) T (n / 2) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 4) T (n / 4) T (n / 4) T (n / 4) ⋮ Question. How many leaves? A. constant B. log2 n C. log4 n D. n E. n2 T(1) T(1) ……….
  • 4. Recursion tree method Solve 25 T(n) = 2 ⋅ T(n/2) + Θ(n) T (n) T (n / 2) T (n / 2) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 4) T (n / 4) T (n / 4) T (n / 4) ⋮ inputs of length n inputs of length n/2 inputs of length n/4 base case: input n=1 log 2 n #leaves = n T(1) T(1) ……….
  • 5. Recursion tree method — analysis idea • charge each operation to the function call (i.e. tree node) at which it is executed • for Mergesort on a subarray of length k we do O(k) work to compute the split point and do the merge • work in further recursive calls is counted separately T (n) T (n / 2) T (n / 2) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 4) T (n / 4) T (n / 4) T (n / 4) ⋮ T(n) = 2 ⋅ T(n/2) + cn where c > 0 T(1) T(1) ………. use cn instead of Θ(n)
  • 6. Recursion tree method — analysis idea • charge each operation to the function call (i.e. tree node) at which it is executed • for Mergesort on a subarray of length k we do O(k) work to compute the split point and do the merge • work in further recursive calls is counted separately T (n) T (n / 2) T (n / 2) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 4) T (n / 4) T (n / 4) T (n / 4) ⋮ T(n) = 2 ⋅ T(n/2) + cn where c > 0 cn cn/2 cn/2 T(1) T(1) ………. use cn instead of Θ(n)
  • 7. Recursion tree method — analysis idea • charge each operation to the function call (i.e. tree node) at which it is executed • for Mergesort on a subarray of length k we do O(k) work to compute the split point and do the merge • work in further recursive calls is counted separately T (n) T (n / 2) T (n / 2) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 8) T (n / 4) T (n / 4) T (n / 4) T (n / 4) T(n) = 2 ⋅ T(n/2) + cn where c > 0 cn cn/2 cn/2 cn/4 cn/4 cn/4 cn/4 cn/2k amount of work per level: c ⋅ n 2 ⋅ c ⋅ n/2 4 ⋅ c ⋅ n/4 2k ⋅ c ⋅ n/2k Θ(n) Θ(1) Total: Θ(n log n) use cn instead of Θ(n)
  • 8. binary search 29 Input: sorted array A, integer x Output: the index in A where the int x is located or “x is not in A” 1.Divide: compare x to middle element 2.Conquer: recursively search one subarray 3.Combine: trivial 2 3 7 10 11 14 16 18 20 23 example: find 16
  • 9. binary search - running time 30 Algorithm 1: BinarySearch( sorted array A, int p, int r, query ) /* find the index of query in A (if present) */ 1 q bp+r 2 c; 2 middle A[q]; 3 if query == middle then 4 return q; 5 if query < middle then 6 BinarySearch(A, p, q, query); 7 else 8 BinarySearch(A, p, q, query); 9 return q not in A;
  • 10. binary search — running time 31 Algorithm 1: BinarySearch( sorted array A, int p, int r, query ) /* find the index of query in A (if present) */ 1 q bp+r 2 c; 2 middle A[q]; 3 if query == middle then 4 return q; 5 if query < middle then 6 BinarySearch(A, p, q, query); 7 else 8 BinarySearch(A, p, q, query); 9 return q not in A; O(c) recursive call on array half the size T(n) = 1 ⋅ T(n/2) + Θ(1) #of subproblems subproblem size work dividing and combining
  • 11. find closed form by “telescoping” Reminder: our goal is to find a closed form formula for the recurrence method: • substitute the recursive formula until we get a good guess • use induction to prove the formula T(n) = T(n/2) + c = T(n/4) + 2c = … = T(n/23 ) + 3c = … = T(n/2log n ) + (log n)c = Θ(log n)
  • 12. proof by “telescoping” claim. proof. by induction on n. • base case: for n = 2 (or any constant) it takes const comparisons and log n = const • suppose that the formula is true for every k < n • proof for n: substitute k = n/2 T(n) = Θ(log n) T(n) = T(n/2) + Θ(1) = Θ(log(n/2)) + Θ(1) = Θ(log n)
  • 13. Proof by “telescoping” for mergesort Proposition. If T (n) satisfies the following recurrence, then T (n) = n log2 n. Pf. The last line corresponds to the base case. We know that the base case is T(1) = 0 by substituting log(n) for k we get 34 assuming n is a power of 2 Substitute the formula in to T(n/2) base case T(1) = T( n 2k ) =) k = log(n) T(n)  2 · T n 2 + cn  4 · T n 4 + 2cn   8 · T n 8 + 3cn  24 · T n 24 + 4cn  · · ·  2k T n 2k + kcn T(n)  2log(n) · T n 2log(n) + log(n) · n = n + O(n log n) T(n) = ( 1 n == 1 2T n 2 + cn otherwise
  • 14. Master Theorem For recurrences defined as T(n) = aT(n/b) + f(n) in which a > 0, b ≥ 1 are constants Then the overall computational cost is given by T(n) = Θ(nlogb a ) ! "# $ leaves + logb n % k=1 ak−1 f(n/bk−1 ) ! "# $ internal nodes Such that T(n) =      Θ(nlogb a) if f(n) = O(nlogb a−!) Θ(nlogb a log n) if f(n) = Θ(nlogb a) Θ(f(n)) if f(n) = Ω(nlogb a+!) and af(n/b) ≤ cf(n) in which a ≥ 1, b > 1, c < 1 and ! > 0 are constants.
  • 15. Solve the following recurrences 1. T(n) = 4T(n/2) + n 2. T(n) = 4T(n/2) + n2 3. T(n) = 4T(n/2) + n3
  • 16. Asymptotic time to execute integer operations Basic operations: adding, subtracting, multiplying, dividing Complexity of mathematical operations: • Usually in this class we count them as constant time • Today: we look at algorithms for efficiently implementing these basic operations. • Today: in our analysis one computational step is counted as one bit operation. Unit of measurement will be bit operations. Input: two n-bit long integers a , b Output: arithmetic operation on the two integers 2
  • 17. Integer addition Addition. Given two n-bit integers a and b, compute a + b. Subtraction. Given two n-bit integers a and b, compute a – b. Grade-school algorithm. Θ(n) bit operations. 3 1 1 0 1 0 1 0 1 + 0 1 1 1 1 1 0 1 2 1 3 + 1 2 5
  • 18. Integer addition Addition. Given two n-bit integers a and b, compute a + b. Subtraction. Given two n-bit integers a and b, compute a – b. Grade-school algorithm. Θ(n) bit operations. 4 1 1 1 1 1 1 0 1 1 1 0 1 0 1 0 1 + 0 1 1 1 1 1 0 1 1 0 1 0 1 0 0 1 0 2 1 3 + 1 2 5 3 3 8
  • 19. Integer addition Addition. Given two n-bit integers a and b, compute a + b. Subtraction. Given two n-bit integers a and b, compute a – b. Grade-school algorithm. Θ(n) bit operations. Remark. Grade-school addition and subtraction algorithms are asymptotically optimal. 5 an-1 an-2…a2 a1 a0 bits of a + b + bn-1 bn-2…b2 b1 b0
  • 20. Integer multiplication Multiply 21310 x 12510 = 110101012 x 011111012 ・(subscript refers to decimal and binary representation) Grade-school algorithm: 6 2 1 3 x 1 2 5 +
  • 21. Integer multiplication Multiply 21310 x 12510 = 110101012 x 011111012 ・(subscript refers to decimal and binary representation) Grade-school algorithm: 7 2 1 3 x 1 2 5 1 0 6 5 4 2 6 + 2 1 3 2 6 6 2 5
  • 22. TopHat Multiplication. Given two n-bit integers a and b, compute a × b. Grade-school algorithm. 8 1 1 0 1 0 1 0 1 × 0 1 1 1 1 1 0 1 1 1 0 1 0 1 0 1 0 0 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 0 0 0 0 0 0 0 0 0 1 1 0 1 0 0 0 0 0 0 0 0 0 0 1 Question: What is the running time? A. O(1) B. O(log n) C. O(n) D. D. (n2)
  • 23. Multiplication. Given two n-bit integers a and b, compute a × b. Grade-school algorithm. Θ(n2) bit operations. Integer multiplication 9 1 1 0 1 0 1 0 1 × 0 1 1 1 1 1 0 1 1 1 0 1 0 1 0 1 0 0 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 0 0 0 0 0 0 0 0 0 1 1 0 1 0 0 0 0 0 0 0 0 0 0 1 n i n t e r m e d i a t e products 2n additions n bits each total of O(n2) bits
  • 24. Integer multiplication 10 n i n t e r m e d i a t e products 2n additions n bits each total of O(n2) bits an-1 an-2…a2 a1 a0 n bits + bn-1 bn-2…b2 b1 b0 a x b 2n bits n bits n bits … + Multiplication. Given two n-bit integers a and b, compute a × b. Grade-school algorithm. Θ(n2) bit operations. Conjecture. [Kolmogorov 1952] Grade-school algorithm is optimal. Theorem. [Karatsuba 1960] Conjecture is wrong. (his result ) O(nlog23 ) = O(n1.59... )
  • 25. Multiplying large integers Input: n bit integers a and b Compute: ab computing ab = multiplying two n bit integers ~ O(n2) time Divide-and-conquer: ・subproblem: same problem on smaller input - input size here is the number of bits in the integers - goal: compute ab by multiplying n/2 bit integers and then combine them 11
  • 26. Divide: n bits to n/2 12 355710 = 3500 + 57 = 35 x 102 + 57 355710 = 1101111001012 = 1101110000002 + 1001012 = 1101112 x 26 + 1001012 Input: n-bit integers a , b Compute: ab How to divide n-bit integers into n/2-bit ints? Note: in decimal multiplying by 10k shifts the number k bits to the left. Same in binary, multiplying by 2k shifts k bits to the left (glues k 0s at the end.)
  • 27. TopHat Question. Write the representation of the base-3 number 1201203 as two parts of n/2 digit base-3 integers. A. B. C. D. 2 ⋅ 1203 3 ⋅ 1203 + 1203 1203 ⋅ 33 + 1203 1203 ⋅ 36 + 1203 ⋅ 33
  • 28. Divide: n bits to n/2 Input: integers a ,b of length n bits How to “divide”? What are the subproblems? ・split a in to two substrings A1, A0 of length n/2 ・then a = A12n/2 + A0 ・same for b: b = B12n/2+B0 14 a = an 1an 2 . . . an 2 an 2 1 . . . a2a1a0 A1 = an−1an−2…an 2 A0 = an 2 −1…a1a0
  • 29. Multiplying large integers using D&C Input: n bit integers a and b Compute: ab ・A1, A0, B1, B0 are n/2 bit integers ab consists of multiplying 2 n bit integers ~ O(n2) time a = A12n/2 + A0 b = B12n/2 + B0 Compute: ab = (A12n/2 + A0)(B12n/2 + B0)= 15
  • 30. Multiplying large integers using D&C Input: n bit integers a and b Compute: ab ・A1, A0, B1, B0 are n/2 bit integers ab consists of multiplying 2 n bit integers ~ O(n2) time a = A12n/2 + A0 b = B12n/2 + B0 Compute ab = A1B12n + (A1B0 + A0B1)2n/2 + A0B0 ・this formula consists of 4 multiplications of n/2 bit numbers and 3 additions Divide and conquer approach: multiply n/2 bit integers recursively 16
  • 31. 17 Multiplying large integers using D&C Algorithm 1: Multiply(ints a, b, n) 1 if n == 1 then 2 return ab; 3 m bn 2 c; 4 A0 ba/2m c /* integer division */ 5 B0 bb/2m c /* integer division */ 6 A1 a mod 2m ; 7 B1 b mod 2m ; 8 x Multiply(A1, B1, m); 9 y Multiply(A1, B0, m); 10 z Multiply(A0, B1, m); 11 w Multiply(A0, B0, m); 12 return x22m + (y + z)2m + w; recursive call on input of size n/2 Note that lines 4-7 can be implemented by simple bit-shifts — no real division is taking place! Recurrence: (reminder: ab = A1B12n + (A1B0 + A0B1)2n/2 + A0B0)
  • 32. 18 Solve the recurrence T(n)  ( 1 n = 1 4T(n 2 ) + O(n) otherwise
  • 33. 19 Solve the recurrence T(n)  ( 1 n = 1 4T(n 2 ) + O(n) otherwise Use telescoping and solve T(n) = 4 ⋅ T( n 2 ) + cn T(n) = 4 ⋅ T( n 2 ) + cn = 4 ( 4 ⋅ T( n 4 ) + c n 2) + cn = (22 )2 ⋅ T( n 22 ) + 3cn = … = (22 )3 ⋅ T( n 23 ) + 5cn = … = (22 )k ⋅ T( n 2k ) + (k + 1)cn = … we want to substitute the base case T(1), which happens when k = log n = (22 )log n T(1) + (log n)cn = n2 + Θ(1)n log n = Θ(n2 ) Now we can prove by induction that T(n) = Θ(n2 )
  • 34. Recurrence: General form: Discussed in detail: Kleinberg-Tardos pages 214 - 218 General formula for recurrences of specific form 20 T(n) = 4T ( n 2) + cn = cnlog24 = Θ(n2 ) T(n) = qT ( n 2 ) + cn = cnlog2 q for q > 2
  • 35. 21 Multiplying large integers using D&C Algorithm 1: Multiply(ints a, b, n) 1 if n == 1 then 2 return ab; 3 m bn 2 c; 4 A0 ba/2m c /* integer division */ 5 B0 bb/2m c /* integer division */ 6 A1 a mod 2m ; 7 B1 b mod 2m ; 8 x Multiply(A1, B1, m); 9 y Multiply(A1, B0, m); 10 z Multiply(A0, B1, m); 11 w Multiply(A0, B0, m); 12 return x22m + (y + z)2m + w; recursive call on input of size n/2 Note that lines 4-7 can be implemented by simple bit-shifts — no real division is taking place! Recurrence: T(n) = 4T(n 2 ) + cn T(n) = ⇥(n2 ) (reminder: ab = A1B12n + (A1B0 + A0B1)2n/2 + A0B0)
  • 36. Karatsuba’s algorithm Input: n bit integers a and b Compute: ab a = A12n/2 + A0 b = B12n/2 + B0 Compute ab = A1B12n + (A1B0 + A0B1)2n/2 + A0B0 Karatsuba’s trick: (A1+A0)(B1+B0) = A1B1 + A0B0 + (A1B0 + A0B1) 22
  • 37. Input: n bit integers a and b Compute: ab a = A12n/2 + A0 b = B12n/2 + B0 Compute ab = A1B12n + (A1B0 + A0B1)2n/2 + A0B0 Karatsuba’s trick: (A1+A0)(B1+B0) = A1B1 + A0B0 + (A1B0 + A0B1) Karatsuba’s algorithm 23
  • 38. Input: n bit integers a and b Compute: ab a = A12n/2 + A0 b = B12n/2 + B0 Compute ab = A1B12n + (A1B0 + A0B1)2n/2 + A0B0 Karatsuba’s trick: (A1+A0)(B1+B0) = A1B1 + A0B0 + (A1B0 + A0B1) Thus we get x = A1B1 , y = A0B0, z = (A1+A0)(B1+B0) ab = A1B12n + (A1B0 + A0B1)2n/2 + A0B0 = x2n + (z - x - y)2n/2 + y The new formula contains only 3 multiplications of n/2 bits. Karatsuba’s algorithm 24 multiply n/2 bit integers n/2 bits!
  • 39. 25 Karatsuba’s algorithm Algorithm 1: Karatsuba(ints a, b, n) 1 if n == 1 then 2 return ab; 3 m bn 2 c; 4 A0 ba/2m c /* integer division */ 5 B0 bb/2m c /* integer division */ 6 A1 a mod 2m ; 7 B1 b mod 2m ; 8 x Karatsuba(A1, B1, m); 9 y Karatsuba(A0, B0, m); 10 z Karatsuba(A1 + A0, B1 + B0, m); 11 return x22m + (z x y)2m + y;
  • 40. 26 Karatsuba’s algorithm - TopHat Algorithm 1: Karatsuba(ints a, b, n) 1 if n == 1 then 2 return ab; 3 m bn 2 c; 4 A0 ba/2m c /* integer division */ 5 B0 bb/2m c /* integer division */ 6 A1 a mod 2m ; 7 B1 b mod 2m ; 8 x Karatsuba(A1, B1, m); 9 y Karatsuba(A0, B0, m); 10 z Karatsuba(A1 + A0, B1 + B0, m); 11 return x22m + (z x y)2m + y; Question: What is the recurrence for this algorithm? A. C. B. D. T(n) = Θ(n) T(n) = 2 ⋅ T( n 3 ) + Θ(n) T(n) = 3 ⋅ T( n 2 ) + Θ(n) T(n) = 3 ⋅ T( n 3 ) + Θ(1)
  • 41. 27 Karatsuba’s algorithm Algorithm 1: Karatsuba(ints a, b, n) 1 if n == 1 then 2 return ab; 3 m bn 2 c; 4 A0 ba/2m c /* integer division */ 5 B0 bb/2m c /* integer division */ 6 A1 a mod 2m ; 7 B1 b mod 2m ; 8 x Karatsuba(A1, B1, m); 9 y Karatsuba(A0, B0, m); 10 z Karatsuba(A1 + A0, B1 + B0, m); 11 return x22m + (z x y)2m + y; T(n) = 3T(n 2 ) + cn T(n) = ⇥(nlog2 3 ) = ⇥(n1.59... )
  • 42. Solve where Recursion tree method 28 c 2 const cn cn 2 cn 2 cn 4 cn 4 cn 4 cn 4 + work per procedure T(n) = 3T(n 2 ) + cn cn 2 cn 4 cn 4 cn 3 2 cn 9 4 cn (3 2 )k cn
  • 43. Solve where Recursion tree method - TopHat 29 c 2 const cn cn 2 cn 2 cn 4 cn 4 cn 4 cn 4 + work per procedure T(n) = 3T(n 2 ) + cn cn 2 cn 4 cn 4 cn 3 2 cn 9 4 cn (3 2 )k cn Question: what is the hight of the tree and number of nodes at layer k? (you may assume the root is at layer 0) A. B. C. D. log3 n & nk log2 n & 3k log2 n & n3 log3 n & k3
  • 44. Solve where Recursion tree method 30 c 2 const cn cn 2 cn 2 cn 4 cn 4 cn 4 cn 4 + work per procedure T(n) = 3T(n 2 ) + cn cn 2 cn 4 cn 4 cn 3 2 cn 9 4 cn (3 2 )k cn (3 2 )log3 n cn = cnlog2 3 = cn1.59... h = log2 n (3 2 log3n ) = 1 n 3log2 n = = 3log3n ⇤ 3 1 log32 = n3log2 3
  • 45. Recurrence for Karatsuba’s: General form: Discussed in detail: Kleinberg-Tardos pages 214 - 218 General formula for recurrences of specific form 31 T(n) = 3T(n 2 ) + cn = cnlog2 3 T(n) = qT(n 2 ) + cn = cnlog2 q for q > 2
  • 46. History of asymptotic complexity of integer multiplication Remark. GNU Multiple Precision Library uses one of five different algorithms depending on size of operands. 32 year algorithm order of growth ? grade-school Θ(n2) 1962 Karatsuba–Ofman Θ(n1.585) 1963 Toom-3, Toom-4 Θ(n1.465), Θ(n1.404) 1966 Toom–Cook Θ(n1 + ε) 1971 Schönhage–Strassen Θ(n log n log log n) 2007 Fürer n log n 2 O(log*n) 2019 Harvey, van der Hoeven number of bit operations to multiply two n-bit integers ⇥(n log n) Faster than grade- school algorithm for about 320–640 bits.