Introduction to Algorithms and Asymptotic Notation
17 likes10,331 views
The document outlines the course 'CS 6212 – Design and Analysis of Algorithms' taught by Professor Amrinder Arora, detailing logistics, course goals, and structure. It covers key topics like algorithm design, analysis, asymptotic notation, and specific algorithms such as insertion sort and binary search. Additionally, it emphasizes the importance of analyzing algorithms for performance estimation and introduces various types of asymptotic notations used in algorithm comparison.
![Given: An array A of n numbers
Purpose: To sort the array
Algorithm:
for j = 1 to n-1
key = A[j]
// A[j] is added in the sorted sequence A[1.. j-1]
i = j - 1
while i >= 0 and A [i] > key
A[ i + 1] = A[i]
i = i - 1
A [i +1] = key
7
EXAMPLE 1: INSERTION SORT
Algorithms Introduction and Asymptotic Notation](https://image.slidesharecdn.com/l1-140817125248-phpapp01/85/Introduction-to-Algorithms-and-Asymptotic-Notation-7-320.jpg)
![BinarySearch(A[0..N-1], value, low, high)
{
if (high < low)
return -1 // not found
mid = (low + high) / 2
if (A[mid] > value)
return BinarySearch(A, value, low, mid-1)
else if (A[mid] < value)
return BinarySearch(A, value, mid+1, high)
else return mid // found
}
9
EXAMPLE 3: BINARY SEARCH
Algorithms Introduction and Asymptotic Notation](https://image.slidesharecdn.com/l1-140817125248-phpapp01/85/Introduction-to-Algorithms-and-Asymptotic-Notation-9-320.jpg)
![j = 1
while (j < n) {
k = 2
while (k < n) {
Sum += a[j]*b[k]
k = k * k
}
j++
}
(n log log n)
For (int j = 1 to n) {
For (int k = j to n) {
x = k
While (x < n) {
Sum +=
a[j]*b[k]*c[x]
If (x % 3 == 0) {
x = n + 1
}
x = x + 1
}
}
}
For (int j = 1 to n) {
k = j
while (k < n) {
Sum += a[k]*b[k]
k += log n
}
}
Algorithms Introduction and Asymptotic Notation 13
WHAT IS THE TIME COMPLEXITY OF THIS
PROGRAM?
Possible Quiz Questions](https://image.slidesharecdn.com/l1-140817125248-phpapp01/85/Introduction-to-Algorithms-and-Asymptotic-Notation-13-320.jpg)
Introduction to Algorithms and Asymptotic Notation
- 1. CS 6212 – Design and Analysis of Algorithms INTRODUCTION AND ASYMPTOTIC NOTATION
- 2. Instructor Prof. Amrinder Arora [email protected] Please copy TA on emails Please feel free to call as well Available for study sessions Science and Engineering Hall GWU Algorithms Introduction and Asymptotic Notation 2 LOGISTICS
- 3. Algorithms Analysis Asymptotic NP- Completeness Design D&C DP Greedy Graph B&B Applications Algorithms Introduction and Asymptotic Notation 3 COURSE OUTLINE
- 4. Design and Analysis of Algorithms Designing – Algorithmic Techniques Analyzing – how much time an algorithm takes Proving inherent complexity of problems 4 PURPOSE OF THIS CLASS Algorithms Introduction and Asymptotic Notation
- 5. A precise statement to solve a problem on a computer A sequence of definite instructions to do a certain job 5 “ALGORITHM” – DEFINITIONS Algorithms Introduction and Asymptotic Notation
- 6. Pseudo-code consisting of: Variables Assignments Arrays Reading/Writing data Loops Switch/Case Function/Procedure Begin/End Algorithms Introduction and Asymptotic Notation 6 REPRESENTING AN ALGORITHM
- 7. Given: An array A of n numbers Purpose: To sort the array Algorithm: for j = 1 to n-1 key = A[j] // A[j] is added in the sorted sequence A[1.. j-1] i = j - 1 while i >= 0 and A [i] > key A[ i + 1] = A[i] i = i - 1 A [i +1] = key 7 EXAMPLE 1: INSERTION SORT Algorithms Introduction and Asymptotic Notation
- 8. Best case running time Worst case running time Average case running time 8 EXAMPLE 2: EUCLID’S ALGORITHM (CONT.) gcd(n,m) { r = n%m if r == 0 return m // else return gcd(m, r) } Algorithms Introduction and Asymptotic Notation
- 9. BinarySearch(A[0..N-1], value, low, high) { if (high < low) return -1 // not found mid = (low + high) / 2 if (A[mid] > value) return BinarySearch(A, value, low, mid-1) else if (A[mid] < value) return BinarySearch(A, value, mid+1, high) else return mid // found } 9 EXAMPLE 3: BINARY SEARCH Algorithms Introduction and Asymptotic Notation
- 10. How long will the algorithm take? How much memory will it require? For Example: Function sum (array a) { sum = 0; for (int j : a) { sum = sum + j } return sum; } 10 ANALYZING AN ALGORITHM Algorithms Introduction and Asymptotic Notation
- 11. Why to do it? A priori estimation of performance To compare algorithms on a level playing field How to do it? (What is the model?) Random access memory model Math operations take constant time Read/write operations take constant time What do we compute? Time complexity: # of operations as a function of input size Space complexity: # of bits used 11 ANALYZING AN ALGORITHM Algorithms Introduction and Asymptotic Notation
- 12. How to Analyze a Given Algorithm (Program) Some tips: When analyzing an if then else condition, consider the arm that takes the longest time When considering a loop, take the sum When considering a nested loop, … Algorithms Introduction and Asymptotic Notation 12 ANALYZING ALGORITHMS
- 13. j = 1 while (j < n) { k = 2 while (k < n) { Sum += a[j]*b[k] k = k * k } j++ } (n log log n) For (int j = 1 to n) { For (int k = j to n) { x = k While (x < n) { Sum += a[j]*b[k]*c[x] If (x % 3 == 0) { x = n + 1 } x = x + 1 } } } For (int j = 1 to n) { k = j while (k < n) { Sum += a[k]*b[k] k += log n } } Algorithms Introduction and Asymptotic Notation 13 WHAT IS THE TIME COMPLEXITY OF THIS PROGRAM? Possible Quiz Questions
- 14. Sets, functions Logs and Exponents Recurrence Relations Sums of series Arithmetic Progression: 1 + 2 + 3 + … + n Geometric Progress: 1 + 3 + 9 + … 3k AGP: 1 + 2.3 + 3.9 + … (k+1) 3k Others: 12 + 22 + 32 + … + n2, Harmonic Series 14 REQUIRED MATH CONSTRUCTS Algorithms Introduction and Asymptotic Notation
- 15. Simulation/Runoff (on worst case/average case) •Provides incomplete picture •Works for complex functions Analysis and comparison of Asymptotic notation •Provides a complete theoretical understanding •May not work for very complex algorithms 15 COMPARING ALGORITHMS Algorithms Introduction and Asymptotic Notation
- 16. Big O notation f(n) = O(g(n)) if there exist constants n0 and c such that f(n) ≤ c g(n) for all n ≥ n0. For example, consider f(n) = n, and g(n) = n2. Then, f(n) = O(g(n)) If f(n) = a0 n0 + a1 n1 + … + am nm, then f(n) = O (nm) Big Omega notation f(n) = Ω(g(n)) if there exist constants n0 and c such that f(n) ≥ c g(n) for all n ≥ n0. 16 ASYMPTOTIC NOTATION Algorithms Introduction and Asymptotic Notation
- 17. Small o notation f(n) = o(g(n)) if for any constant c > 0, there exists n0 such that 0 ≤ f(n) < c g(n) for all n ≥ n0. For example, n = o(n2) Small omega () notation f(n) = (g(n)) if for any constant c > 0, there exists n0 such that f(n) ≥ c g(n) ≥ 0, for all n ≥ n0 For example, n3 = (n2) 17 ASYMPTOTIC NOTATIONS (CONT.) Algorithms Introduction and Asymptotic Notation
- 18. f(n) = O(g(n)) if and only if g(n) = Ω(f(n)) If f(n) = O(g(n)) and g(n) = O(f(n)), then f(n) = (g(n)) f(n) = o(g(n)) if and only if g(n) = (f(n)) f(n) = o(g(n)) implies limnf(n)/g(n) = 0 18 ASYMPTOTIC NOTATIONS (CONT.) Algorithms Introduction and Asymptotic Notation
- 19. In some cases, we need to use the L’Hopital’s rule to prove the small oh notation. L’Hopital’s rule states that assuming certain conditions hold, For example, suppose we want to prove the following: (log n)3 + 3 (log n)2 = o(n) We can find the limit of these functions as follows: limn(log n)3 + 3 (log n)2 / n = limn6 (log n)2 + 12 log n / n1/2 // Using L’Hopital’s rule = limn24 log n + 24/ n1/2 // Using L’Hopital’s rule = limn48/ n1/2 // Using L’Hopital’s rule = 0 Algorithms Introduction and Asymptotic Notation 19 PROVING SMALL OH USING L’HOPITAL’S RULE
- 20. O o Ω ≤ < = > ≥ 20 ASYMPTOTIC NOTATIONS (CONT.) Analogy with real numbers Algorithms Introduction and Asymptotic Notation
- 21. Which properties apply to which (of 5) asymptotic notations? Transitivity Reflexivity Symmetry Transpose Symmetry Trichotomy 21 ASYMPTOTIC NOTATIONS (CONT.) Algorithms Introduction and Asymptotic Notation
- 22. Which properties apply to which (of 5) asymptotic notations? Transitivity: O, o, , , Ω Reflexivity: O, , Ω Symmetry: Transpose Symmetry: (O with Ω, o with ) Trichotomy: Does not hold. For real numbers x and y, we can always say that either x < y or x = y or x > y. For functions, we may not be able to say that. For example if f(n) = sin(n) and g(n)=cos(n) 22 ASYMPTOTIC NOTATIONS (CONT.) Algorithms Introduction and Asymptotic Notation
- 23. LOGISTICS Rigor required by this class Saturday study sessions (Optional) Study sessions on other days – You need to coordinate Grading – Review course information sheet in Blackboard 23Algorithms Introduction and Asymptotic Notation
- 24. SOME LESSONS FROM PREVIOUS CLASSES Almost no correlation between grades and background Correlation between grades and number of classes attended Correlation between grades and time spent on course Strong correlation between grades and homeworks/projects 24Algorithms Introduction and Asymptotic Notation
- 25. 80% OF SUCCESS IS SHOWING UP. Algorithms Introduction and Asymptotic Notation 25
- 26. Review Project P1 Review HW1 Review Course Outline Algorithms Introduction and Asymptotic Notation 26 READING ASSIGNMENTS
- 27. http://en.wikipedia.org/wiki/Arithmetic_progression http://en.wikipedia.org/wiki/Geometric_series https://www.boundless.com/algebra/textbooks/boundless- algebra-textbook/sequences-series-and-combinatorics- 8/sequences-and-series-53/introduction-to-sequences-224- 5904/ http://en.wikipedia.org/wiki/L%27H%C3%B4pital%27s_rule https://www.youtube.com/watch?v=PdSzruR5OeE Algorithms Introduction and Asymptotic Notation 27 HELPFUL LINKS