Time and space complexity
Download as DOC, PDF35 likes101,241 views
The document discusses time and space complexity analysis of algorithms. Time complexity measures the number of steps to solve a problem based on input size, with common orders being O(log n), O(n), O(n log n), O(n^2). Space complexity measures memory usage, which can be reused unlike time. Big O notation describes asymptotic growth rates to compare algorithm efficiencies, with constant O(1) being best and exponential O(c^n) being worst.
Time and space complexity
- 1. TIME AND SPACE COMPLEXITY Time Complexity The total number of steps involved in a solution to solve a problem is the function of the size of the problem, which is the measure of that problem’s time complexity. some general order that we can consider (c) < O(log n) < O(n) < O(n log n) < O(nc) < O(cn) < O(n!), where c is some constant. Space Complexity Space complexity is measured by using polynomial amounts of memory, with an infinite amount of time. The difference between space complexity and time complexity is that space can be reused. Space complexity is not affected by determinism or nondeterminism. Amount of computer memory required during the program execution, as a function of the input size A small amount of space, deterministic machines can simulate nondeterministic machines, where as in time complexity, time increase exponentially in this case. A nondeterministic TM using O(n) space can be changed to a deterministic TM using only O 2(n) space. Complexity: why bother? Estimation/Prediction When you write/run a program, you need to be able to predict its needs, its requirements. Usual requirements - execution time - memory space Quantities to estimate execution time time complexity memory space space complexity It is pointless to run a program that requires: - 6TeraB of RAM on a desktop machine; - 10,000 years to run. . . You do not want to wait for an hour: - for the result of your query on Google; - when you are checking your bank account online; - when you are opening a picture file on Photoshop; - etc. It is important to write efficient algorithms
- 2. Complexity Classes Deterministic Polynomial Time P-complete Hardest problems in P solvable on parallel computers Nondeterministic polynomial time and YES answers checkable in polynomial time Nondeterministic polynomial time and NO answers checkable in Co-NP polynomial time NP- Hardest problems in NP complete Co-NP-complete Hardest problems in CO-NP NP-hard At least as hard as NP-complete problems NC Solvable parallel computation efficiency PSPACE Polynomial memory with unlimited time PSPACE- complete Hardest problems in PSPACE EXPTIME Exponential time EXPSPACE Exponential memory with unlimited time BQP Polynomial time on a quantum computer Find the greatest common divisor (GCD) of two integers, m and n. Program (in C): int gcd(int m, int n) /* precondition: m>0 and n>0.Let g=gcd(m,n). */ { while( m > 0 ) { /* invariant: gcd(m,n)=g */ if( n > m ) { int t = m; m = n; n = t; } /* swap */ /* m >= n > 0 */ m -= n; } return n; } At the start of each iteration of the loop, either n>m or m?n. (i) If m?n, then m is replaced by m-n which is smaller than the previous value of m, and still non-negative. (ii) If n>m, m and n are exchanged, and at the next iteration case (i) will apply.
- 3. So at each iteration, max(m,n) either remains unchanged (for just one iteration) or it decreases. This cannot go on for ever because m and n are integers (this fact is important), and eventually a lower limit is reached, when m=0 and n=g. So the algorithm does terminate. Good test values would include: • special cases where m or n equals 1, or • m, or n, or both equal small primes 2, 3, 5, …, or • products of two small primes such as p1×p2 and p3×p2, • some larger values, but ones where you know the answers, • swapped values, (x,y) and (y,x), because gcd(m,n)=gcd(n,m). The objective in testing is to "exercise" all paths through the code, in different combinations. We can also consider the best,average and worst cases. Average vs. worst-case complexity Definition (Worst-case complexity) The worst-case complexity is the complexity of an algorithm when the input is the worst possible with respect to complexity. Definition (Average complexity) The average complexity is the complexity of an algorithm that is averaged over all the possible inputs (assuming a uniform distribution over the inputs). We assume that the complexity of the algorithm is T(i) for an input i. The set of possible inputs of size n is denoted In. Big-O Notation Analysis of Algorithms Big Oh Notation- A convenient way of describing the growth rate of a function and hence the time complexity of an algorithm. Let n be the size of the input and f (n), g(n) be positive functions of n. The time efficiency of almost all of the algorithms we have discussed can be characterized by only a few growth rate functions: I. O(l) - constant time This means that the algorithm requires the same fixed number of steps regardless of the size of the task. Examples (assuming a reasonable implementation of the task):
- 4. A. Push and Pop operations for a stack (containing n elements); B. Insert and Remove operations for a queue. II. O(n) - linear time This means that the algorithm requires a number of steps proportional to the size of the task. Examples (assuming a reasonable implementation of the task): A. Traversal of a list (a linked list or an array) with n elements; B. Finding the maximum or minimum element in a list, or sequential search in an unsorted list of n elements; C. Traversal of a tree with n nodes; D. Calculating iteratively n-factorial; finding iteratively the nth Fibonacci number. III. O(n2) - quadratic time The number of operations is proportional to the size of the task squared. Examples: A. Some more simplistic sorting algorithms, for instance a selection sort of n elements; B. Comparing two two-dimensional arrays of size n by n; C. Finding duplicates in an unsorted list of n elements (implemented with two nested loops). IV. O(log n) - logarithmic time Examples: A. Binary search in a sorted list of n elements; B. Insert and Find operations for a binary search tree with n nodes; C. Insert and Remove operations for a heap with n nodes. V. O(n log n) - "n log n " time Examples: A. More advanced sorting algorithms - quicksort, mergesort VI. O(an) (a > 1) - exponential time Examples: A. Recursive Fibonacci implementation B. Towers of Hanoi C. Generating all permutations of n symbols The best time in the above list is obviously constant time, and the worst is exponential time which, as we have seen, quickly overwhelms even the fastest computers even for relatively small n. Polynomial growth (linear, quadratic, cubic, etc.) is considered manageable as compared to exponential growth. Order of asymptotic behavior of the functions from the above list: Using the "<" sign informally, we can say that O(l) < O(log n) < O(n) < O(n log n) < O(n2) < O(n3) < O(an)
- 5. A word about O(log n) growth: As we know from the Change of Base Theorem, for any a, b > 0, and a, b != 1 Therefore, loga n = C logb n where C is a constant equal to loga b. Since functions that differ only by a constant factor have the same order of growth, O(log2 n) is the same as O(log n). Therefore, when we talk about logarithmic growth, the base of the logarithm is not important, and we can say simply O(log n). A word about Big-O when a function is the sum of several terms: If a function (which describes the order of growth of an algorithm) is a sum of several terms, its order of growth is determined by the fastest growing term. In particular, if we have a polynomial p(n) = aknk + ak-1nk-1 + … + a1n + a0 its growth is of the order nk: p(n) = O(nk) Example: