Algorithms Analysis.pdf
- 2. Algorithm Analysis • The purpose of algorithm analysis is, in general , to determine the performance of the algorithm in terms of time taken and storage requirements to solve a given problem. • An other objective can be to check whether the the algorithm produces consistent, reliable, and accurate outputs for all instances of the problem .It may also ensured that algorithm is robust and would prove to be failsafe under all circumstances. • The common metrics that are used to gauge the performance are referred to as time efficiency, space efficiency, and correctness. Objectives 15
- 3. Algorithm Analysis i. Count of data items in data collections such as arrays, queues ii. Number of nodes in trees and linked lists iii. Number of vertices and number of edges in a graph iv. Number of rows and columns in an input table v. Character count in an input text block Input Size • The performance is often expressed in terms problem size, or more precisely, by the number of data item items processed by an algorithm. • The key parameters used in the analysis of algorithms for some common application types are: 16
- 4. Analysis of Algorithm • The time efficiency determines how fast an algorithm solves a given problem. Quantitatively, it is the measures of time taken by the algorithm to produce the output with a given input. The time efficiency is denoted by a mathematical function of the input size. • Assuming that an algorithm takes T(n) time to process n data items. The function T(n) is referred to as the running time of the algorithm. It is also known as time complexity. The time complexity can depend on more than one parameter. For example, the running time of a graph algorithm can depend both on the number of vertices and edges in the graph. • The running time is an important indicator of the behavior of an algorithm for varying inputs. It tells, for example, whether the time taken to process would increase in direct proportional to input size, would increase four fold if size is doubled, or increase exponentially. • It would be seen that that time efficiency is the most significant metric for algorithm analysis. For this reason, the main focus of algorithm analysis would be on determining the time complexity. Time Efficiency 17
- 5. Time Efficiency Approaches The time efficiency can be determined in several ways . The common methods are categorized as empirical, analytical, and visualization • In empirical approach , the running time is determined experimentally. The performance is measured by testing the algorithm with inputs of different sizes • The analytical method uses mathematical and statistical techniques to examine the time efficiency. The running time is expressed as mathematical function of input size • The visualization technique is sometimes used to study the behavior and performance of an algorithm by generating graphics and animation, based on interactive inputs . 18
- 6. Empirical Analysis Methodology 1) The algorithm is coded and run on a computer. The running times are measured, by using some timer routine , with inputs of different sizes.. 2) The output is logged and analyzed with the help of graphical and statistical tools to determine the behavior and growth rate of running time in terms of input size. 3) A best curve is fitted to depict trend of the algorithm in terms input sizes The empirical methodology broadly consists of following steps
- 7. Empirical Analysis • The graph illustrates the results of an empirical analysis. The running times of a sorting Sorting Time Example • The analysis indicates that time increases roughly in proportion to the square of input size, which means that doubling the input size increases the running time four fold. algorithm are plotted against the number input size. The measured values are shown in red dots. The graph shows the best-fit curve to scattered points.
- 8. Empirical Analysis Limitations The empirical analysis provides estimates of running times in a real life situation. It has, however, several limitations, because the running time crucially depends on several factors. Some key factors that influence the time measurements are: • Hardware types ( CPU speed, IO throughput, RAM size etc.) • Software environment (Compiler, Programming Language etc.) • Program design ( Conventional, Structured, Object Oriented) • Composition of data set sets ( Choice of data values and the range of input )
- 9. Analytical Analysis In analytic approach the running time is estimated by studying and analyzing the basic or primitive operations involved in an algorithm. Broadly, the following methodology is adopted: Methodology • The code for the algorithm is examined to identify basic operations • The number of times each basic operation is executed, for a given input, is determined. • The running time is estimated by taking into consideration the frequency and cost of significant operations • The total time is expressed as a function of the input size
- 10. Analytical Analysis i. Assigning value to a variable ii. Comparing a pair of data items iii. Incrementing a variable iv. Performing arithmetic and floating point operations v. Moving a data item from one storage location to another location vi. Calling a procedure vii. Returning a value viii. Accessing an array element Basic Operations • The code for an algorithm, generally, consists of a mix of following basic operations:. • The time taken by the basic operation to complete a single step is often referred to as the cost of the operation. Some operations are relatively more expensive than others. For example, operations involving data movement and arithmetical computations are costly compared to logical or assignment operations .
- 11. Running Time Classification • Let k denote all possible orderings of input of size n, and T1(n),T2(n),..Tk(n)) be the running times for each instance . We can formally, define the best, worst and average running times, Tbest(n), Tworst(n), Taverage(n), as follows Best Case: In this case the algorithm has minimum running time. : Tbest(n) = minimum(T1,T2,…Tk) This is also called the optimistic time Worst Case: In this case the algorithm has maximum running time Tworst(n) = maximum(T1,T2,…Tk) This is also known as pessimistic time Average Case: The average running time is the average of running times for all possible ordering of inputs of the same size: Taverage(n) = (T1+T2+….+Tk) / k Worst, Best, Average Cases • We have seen that the running time of an algorithm depends on the input size. For some applications, the running time also depends on the order in which the data items are input to the algorithm.
- 12. Best, Worst, Average Cases • The graph shows best, worst, and average running times of an algorithm. In this example, times are shown for 20 inputs of same size but in different order. The algorithm takes minimum time to process Input #6 , and maximum time to process Input #17 , which are referred to as best and worst times. The average of all the running times for 20 inputs is also shown. • For an accurate analysis , all possible arrangements of input should be considered to identify the best, worst, and average running times Example
- 13. Algorithm Analysis Space Efficiency • The space efficiency determines total memory requirement of RAM and disk storage to run an algorithm with given input. • The space requirement consists of the amount of real storage needed to execute the algorithm code and the storage to hold the application data. The algorithm code occupies a fixed amount of space, which is independent of the input size. The storage requirement for the application depends on the nature of data structure used to provide faster and flexible access to stored information . For an arrays and linked lists, for example, the space requirement is directly proportional to the input size. • For most algorithms, the space efficiency is not of much concern. However, some algorithms require extra storage to hold results of intermediate computations. This is the case, for example, with merge sort and dynamic programming techniques.
- 14. Algorithm Correctness ƒ A loop invariant is set of conditions and relationships that remain true prior to, during , and after the execution of a loop. ƒ The loop invariant condition / statement depends on the nature of problem being analyzed In a sorting problem, for example, the condition might be the order of keys in a sub-array, which should remain in ascending/descending order ,prior to, and after the execution of each iteration Loop Invariant Method ƒ There are no standard methods for proving the correctness of a given algorithm. However, many useful algorithms consist of one or more iterative computations, using loop structures. The correctness of such algorithms can be formally established by a technique called Loop Invariant Method .
- 15. Algorithm Correctness The loop invariant method establishes the correctness of algorithm in three steps, which are known as initialization, maintenance, and termination ( Reference: T. Cormen et al ) At each step, the loop invariant is examined. If the loop conditions at each of the steps hold true, then algorithm is said be correct. Initialization: Loop invariant is true prior to execution of first iteration of the loop Maintenance: If loop invariant is assumed to be true at some iteration, it remains true after the next iteration Termination: After the termination of the loop, the invariant holds true for the problem size Formal Proof
- 16. Algorithm Analysis Algorithm visualization techniques are used to study • Studying inner working of an algorithm through trace of basic operations • Illustrating algorithm steps with animations • Studying algorithm performance with interactive inputs • Counting primitive operations for analysis • Doing simulations with a variety of data sets • Reporting on the performance Visualization