DSA Complexity.pptx What is Complexity Analysis? What is the need for Complexity Analysis? Asymptotic Notations How to measure complexity?

Design and analysis of
algorithm
Unit 1: introduction
Shikha Sharma

Algorithm
“ An algorithm is any well – defined computation procedure that takes
some values or set of values, as input and produce some value or set of
values as output.”
Computation: Arithmetic ,Logics and Reasoning.
Problem:
20Km
10 Km
5 Km
Step wise solution of above problem:
Step 1: Start from point A.
Step 2: Travel 10 km in straight direction.
Step3: Take a right turn and move 5 km.
Step 4: Move 20 km straight.
Step 5: Stop at point B.

Features of algorithm:
Not all procedures can be called an algorithm. An algorithm should
have the below mentioned characteristics −
• Unambiguous − Algorithm should be clear and unambiguous. Each
of its steps (or phases), and their input/outputs should be clear and
must lead to only one meaning.
• Input − An algorithm should have 0 or more well defined inputs.
• Output − An algorithm should have 1 or more well defined outputs,
and should match the desired output.
• Finiteness − Algorithms must terminate after a finite number of steps.
• Feasibility − Should be feasible with the available resources.
• Independent − An algorithm should have step-by-step directions
which should be independent of any programming code.

Expressing Algorithm
An algorithm can be represented in different ways – a flow chart , pseudo code,
simple English sentences , computer program etc.
For representing algorithm we use following conventions:
1.Every algorithm must have a name.
2.The input provided to the algorithm are written in parentheses along with its
name.
3. Any assignment statement is expressed using the assignment operator”:=“ .
Like , if we wish to assign value 5 to variable a, then it is written as a:=5.
4.The symbol ‘//’ means a comment.
5.A condition statement is written using if – then statement and if- then –else
statement . End if is used to mark end of if statement.
6.Loops are represented by while –End while structure and for –End for
structure syntax is: For<variable>:= <initial value> to <final value> step
<size>
7.Array indices are expressed within square brackets, as Arr[i].
8. Result are returned using return <value> statement.

Example
Consider an algorithm for finding sum of n numbers, stored in an array. The
algorithm in simple English sentences is:
Step 1: take n numbers
Set sum to zero.
Repeat step 4 for first number to nth number.
Add number to sum.
Return sum.
We convert this algorithm into pseudo code using the conventions.
//Nums is an array of numbers and n is size of array and initialize sum to zero.
1.sum: =0
2.For i:= 1 to n
3.sum: sum +Nums[i]
4.End For
5.Return sum

Analysis and complexity
There are various ways available to convert algorithm into a computer
program. But selection of an algorithm is guarded by following facts:
1.Memory of a computer may be very large , but it is finite . So we
need an algorithm which does not require much space.
2.Computer are used for saving our time. We cannot adopt an
algorithm which take lots of time to solve a particular problem. Thus ,
runtime of an algorithm is an important criterion.
• A decision is to be made whether to save time or save space. This is
called time vs space trade off.
• To make a good decision, first we need some device to measure how
much time an algorithm will take to finish or how much space it will
consume.
• We cannot calculate exact value , but can calculate estimate value at
least in order to compare two algorithm.

Analysis
Analysis of algorithm or performance analysis refer to theoretical
estimate of the resources, like computing time and storage, an
algorithm require in different situations.
This will provide quantitative judgment about the performance of one
algorithm over the another.
The analysis can be:
1.Exact analysis (using turing machine)
2.Asymptotic analysis
3.Worst case analysis
4.Average case analysis
5.Probabilistic

Complexity
The complexity means how complex an algorithm is, which can be reflected
as relative amount of time or space they require.
It gives the idea about the cost of running an algorithm.
The cost can be either in terms of time and space occupied .
It can be of two types:
Time complexity: It is defined as an estimate of time required by an algorithm
to run to completion.
Suppose input size n , so number of steps require by algorithm to solve an
instance of size n is called time complexity of algorithm in terms of n. The
algorithm with lesser time complexity will save time and run faster than
others.
Space complexity: It is an estimate of space in memory that the algorithm
require to complete the task.
That is amount of storage needed while solving a problem using the algorithm
is called space complexity of that algorithm. An algorithm with lesser space
complexity will require less memory than others.

Asymptotic notation
“
Asymptotic notation of an algorithm is a mathematical representation of its complexity.
It is also known as Algorithm's growth rate.
They are of different types:
1. Theta (Θ) Notation:
2. Big O Notation:
3. Big-Ω (Big-Omega) notation:

ASYMPTOTIC NOTATION
Big O Notation: (Worst Case)
The notation Ο(n) is the formal way to express the upper bound of
an algorithm's running time. It measures the worst case time
complexity or the longest amount of time an algorithm can
possibly take to complete.
Ο(f(n)) = { g(n) : there exists c > 0 and n0 such that f(n) ≤ c.g(n) for all
n > n0. }
EG. 3n+2

ASYMPTOTIC NOTATION
Theta (Θ) Notation: (Average Case) The notation θ(n) is the formal
way to express both the lower bound and the upper bound of an
algorithm's running time.
θ(f(n)) = { g(n) if and only if g(n) = Ο(f(n)) and g(n) = Ω(f(n)) for all
n > n0. }

ASYMPTOTIC NOTATION
Big-Ω (Big-Omega) notation: (Best Case)
The notation Ω(n) is the formal way to express the lower bound of
an algorithm's running time. It measures the best case time
complexity or the best amount of time an algorithm can possibly take
to complete.
Ω(f(n)) ≥ { g(n) : there exists c > 0 and n0 such thatc. g(n) ≤ f(n) for all
n > n0. }

Evaluating complexities
Time complexity is calculated using following rules:
• Every assignment is of O(1).
• Every procedure entry (function call) is of O(1)
• Every procedure exit (function return) is of O(1)
• For any if statement total time is the time taken for checking condition added
with O(max of the two branches)
• For a loop we total up the time taken for each iteration.
• Put all these together using sum rule and product rule.
Space complexity is calculate by following rules:
• Every variable of basic data type occupies 1 unit of memory.
• An array occupies n unit of memory , where n is size of array.
• A record or structure occupies memory equal to sum of memory units of its
data members.
• Only space occupied by variable , input ,output is considered.

How to calculate complexity…
Q1. Calculate time complexity of following code
begin
for i:=1 to n do
sum:= sum+i;
end;
Solution: the for loop will run n times, and hence the statement will
also be executed n times.
Let the time taken to execute inner statements be k (because it is
constant) , then the running time can be calculated as follows:
T(n)=k+k+k……. n times
=kn
=O(n)

Q2. Evaluate time complexity for the following code:
begin
for i: 1 to n do
for j: 1 to m do
sum:= sum+i+j;
end;
Solution: inner for loop (variable j) will run m times, hence the time taken by
this loop will be
k+k+k….. m times =km
The outer loop will run n times , but it contains the inner loop. As many times
outer loop runs, it executes the inner loop fully for every iteration.
Hence total times taken will be
T(n)= km+km+km…. n times
=kmn
=O(mn) if (m=n) then T(n)= O(𝑛2
)

Q3 . Consider this code with nested loops:
begin
for i: = 1 to n do
for j: = 1 to I do
sum := sum +i+j;
end;
Solution: the outer for loop will run n times , but inner loop will run i times
which is varying with every iteration.
When i=1 inner loop runs once , when i=2 inner loop runs twice ; when i=3
inner loop runs thrice and so on. Hence total time taken will be
T(n)= 1+2+3…n
=n(n+1)/2 (A.P.)
=O(𝑛2
)

Q4 . For following code to read an array of n numbers and calculate the
sum , evaluate time and space complexity.
Solution:
begin
read Arr[1….n]
for i: = 1 to n do
sum:= sum+Arr[i];
end;
Solution : The step of reading the array will take input n numbers,
hence take n units of time. It is followed by for loop, which run n times.
Inner statement takes one unit of time to complete.
T(n)= time to read+time in for loop =n+n =O(n)
For space complexity , the space occupied by Arr is n units , variable I
and sum occupy one unit each. S(n)= n+1+1 =O(n)

Q5. Evaluate time and space complexity of a matrix multiplication algorithm for multiplying two
square matrix of size n*n.
Solution:
matrix –multiplication(A,B)
Step 1:for i= 1 to n
Step2:for j:= 1 to n
Step3:c[i,j]= 0
Step4:for i=1 to n
Step5:for j:= 1 to n
Step6:for k:= 1 to n
Step7:c[i,j]=c[i,j]+a[i,k]*b[j,k];
Step8:end for
Step9:end for
Step10:end for
Step11:return c

Solution:
The initialization steps 1 to 3 are a nested loop of n size each. Hence, total number of
iterations is n*n=𝑛2
.
The actual multiplication steps 4 to 9 are nested loops of n size.
Hence total number of iteration is n*n*n=𝑛3
.
Time taken for calculation in step 7 is 3 units( one multiplication, one addition, one
assignment).
Thus , total time taken by the algorithm is :
T(n)= 1.𝑛2
+3.𝑛3
= O(𝑛3
)
Time complexity of matrix multiplication algorithm . O(𝑛3
)
Space complexity:
The algorithm needs memory space for matrix A, B, and C and for variables i ,j ,and k.
Space occupied by any matrix of size n*n=𝑛2
. We have three such matrix.
Space occupied by other three variable is one unit each. Thus ,the space complexity of
algorithm can be expressed using sum rule.
S(n)=3.𝑛2
+ 3.1 =O(𝑛2
)
Space complexity of matrix multiplication algorithm is O(𝑛2
)

DSA Complexity.pptx What is Complexity Analysis? What is the need for Complexity Analysis? Asymptotic Notations How to measure complexity?

DSA Complexity.pptx What is Complexity Analysis? What is the need for Complexity Analysis? Asymptotic Notations How to measure complexity?

More Related Content

Similar to DSA Complexity.pptx What is Complexity Analysis? What is the need for Complexity Analysis? Asymptotic Notations How to measure complexity? (20)

Recently uploaded (20)

DSA Complexity.pptx What is Complexity Analysis? What is the need for Complexity Analysis? Asymptotic Notations How to measure complexity?