SlideShare a Scribd company logo
Data Structures and Algorithms
Lecture 1: Asymptotic Notations
Basic Terminologies
• Algorithm
– Outline
– Essence of a computational procedure
– Step by step instructions

• Program
– Implementation of an Algorithm in some
programming language

• Data Structure
– Organization of Data needed to solve the problem
effectively
– Data Structure you already know: Array
Algorithmic Problem
Specification of
Input

?

Specification of
Output as a
function of Input

• Infinite number of input instances satisfying the
specification
• E.g.: Specification of Input:
• A sorted non-decreasing sequence of natural numbers
of non-zero, finite length:
• 1, 20,908,909,1000,10000,1000000
• 3
Algorithmic Solution
Input instance
adhering to
Specification

Algorithm

Output related
to the Input as
required

• Algorithm describes the actions on the input
instances
• Infinitely many correct algorithms may exist
for the same algorithmic problem
Characteristics of a Good Algorithm
• Efficient
– Small Running Time
– Less Memory Usage

• Efficiency as a function of input size
– The number of bits in a data number
– The number of data elements
Measuring the running time
• Experimental Study
– Write a program that implements the algorithm
– Run the program with data sets of varying size and
composition
– Use system defined functions like clock() to get an
measurement of the actual running time
Measuring the running time
• Limitations of Experimental Study
– It is necessary to implement and test the
algorithm in order to determine its running time.
– Experiments can be done on a limited set of
inputs, and may not be indicative of the running
time on other inputs not included in the
experiment
– In order to compare 2 algorithms same hardware
and software environments must be used.
Beyond Experimental Study
• We will develop a general methodology for
analyzing the running time of algorithms. This
approach
– Uses a high level description of the algorithm
instead of testing one of its implementations
– Takes into account all possible inputs
– Allows one to evaluate the efficiency of any
algorithm in a way that is independent of the
hardware and software environment.
Pseudo - Code
• A mixture of natural language and high level
programming concepts that describes the main
ideas behind a generic implementation of a data
structure or algorithm.
• E.g. Algorithm arrayMax(A,n)
– Input: An array A storing n integers
– Output: The maximum element in A.
currMax
A[0]
for i
1 to n-1 do
if currMax < A[i] then currMax
return currMax

A[i]
Pseudo - Code
• It is more structured that usual prose but less
formal than a programming language
• Expressions:
– Use standard mathematical symbols to describe
numeric and Boolean expressions
– Use ‘
’ for assignment instead of “=”
– Use ‘=’ for equality relationship instead of “==”

• Function Declarations:
– Algorithm name (param1 , param2)
Pseudo - Code
• Programming Constructs:
–
–
–
–
–

Decision structures: if…then….[else….]
While loops: while…..do…..
Repeat loops: Repeat………until…….
For loop: for……do………..
Array indexing: A[i], A[I,j]

• Functions:
– Calls: return_type function_name(param1 ,param2)
– Returns: return value
Analysis of Algorithms
• Primitive Operation:
– Low level operation independent of programming
language.
– Can be identified in a pseudo code.

• For e.g.
– Data Movement (assign)
– Control (branch, subroutine call, return)
– Arithmetic and Logical operations (e.g. addition
comparison)
– By inspecting the pseudo code, we can count the
number of primitive operations executed by an
algorithm.
Example: Sorting
OUPUT
Permutation of the sequence of
numbers in non-decreasing order

INPUT
Sequence of
numbers

a1,a2,a3,a4,….an
2, 5 , 4, 10, 7

Sort

Correctness (Requirements for the
output)
For any given input the algorithm
halts with the output:
• b1 < b2 < b3 < b4 < ….. < bn
• b1, b2, b3, b4,…. bn is a permutation
of a1, a2, a3, a4, …. an

b1,b2,b3,b4,….bn
2, 4 , 5, 7, 10
Running Time depends on
• No. of elements (n)
• How sorted (partial) the numbers are?
• Algorithm
Insertion Sort

• Used while playing cards
3
4
6
8
9
A
1
i j
Strategy
• Start “empty
handed”
• Insert a card in the
right position of the
already sorted hand
• Continue until all
cards are inserted or
sorted

7

2

5

1

n

INPUT: An array of Integers A[1..n]
OUTPUT: A permutation of A such that A[1] <=
A[2] <= A[3]<= ..<=A[n]
For j
2 to n do
key A[j]
Insert A*j+ into sorted sequence A*1…j-1]
i
j-1
while i >0 and A[i] >key
do A[i+1] A[i]
i- A[i+1]
key
Analysis of Insertion Sort
• For j
2 to n do
•
key A[j]
•
Insert A[j] into sorted
sequence A*1…j-1]
•
i
j-1
•
while i >0 and A[i] >key
•
do A[i+1] A[i]
•
i- •
A[i+1] key
Total Time = n(c1 + c2 + c3 + c7) +

Cost
c1
c2

Times
n
n-1

c3
c4
c5
c6
c7

n-1

n-1

(c4 + c5 + c6)–( c2 +c3 +c5+c6+c7)

tj counts the number of times the values have to be shifted in one iteration
Best / Average/ Worst Case:
Difference made by tj
• Total Time = n(c1 + c2 + c3 + c7) +
+c3 +c5+c6+c7)
• Best Case

(c4 + c5 + c6)–( c2

– Elements already sorted
– tj = 1
– Running time = f(n) i.e. Linear Time

• Worst Case
– Elements are sorted in inverse order
– tj = j
– Running time = f(n2) i.e. Quadratic Time

• Average Case
– tj = j/2
– Running Time = f(n2) i.e. Quadratic Time
Best / Average/ Worst Case
• For a Specific input size say n

Running Time (sec)

5

Worst Case

4

Average Case

3

Best Case

2

1
A B C D E F G H I J K L M N
Input instance
Best / Average/ Worst Case
• Varying input size
Worst case

Running Time (sec)

50

Average Case
40
Best Case

30
20

10
10

20

30

40

Input Size

50

60
Best / Average/ Worst Case
• Worst Case:
– Mostly used
– It is an Upper bound of how bad a system can be.
– In cases like surgery or air traffic control knowing
the worst case complexity is of crucial importance.

• Average case is often as bad as worst case.
• Finding an average case can be very difficult.
Asymptotic Analysis
• Goal: Simplify analysis of running time by getting
rid of details which may be affected by specific
implementation and hardware
– Like rounding 1000001 to 1000000
– 3n2 to n2

• Capturing the essence: How the running time of
an algorithm increases with the size of the input
in the limit
– Asymptotically more efficient algorithms are best for
all but small inputs.
Asymptotic Notation
• The “big Oh” (O)
Notation
– Asymptotic Upper Bound
– f(n) = O(g(n)), if there
exists constants c and n0
such that f(n) ≤ cg(n) for n
≥ n0
– f(n) and g(n) are functions
over non negative nondecreasing integers

• Used for worst case
analysis

c. g(n)

f(n)

n0

n

f(n) = O(g(n))
Examples
• E.g. 1:
–
–
–
–
–

f(n) = 2n + 6
g(n) = n
c =4
n0 = 3
Thus, f(n) = O(n)

• E.g. 2:
– f(n) = n2
– g(n) = n
– f(n) is not O(n) because there exists no
constants c and n0 such that f(n) ≤
cg(n) for n ≥ n0

• E.g. 3:
–
–
–
–
–

f(n) = n2 + 5
g(n) = n2
c=2
n0 = 2
Thus, f(n) = O(n2)
Asymptotic Notation
• Simple Rules:
– Drop lower order terms and constant factors
• 50 nlogn is O(nlogn)
• 7n – 3 is O(n)
• 8n2logn + 5n2 + n is O(n2logn)

– Note that 50nlogn is also O(n5) but we will
consider it to be O(nlogn) it is expected that the
approximation should be of as lower value as
possible.
Comparing Asymptotic Analysis of
Running Time
• Hierarchy of f(n)
– Log n < n < n2 < n3 < 2n

• Caution!!!!!
– An algorithm with running time of 1,000,000 n is
still O(n) but might be less efficient than one
running in time 2n2, which is O(n2) when n is not
very large.
Examples of Asymptotic Analysis
• Algorithm: prefixAvg1(X)
• Input: An n element array X of numbers
• Output: An n element array A of numbers such
that A[i] is the average of elements X[0]…X[i].
for i 0 to n-1 do
a 0
for j 0 to i do
a a+X[j]
A[i] a/(i+1)
return A

n iterations
1 Step

i iterations
i = 0 , 1, …., i-1

• Analysis: Running Time O(n2) roughly.
Examples of Asymptotic Analysis
• Algorithm: prefixAvg2(X)
• Input: An n element array X of numbers
• Output: An n element array A of numbers such
that A[i] is the average of elements X[0]…X[i].
s 0
for i 0 to n do
s s + X[i]
A[i] s/(i+1)
return A
• Analysis: Running time is O(n)
Asymptotic Notation (Terminologies)
• Special cases of Algorithms
– Logarithmic : O(log n)
– Linear: O(n)
– Quadratic: O(n2)
– Polynomial: O(nk), k ≥ 1
– Exponential: O(an), a>1

• “Relatives” of Big Oh
– Big Omega (Ω(f(n)) : Asymptotic Lower Bound
– Big Theta (Θ(f(n)): Asymptotic Tight Bound
Asymptotic Notation
• The big Omega (Ω) Notation
– Asymptotic Lower bound
– f(n) = Ω(g(n), if there exists
constants c and n0 such that
c.g(n) ≤ f(n) for n > n0

• Used to describe best case
running times or lower
bounds of asymptotic
problems
– E.g: Lower bound of
searching in an unsorted
array is Ω(n).
Asymptotic Notation
• The Big Theta (Θ) notation
– Asymptotically tight bound
– f(n) = Θ(g(n)), if there exists
constants c1, c2 and n0 such
that c1.g(n) ≤ f(n) ≤ c2.g(n) for
n > n0
– f(n) = Θ(g(n)), iff f(n) = O(g(n))
and f(n) = Ω(g(n),
– O(f(n)) is often misused
instead of Θ(f(n))
Asymptotic Notation
• There are 2 more notations
– Little oh notation (o), f(n) = o(g(n))
• For every c there should exist a no. n0 such that f(n) ≤ o(g(n) for n >
n0.

– Little Omega notation (ω)

• Analogy with real numbers
–
–
–
–
–

f(n) = O(g(n)) ,
f(n) = Ω(g(n)),
f(n) = Θ(g(n)),
f(n) = o(g(n)),
f(n) = ω(g(n)),

f≤g
f≥g
f=g
f<g
f>g

• Abuse of Notation:
– f(n) = O(g(n)) actually means f(n) є O(g(n))
Comparison of Running times
Assignment
• Write a C program to implement i) Insertion
sort and ii) Bubble Sort.
• Perform an Experimental Study and
graphically show the time required for both
the program to run.
• Consider varied size of inputs and best case
and worst case for each of the input sets.

More Related Content

PPTX
Associative memory 14208
PPTX
Agents in Artificial intelligence
PPTX
Fuzzy logic and application in AI
PPT
Unit 1 chapter 1 Design and Analysis of Algorithms
PPT
Knowledge Representation in Artificial intelligence
PDF
Algorithm chapter 10
PPTX
Asymptotic notations
PPT
Asymptotic notations
Associative memory 14208
Agents in Artificial intelligence
Fuzzy logic and application in AI
Unit 1 chapter 1 Design and Analysis of Algorithms
Knowledge Representation in Artificial intelligence
Algorithm chapter 10
Asymptotic notations
Asymptotic notations

What's hot (20)

PPT
Primitive Recursive Functions
PPTX
Problem solving agents
PPT
CS8461 - Design and Analysis of Algorithms
PPTX
Daa unit 1
PPTX
Address calculation-sort
PPTX
Asymptotic notation
PPTX
Natural Language Processing
PPTX
PROCEDURAL AND DECLARATIVE KNOWLEDGE IN AI & ML (1).pptx
PPTX
Ch2 properties of the task environment
PPTX
Principal source of optimization in compiler design
PDF
3.Properties of signals
PPT
similarity measure
PPT
Time andspacecomplexity
PPT
AI Lecture 4 (informed search and exploration)
PDF
Rough K Means - Numerical Example
PPTX
Adversarial search
PPTX
introduction to embedded system presentation
PDF
Syntax analysis
PDF
Algorithms Lecture 2: Analysis of Algorithms I
PDF
AI_Unit I notes .pdf
Primitive Recursive Functions
Problem solving agents
CS8461 - Design and Analysis of Algorithms
Daa unit 1
Address calculation-sort
Asymptotic notation
Natural Language Processing
PROCEDURAL AND DECLARATIVE KNOWLEDGE IN AI & ML (1).pptx
Ch2 properties of the task environment
Principal source of optimization in compiler design
3.Properties of signals
similarity measure
Time andspacecomplexity
AI Lecture 4 (informed search and exploration)
Rough K Means - Numerical Example
Adversarial search
introduction to embedded system presentation
Syntax analysis
Algorithms Lecture 2: Analysis of Algorithms I
AI_Unit I notes .pdf
Ad

Viewers also liked (20)

PPTX
Insertion sort analysis
PPTX
Yzm 2116 - Bölüm 2 (Algoritma Analizi)
DOCX
Siralama algoritmalari ileri algoritma analizi
PPTX
Ayrık yapılar algoritmalar
PPT
Asymptotic analysis
PPTX
Insertion sort
PPTX
Asymptotic Notations
PPTX
Asymptotic Notation and Data Structures
PPTX
Insertion sort
PPT
Analysis algorithm introduction Lecture# 1
PPTX
Rúbrica del comentario de texto lírico
PDF
Alg1 8.2 Substitution Method
PDF
Programlama I (C) Ders Notu
PDF
Data Structure: Algorithm and analysis
PPTX
Lecture 6-cs648 Randomized Algorithms
PPT
Asymptotic notation
PDF
Lecture 4 asymptotic notations
PPTX
Yzm 2116 Bölüm 11 - Graph - Çizge
PDF
Lz77 / Lempel-Ziv Algorithm
DOC
Insertion sort
Insertion sort analysis
Yzm 2116 - Bölüm 2 (Algoritma Analizi)
Siralama algoritmalari ileri algoritma analizi
Ayrık yapılar algoritmalar
Asymptotic analysis
Insertion sort
Asymptotic Notations
Asymptotic Notation and Data Structures
Insertion sort
Analysis algorithm introduction Lecture# 1
Rúbrica del comentario de texto lírico
Alg1 8.2 Substitution Method
Programlama I (C) Ders Notu
Data Structure: Algorithm and analysis
Lecture 6-cs648 Randomized Algorithms
Asymptotic notation
Lecture 4 asymptotic notations
Yzm 2116 Bölüm 11 - Graph - Çizge
Lz77 / Lempel-Ziv Algorithm
Insertion sort
Ad

Similar to asymptotic analysis and insertion sort analysis (20)

PPT
1 Analysis of algorithmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm.ppt
PPTX
complexity big oh notation notation.pptx
PDF
Unit 1_final DESIGN AND ANALYSIS OF ALGORITHM.pdf
PDF
Annotations.pdf
PDF
Data Structure & Algorithms - Introduction
PPTX
complex computer systems and its applications.pptx
PPTX
Algorithms & Complexity Calculation
PDF
ESINF03-AlgAnalis.pdfESINF03-AlgAnalis.pdf
PDF
Advanced Datastructures and algorithms CP4151unit1b.pdf
PPTX
Data Structure Algorithm -Algorithm Complexity
PPTX
cse couse aefrfrqewrbqwrgbqgvq2w3vqbvq23rbgw3rnw345
PPTX
DS Unit-1.pptx very easy to understand..
PPT
Lecture 1 and 2 of Data Structures & Algorithms
PPT
Analysis design and analysis of algorithms ppt
PPT
algorithm and Analysis daa unit 2 aktu.ppt
PPTX
Asymptotic Notations.pptx
PPTX
02 Introduction to Data Structures & Algorithms.pptx
PPTX
Asymptotics 140510003721-phpapp02
1 Analysis of algorithmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm.ppt
complexity big oh notation notation.pptx
Unit 1_final DESIGN AND ANALYSIS OF ALGORITHM.pdf
Annotations.pdf
Data Structure & Algorithms - Introduction
complex computer systems and its applications.pptx
Algorithms & Complexity Calculation
ESINF03-AlgAnalis.pdfESINF03-AlgAnalis.pdf
Advanced Datastructures and algorithms CP4151unit1b.pdf
Data Structure Algorithm -Algorithm Complexity
cse couse aefrfrqewrbqwrgbqgvq2w3vqbvq23rbgw3rnw345
DS Unit-1.pptx very easy to understand..
Lecture 1 and 2 of Data Structures & Algorithms
Analysis design and analysis of algorithms ppt
algorithm and Analysis daa unit 2 aktu.ppt
Asymptotic Notations.pptx
02 Introduction to Data Structures & Algorithms.pptx
Asymptotics 140510003721-phpapp02

Recently uploaded (20)

PPTX
Microbial diseases, their pathogenesis and prophylaxis
PPTX
PPT- ENG7_QUARTER1_LESSON1_WEEK1. IMAGERY -DESCRIPTIONS pptx.pptx
PPTX
master seminar digital applications in india
PDF
A systematic review of self-coping strategies used by university students to ...
PPTX
Tissue processing ( HISTOPATHOLOGICAL TECHNIQUE
PDF
RMMM.pdf make it easy to upload and study
PDF
Saundersa Comprehensive Review for the NCLEX-RN Examination.pdf
PDF
Anesthesia in Laparoscopic Surgery in India
PPTX
school management -TNTEU- B.Ed., Semester II Unit 1.pptx
PDF
Module 4: Burden of Disease Tutorial Slides S2 2025
PPTX
Cell Structure & Organelles in detailed.
PPTX
Cell Types and Its function , kingdom of life
PDF
Complications of Minimal Access Surgery at WLH
PDF
VCE English Exam - Section C Student Revision Booklet
PPTX
1st Inaugural Professorial Lecture held on 19th February 2020 (Governance and...
PDF
Black Hat USA 2025 - Micro ICS Summit - ICS/OT Threat Landscape
PPTX
Final Presentation General Medicine 03-08-2024.pptx
PDF
ANTIBIOTICS.pptx.pdf………………… xxxxxxxxxxxxx
PDF
Chinmaya Tiranga quiz Grand Finale.pdf
PDF
OBE - B.A.(HON'S) IN INTERIOR ARCHITECTURE -Ar.MOHIUDDIN.pdf
Microbial diseases, their pathogenesis and prophylaxis
PPT- ENG7_QUARTER1_LESSON1_WEEK1. IMAGERY -DESCRIPTIONS pptx.pptx
master seminar digital applications in india
A systematic review of self-coping strategies used by university students to ...
Tissue processing ( HISTOPATHOLOGICAL TECHNIQUE
RMMM.pdf make it easy to upload and study
Saundersa Comprehensive Review for the NCLEX-RN Examination.pdf
Anesthesia in Laparoscopic Surgery in India
school management -TNTEU- B.Ed., Semester II Unit 1.pptx
Module 4: Burden of Disease Tutorial Slides S2 2025
Cell Structure & Organelles in detailed.
Cell Types and Its function , kingdom of life
Complications of Minimal Access Surgery at WLH
VCE English Exam - Section C Student Revision Booklet
1st Inaugural Professorial Lecture held on 19th February 2020 (Governance and...
Black Hat USA 2025 - Micro ICS Summit - ICS/OT Threat Landscape
Final Presentation General Medicine 03-08-2024.pptx
ANTIBIOTICS.pptx.pdf………………… xxxxxxxxxxxxx
Chinmaya Tiranga quiz Grand Finale.pdf
OBE - B.A.(HON'S) IN INTERIOR ARCHITECTURE -Ar.MOHIUDDIN.pdf

asymptotic analysis and insertion sort analysis

  • 1. Data Structures and Algorithms Lecture 1: Asymptotic Notations
  • 2. Basic Terminologies • Algorithm – Outline – Essence of a computational procedure – Step by step instructions • Program – Implementation of an Algorithm in some programming language • Data Structure – Organization of Data needed to solve the problem effectively – Data Structure you already know: Array
  • 3. Algorithmic Problem Specification of Input ? Specification of Output as a function of Input • Infinite number of input instances satisfying the specification • E.g.: Specification of Input: • A sorted non-decreasing sequence of natural numbers of non-zero, finite length: • 1, 20,908,909,1000,10000,1000000 • 3
  • 4. Algorithmic Solution Input instance adhering to Specification Algorithm Output related to the Input as required • Algorithm describes the actions on the input instances • Infinitely many correct algorithms may exist for the same algorithmic problem
  • 5. Characteristics of a Good Algorithm • Efficient – Small Running Time – Less Memory Usage • Efficiency as a function of input size – The number of bits in a data number – The number of data elements
  • 6. Measuring the running time • Experimental Study – Write a program that implements the algorithm – Run the program with data sets of varying size and composition – Use system defined functions like clock() to get an measurement of the actual running time
  • 7. Measuring the running time • Limitations of Experimental Study – It is necessary to implement and test the algorithm in order to determine its running time. – Experiments can be done on a limited set of inputs, and may not be indicative of the running time on other inputs not included in the experiment – In order to compare 2 algorithms same hardware and software environments must be used.
  • 8. Beyond Experimental Study • We will develop a general methodology for analyzing the running time of algorithms. This approach – Uses a high level description of the algorithm instead of testing one of its implementations – Takes into account all possible inputs – Allows one to evaluate the efficiency of any algorithm in a way that is independent of the hardware and software environment.
  • 9. Pseudo - Code • A mixture of natural language and high level programming concepts that describes the main ideas behind a generic implementation of a data structure or algorithm. • E.g. Algorithm arrayMax(A,n) – Input: An array A storing n integers – Output: The maximum element in A. currMax A[0] for i 1 to n-1 do if currMax < A[i] then currMax return currMax A[i]
  • 10. Pseudo - Code • It is more structured that usual prose but less formal than a programming language • Expressions: – Use standard mathematical symbols to describe numeric and Boolean expressions – Use ‘ ’ for assignment instead of “=” – Use ‘=’ for equality relationship instead of “==” • Function Declarations: – Algorithm name (param1 , param2)
  • 11. Pseudo - Code • Programming Constructs: – – – – – Decision structures: if…then….[else….] While loops: while…..do….. Repeat loops: Repeat………until……. For loop: for……do……….. Array indexing: A[i], A[I,j] • Functions: – Calls: return_type function_name(param1 ,param2) – Returns: return value
  • 12. Analysis of Algorithms • Primitive Operation: – Low level operation independent of programming language. – Can be identified in a pseudo code. • For e.g. – Data Movement (assign) – Control (branch, subroutine call, return) – Arithmetic and Logical operations (e.g. addition comparison) – By inspecting the pseudo code, we can count the number of primitive operations executed by an algorithm.
  • 13. Example: Sorting OUPUT Permutation of the sequence of numbers in non-decreasing order INPUT Sequence of numbers a1,a2,a3,a4,….an 2, 5 , 4, 10, 7 Sort Correctness (Requirements for the output) For any given input the algorithm halts with the output: • b1 < b2 < b3 < b4 < ….. < bn • b1, b2, b3, b4,…. bn is a permutation of a1, a2, a3, a4, …. an b1,b2,b3,b4,….bn 2, 4 , 5, 7, 10 Running Time depends on • No. of elements (n) • How sorted (partial) the numbers are? • Algorithm
  • 14. Insertion Sort • Used while playing cards 3 4 6 8 9 A 1 i j Strategy • Start “empty handed” • Insert a card in the right position of the already sorted hand • Continue until all cards are inserted or sorted 7 2 5 1 n INPUT: An array of Integers A[1..n] OUTPUT: A permutation of A such that A[1] <= A[2] <= A[3]<= ..<=A[n] For j 2 to n do key A[j] Insert A*j+ into sorted sequence A*1…j-1] i j-1 while i >0 and A[i] >key do A[i+1] A[i] i- A[i+1] key
  • 15. Analysis of Insertion Sort • For j 2 to n do • key A[j] • Insert A[j] into sorted sequence A*1…j-1] • i j-1 • while i >0 and A[i] >key • do A[i+1] A[i] • i- • A[i+1] key Total Time = n(c1 + c2 + c3 + c7) + Cost c1 c2 Times n n-1 c3 c4 c5 c6 c7 n-1 n-1 (c4 + c5 + c6)–( c2 +c3 +c5+c6+c7) tj counts the number of times the values have to be shifted in one iteration
  • 16. Best / Average/ Worst Case: Difference made by tj • Total Time = n(c1 + c2 + c3 + c7) + +c3 +c5+c6+c7) • Best Case (c4 + c5 + c6)–( c2 – Elements already sorted – tj = 1 – Running time = f(n) i.e. Linear Time • Worst Case – Elements are sorted in inverse order – tj = j – Running time = f(n2) i.e. Quadratic Time • Average Case – tj = j/2 – Running Time = f(n2) i.e. Quadratic Time
  • 17. Best / Average/ Worst Case • For a Specific input size say n Running Time (sec) 5 Worst Case 4 Average Case 3 Best Case 2 1 A B C D E F G H I J K L M N Input instance
  • 18. Best / Average/ Worst Case • Varying input size Worst case Running Time (sec) 50 Average Case 40 Best Case 30 20 10 10 20 30 40 Input Size 50 60
  • 19. Best / Average/ Worst Case • Worst Case: – Mostly used – It is an Upper bound of how bad a system can be. – In cases like surgery or air traffic control knowing the worst case complexity is of crucial importance. • Average case is often as bad as worst case. • Finding an average case can be very difficult.
  • 20. Asymptotic Analysis • Goal: Simplify analysis of running time by getting rid of details which may be affected by specific implementation and hardware – Like rounding 1000001 to 1000000 – 3n2 to n2 • Capturing the essence: How the running time of an algorithm increases with the size of the input in the limit – Asymptotically more efficient algorithms are best for all but small inputs.
  • 21. Asymptotic Notation • The “big Oh” (O) Notation – Asymptotic Upper Bound – f(n) = O(g(n)), if there exists constants c and n0 such that f(n) ≤ cg(n) for n ≥ n0 – f(n) and g(n) are functions over non negative nondecreasing integers • Used for worst case analysis c. g(n) f(n) n0 n f(n) = O(g(n))
  • 22. Examples • E.g. 1: – – – – – f(n) = 2n + 6 g(n) = n c =4 n0 = 3 Thus, f(n) = O(n) • E.g. 2: – f(n) = n2 – g(n) = n – f(n) is not O(n) because there exists no constants c and n0 such that f(n) ≤ cg(n) for n ≥ n0 • E.g. 3: – – – – – f(n) = n2 + 5 g(n) = n2 c=2 n0 = 2 Thus, f(n) = O(n2)
  • 23. Asymptotic Notation • Simple Rules: – Drop lower order terms and constant factors • 50 nlogn is O(nlogn) • 7n – 3 is O(n) • 8n2logn + 5n2 + n is O(n2logn) – Note that 50nlogn is also O(n5) but we will consider it to be O(nlogn) it is expected that the approximation should be of as lower value as possible.
  • 24. Comparing Asymptotic Analysis of Running Time • Hierarchy of f(n) – Log n < n < n2 < n3 < 2n • Caution!!!!! – An algorithm with running time of 1,000,000 n is still O(n) but might be less efficient than one running in time 2n2, which is O(n2) when n is not very large.
  • 25. Examples of Asymptotic Analysis • Algorithm: prefixAvg1(X) • Input: An n element array X of numbers • Output: An n element array A of numbers such that A[i] is the average of elements X[0]…X[i]. for i 0 to n-1 do a 0 for j 0 to i do a a+X[j] A[i] a/(i+1) return A n iterations 1 Step i iterations i = 0 , 1, …., i-1 • Analysis: Running Time O(n2) roughly.
  • 26. Examples of Asymptotic Analysis • Algorithm: prefixAvg2(X) • Input: An n element array X of numbers • Output: An n element array A of numbers such that A[i] is the average of elements X[0]…X[i]. s 0 for i 0 to n do s s + X[i] A[i] s/(i+1) return A • Analysis: Running time is O(n)
  • 27. Asymptotic Notation (Terminologies) • Special cases of Algorithms – Logarithmic : O(log n) – Linear: O(n) – Quadratic: O(n2) – Polynomial: O(nk), k ≥ 1 – Exponential: O(an), a>1 • “Relatives” of Big Oh – Big Omega (Ω(f(n)) : Asymptotic Lower Bound – Big Theta (Θ(f(n)): Asymptotic Tight Bound
  • 28. Asymptotic Notation • The big Omega (Ω) Notation – Asymptotic Lower bound – f(n) = Ω(g(n), if there exists constants c and n0 such that c.g(n) ≤ f(n) for n > n0 • Used to describe best case running times or lower bounds of asymptotic problems – E.g: Lower bound of searching in an unsorted array is Ω(n).
  • 29. Asymptotic Notation • The Big Theta (Θ) notation – Asymptotically tight bound – f(n) = Θ(g(n)), if there exists constants c1, c2 and n0 such that c1.g(n) ≤ f(n) ≤ c2.g(n) for n > n0 – f(n) = Θ(g(n)), iff f(n) = O(g(n)) and f(n) = Ω(g(n), – O(f(n)) is often misused instead of Θ(f(n))
  • 30. Asymptotic Notation • There are 2 more notations – Little oh notation (o), f(n) = o(g(n)) • For every c there should exist a no. n0 such that f(n) ≤ o(g(n) for n > n0. – Little Omega notation (ω) • Analogy with real numbers – – – – – f(n) = O(g(n)) , f(n) = Ω(g(n)), f(n) = Θ(g(n)), f(n) = o(g(n)), f(n) = ω(g(n)), f≤g f≥g f=g f<g f>g • Abuse of Notation: – f(n) = O(g(n)) actually means f(n) є O(g(n))
  • 32. Assignment • Write a C program to implement i) Insertion sort and ii) Bubble Sort. • Perform an Experimental Study and graphically show the time required for both the program to run. • Consider varied size of inputs and best case and worst case for each of the input sets.