Advanced Datastructures and algorithms CP4151unit1b.pdf

Advanced Data Structures
and Algorithms
Unit 1
Unit 1

Data Structures and Algorithms
 Algorithm: Step-by-step procedure to
solve a problem
 Program: an implementation of an
 Program: an implementation of an
algorithm in some programming language
 Data structure: Organization of data
needed to solve the problem

Algorithmic problem
Specification
of input
?
Specification
of output as
a function of
input
Infinite number of input instances satisfying the
specification.
 For eg: A sorted, non-decreasing sequence of natural
numbers of non-zero, finite length:
 1, 20, 908, 909, 100000, 1000000000.

Algorithmic Solution
Input instance,
adhering to
the
specification
Algorithm Output
related to
the input as
required
Algorithm describes actions on the input instance
There may be many correct algorithms for the
same algorithmic problem
specification required

What is a Good Algorithm?
 Efficient:
Running time
Space used
 Efficiency as a function of input size:
Efficiency as a function of input size:
The number of bits in an input number
Number of data elements (numbers, points)

Measuring the Running Time
Experimental Study n
20
10
30
40
50
t (ms)
60
How should we measure the
running time of an algorithm?
Experimental Study
 Write a program that implements the algorithm
 Run the program with data sets of varying size
and composition.
 Use a system call to get an accurate measure of
the actual running time.
50 100
0
n

Limitations of Experimental Studies
It is necessary to implement and test the
algorithm in order to determine its running time.
Experiments done only on a limited set of inputs,
Experiments done only on a limited set of inputs,
 may not be indicative of the running time on other
inputs not included in the experiment.
In order to compare two algorithms, the same
hardware and software environments needed

Beyond Experimental Studies
We will develop a general methodology for
analyzing running time of algorithms. This
approach
Uses a high-level description of the algorithm
Uses a high-level description of the algorithm
instead of testing one of its implementations.
Takes into account all possible inputs.
Allows one to evaluate the efficiency of any
algorithm in a way that is independent of the
hardware and software environment.

Example
 Algorithm arrayMax(A, n):
Input: An array A storing n integers.
Output: The maximum element in A.

Pseudo-code (Functional / Recursive)
algorithm arrayMax(A[0..n-1])
{
A[0]
max(arrayMax(A[0..n-2]), A[n-1])
if n=1
o.w.
max(arrayMax(A[0..n-2]), A[n-1]) o.w.
}

Pseudo-Code (imperative)
 A mixture of natural language and high-level
programming concepts that describes the main
ideas behind a generic implementation of a data
structure or algorithm.
 Eg: algorithm arrayMax(A, n):
Output: The maximum element in A.
currentMax  A[0]
for i  1 to n-1 do
if currentMax < A[i] then currentMax  A[i]
return currentMax

Pseudo-Code
It is more structured than usual prose but
less formal than a programming language
 Expressions:
 use standard mathematical symbols to
 use standard mathematical symbols to
describe numeric and boolean expressions
 use  for assignment (“=” in Java)
 use = for equality relationship (“==” in Java)
 Method Declarations:
 algorithm name(param1, param2)

Pseudo Code
 Programming Constructs:
decision structures: if ... then ... [else ... ]
while-loops: while ... do
repeat-loops: repeat ... until ...
for-loop: for ... do
for-loop: for ... do
array indexing: A[i], A[i,j]
 Methods:
calls: object method(args)
returns: return value

Analysis of Algorithms
 Primitive Operation: Low-level operation
independent of programming language.
Can be identified in pseudo-code. For eg:
Data movement (assign)
Control (branch, subroutine call, return)
arithmetic an logical operations (e.g. addition,
comparison)
 By inspecting the pseudo-code, we can
count the number of primitive operations
executed by an algorithm.

Sort
Example: Sorting
INPUT
sequence of numbers
OUTPUT
a permutation of the
sequence of numbers
a1, a2, a3,….,an
2 5 4 10 7
b1,b2,b3,….,bn
2 4 5 7 10
2 5 4 10 7 2 4 5 7 10
Correctness (requirements for the
output)
For any given input the algorithm
halts with the output:
• b1 < b2 < b3 < …. < bn
•b1, b2, b3, …., bn is a
permutation of a1, a2, a3,….,an
Running time
Depends on
• number of elements (n)
• how (partially) sorted
they are algorithm

Insertion Sort
A
1 n
j
3 4 6 8 9 7 2 5 1
i
Strategy INPUT: A[0..n-1] – an array of integers
OUTPUT: a permutation of A such that
• Start “empty handed”
• Insert a card in the right
position of the already sorted
hand
• Continue until all cards are
inserted/sorted
A[0] A[1] …A[n-1]

Pseudo-code (Functional / Recursive)
algorithm insertionSort(A[0..n-1])
{
A[0]
insert(insertionSort(A[0..n-2]), A[n-1])
if n=1
o.w.
}
}
algorithm insert(A[0..n-1], key)
{
append(A[0..n-1], key)
append(newarray(key), A[0])
if key>=A[n-1]
if n=1&key<A[0]
append(insert(A[0..n-2],key), A[n-1]) o.w.
}

Insertion Sort
A
1 n
j
3 4 6 8 9 7 2 5 1
i
Strategy INPUT: A[0..n-1] – an array of integers
• Start “empty handed”
• Insert a card in the right
position of the already sorted
hand
• Continue until all cards are
inserted/sorted
A[0] A[1] …A[n-1]
for j1 to n-1 do
key  A[j]
//insertA[j] into the sorted sequence
A[0..j-1]
ij-1
while i>=0 and A[i]>key
do A[i+1]A[i]
i--
A[i+1]key

Analysis of Insertion Sort
for j1 to n-1 do
keyA[j]
//insertA[j] into the sorted
sequence A[0..j-1]
ij-1
c1
c2
0
c3
cost Times
n
n-1
n-1
n-1
𝒏−𝟏
∑ 𝒕𝒋
ij-1
while i>=0 and A[i]>key
do A[i+1]A[i]
i--
A[i+1]  key
c3
c4
c5
c6
c7 n-1
(c4+c5+c6)
Total time = n(c1+c2+c3+c7) + n-1
j=1 tj
– (c2+c3+c5+c6+c7)
𝒋=𝟏
n-1
𝒏−𝟏
∑ 𝒕𝒋
𝒋=𝟏
𝒏−𝟏
∑ (𝒕𝒋 − 𝟏)
𝒋=𝟏
𝒏−𝟏
∑ (𝒕𝒋 − 𝟏)

Best/Worst/Average Case
(c4+c5+c6)
Total time = n(c1+c2+c3+c7) + n-1
j=1 tj
– (c2+c3+c5+c6+c7)
 Best case:
elements already sorted; tj=1, running time = f(n),
elements already sorted; tj=1, running time = f(n),
i.e., linear time.
 Worst case:
elements are sorted in inverse order; tj=j+1,
running time = f(n2), i.e., quadratic time
 Average case:
tj=(j+1)/2, running time = f(n2), i.e., quadratic time

Best/Worst/Average Case (2)
 For a specific size of input n, investigate
running times for different input instances:

For inputs of all sizes:
6n average-case
worst-case
4n
5n
Running
time
3n
2n
1n
1 2 3 4 5 6 7 8 9 10 11 12 …..
Input instance size
best-case

 Worst case is usually used: It is an upper-
bound and in certain application domains (e.g.,
air traffic control, surgery) knowing the worst-
case time complexity is of crucial importance
 For some algos worst case occurs fairly often
 Average case is often as bad as worst case
 Finding average case can be very difficult

Asymptotic Analysis
 Goal: to simplify analysis of running time by
getting rid of ”details”, which may be affected by
specific implementation and hardware
 like “rounding”: 1,000,001  1,000,000
 3n2  n2
 3n  n
 Capturing the essence: how the running time of
an algorithm increases with the size of the input
in the limit.
 Asymptotically more efficient algorithms are best for
all but small inputs

Asymptotic Notation
 The “big-Oh” O-Notation
asymptotic upper bound
f(n) is O(g(n)), if there exists constants c and n0,
s.t. f(n)  c g(n) for all n  n0
f(n) and g(n) are functions over non-negative
f(n) and g(n) are functions over non-negative
integers
cg(n)
f (n)
 Used for worst-case
analysis
n0 Input Size
Running
Time

Asymptotic Notation
 Simple Rule: Drop lower order terms and
constant factors.
50 n log n is O(n log n)
7n - 3 is O(n)
8n2 log n + 5n2 + n is O(n2 log n)
 Note: Even though (50 n log n) is O(n5), it
is expected that such an approximation be
of as small an order as possible

Asymptotic Analysis of Running Time
 Use O-notation to express number of primitive
operations executed as function of input size.
 Comparing asymptotic running times
an algorithm that runs in O(n) time is better
than one that runs in O(n2) time
than one that runs in O(n2) time
similarly, O(log n) is better than O(n)
hierarchy of functions: log n < n < n2 < n3 < 2n
 Caution! Beware of very large constant factors.
An algorithm running in time 1,000,000 n is still
O(n) but might be less efficient than one running
in time 2n2, which is O(n2)

Example of Asymptotic Analysis
Algorithm prefixAverages1(X):
Input: An n-element array X of numbers.
Output: An n-element array X of numbers such that
x[i] is the average of elements X[0], ... , X[i].
a  0
for j  0 to i do
a  a + X[j]
A[i]  a/(i+1)
return array A
Analysis: running time is O(n2)
1 step
i iterations
with
i=0,1,2...n-1
n iterations

A Better Algorithm
Algorithm prefixAverages2(X):
Input: An n-element array X of numbers.
Output: An n-element array A of numbers such
that A[i] is the average of elements X[0], ... , X[i].
s  0
s  0
for i  0 to n do
s  s + X[i]
A[i]  s/(i+1)
return array A
Analysis: Running time is O(n)

Asymptotic Notation (terminology)
 Special classes of algorithms:
 Logarithmic: O(log n)
 Linear: O(n)
 Quadratic: O(n2)
 Polynomial: O(nk), k ≥ 1
 Polynomial: O(nk), k ≥ 1
 Exponential: O(an), a > 1
 “Relatives” of the Big-Oh
  (f(n)): Big Omega -asymptotic lower bound
  (f(n)): Big Theta -asymptotic tight bound

 The “big-Omega” 
Notation
 asymptotic lower bound
 f(n) is (g(n)) if there exists
constants c and n0, s.t.
c g(n)  f(n) for n  n0
Running
Time
f (n)
c g(n)
Asymptotic Notation
c g(n)  f(n) for n  n0
 Used to describe best-
case running times or
lower bounds for
algorithmic problems
 E.g., lower-bound for
searching in an unsorted
array is (n).
Input Size
Running
n0

 The “big-Theta” 
Notation
 asymptotically tight bound
 f(n) is (g(n)) if there exists
constants c1, c2, and n0, s.t.
Asymptotic Notation
constants c1, c2, and n0, s.t.
c1 g(n)  f(n)  c2 g(n) for n 
n0
 f(n) is (g(n)) if and only if
f(n) is (g(n)) and f(n) is
(g(n))
 O(f(n)) is often misused
instead of (f(n))
Input Size
Running
Time n0
c2 g(n)
f (n)
c1 g(n)

Asymptotic Notation
Two more asymptotic notations
 "Little-Oh" notation f(n) is o(g(n))
non-tight analogue of Big-Oh
For every c>0, there should exist n0 , s.t.
f(n)  c g(n) 
0
f(n)  c g(n) for n  n0
Used for comparisons of running times.
If f(n) is o(g(n)), it is said that g(n)
dominates f(n).
 "Little-omega" notation f(n) is (g(n))
non-tight analogue of Big-Omega

Asymptotic Notation
 Analogy with real numbers
f(n) is O(g(n))  f  g
f(n) is (g(n))  f  g
f(n) is (g(n))  f  g
f(n) is o(g(n))  f  g
f(n) is (g(n))  f  g
 Abuse of notation: f(n) = O(g(n)) actually
means f(n) O(g(n))

Comparison of Running Times
Running
Time
Maximum problem size (n)
1 second 1 minute 1 hour
400n 2500 150000 9000000
400n 2500 150000 9000000
20n log n 4096 166666 7826087
2n2 707 5477 42426
n4 31 88 244
2n 19 25 31

Advanced Datastructures and algorithms CP4151unit1b.pdf

More Related Content

Similar to Advanced Datastructures and algorithms CP4151unit1b.pdf (20)

More from Sheba41 (8)

Recently uploaded (20)

Advanced Datastructures and algorithms CP4151unit1b.pdf