algorithms and data structure Time complexity

M269
Algorithms, Data Structures
and Computability
BY:
DR. AHMED GAWISH
A.GAWISH@ARABOU.EDU.KW
Lec 2

Agenda
1- Python:
Overview, Hello, World!, Variables and Types, Lists, Basic
Operators, String Formatting, Basic String Operations,
Conditions, Loops, Functions, Classes and Objects,
Dictionaries, Modules and Packages
2- What is computation and computational thinking?
3- Complexity analysis:
• Asymptotic performance
• Asymptotic (big-O) notation
• Analysis of Algorithms

https://www.programiz.com/pyt
hon-programming
http://www.learnpython.o
rg/
https://python-course.eu/pytho
n3_course.php
https://www.tutorialspoint.com/
python/
Self-Learning Resources
Interactive tutorials
1
2
3
4

Self-Learning Resources
books
Starting Out with
Python (4th Edition)
Python for Everybody:
Exploring Data In Python 3
Programming Python
(4th Edition)

Python Video Courses
• Tutorial 1 Series: https://www.yout
ube.com/watch?v=P74JAYCD45A&li
st=PLGzru6ACxEALhcvY18A-iox-mE
oieHMVG
• Tutorial2 Series: https://www.youtu
be.com/watch?v=HBxCHonP6Ro&li
st=PL6gx4Cwl9DGAcbMi1sH6oAMk
4JHw91mC_
• Learn Python in 45 Minutes: https:/
/www.youtube.com/watch?v=N4m

Python features
no compiling or linking rapid development cycle
no type declarations simpler, shorter, more flexible
automatic memory management garbage collection
high-level data types and
operations
fast development
object-oriented programming code structuring and reuse, C++
embedding and extending in C mixed language systems
classes, modules, exceptions "programming-in-the-large"
support
dynamic loading of C modules simplified extensions, smaller
binaries
dynamic reloading of C modules programs can be modified without
stopping

Python features
universal "first-class" object
model
fewer restrictions and rules
run-time program construction handles unforeseen needs, end-user
coding
interactive, dynamic nature incremental development and testing
access to interpreter information metaprogramming, introspective objects
wide portability cross-platform programming without
ports
compilation to portable byte-
code
execution speed, protecting source code
built-in interfaces to external
services
system tools, GUIs, persistence,
databases, etc.

Language comparison
Tc
l
Per
l
Python JavaScr
ipt
Visual
Basic
Speed development     
regexp   
breadth extensible   
embeddable  
easy GUI   (Tk) 
net/web     
enterprise cross-platform    
I18N    
thread-safe   
database
access
    

Uses of Python
• shell tools
• system admin tools, command line programs
• extension-language work
• rapid prototyping and development
• language-based modules
• instead of special-purpose parsers
• graphical user interfaces
• database access
• distributed programming
• Internet scripting

Brief History of Python
• Invented in the Netherlands, early 90s by Guido
van Rossum
• Named after Monty Python
• Open sourced from the beginning
• Considered a scripting language, but is much
more
• Scalable, object oriented and functional from the
beginning
• Used by Google from the beginning
• Increasingly popular

Installing
•Python is pre-installed on most Unix systems,
including Linux and MAC OS X
•Download from http://python.org/download/
•Python comes with a large library of standard
modules
•There are several options for an IDE
• IDLE – works well with Windows
• Emacs with python-mode or your favorite text editor
• Eclipse with Pydev (http://pydev.sourceforge.net/)

Now Let’s Start
http://www.learnpython.o
rg/
We will use the following interactive tutorial website

What is Computation?
In this part, we will discuss two points:
• Computational Thinking
• Computational Problem
One can major [i.e. graduate] in computer
science and do anything. One can major
in English or mathematics and go on to a
multitude of different careers. Ditto
computer science. One can major in
computer science and go on to a career in
medicine, law, business, politics, any type
of science or engineering, and even the
arts.
Wing (2006)
Jeannette M. Wing
Professor of Computer Science (Carnegie
Mellon University, United States) and Head
of Microsoft Research International

What does each of them mean (try to write something down in your
own words, without looking them up)?
• Computational thinking
• Computational problem
Answer:
• computational thinking is not merely knowing how to use
an algorithm or a data structure, but, when faced with a
problem, to be able to analyze it with the techniques and
skills that computer science puts at our disposal.
• A computational problem is described as a problem that is
expressed sufficiently precisely that it is possible to
attempt to build an algorithm to solve it.

The point is that computational thinking is not about
thinking like a computer rather, computational thinking is first
and foremost. Computational thinking consists of the skills to:
• formulate a problem as a computational problem
• construct a good computational solution (i.e. an algorithm) for
the problem, or explain why there is no such solution.
A computational thinker won’t, however, be satisfied with just any solution: the
solution has to be a ‘good’ one. You have already seen that some solutions for
finding a word in a dictionary are much better (in particular, faster) than others.
The search for good computational solutions is a theme that runs throughout
this module. Finally, computational thinking goes beyond finding solutions: if no
good solution exists, one should be able to explain why this is so. This requires
insight into the limits of computational problem solving.

Computational thinking :(Automation)
the feedback loop that one has when you’re abstracting from some
physical-world phenomenon, creating a mathematical model of this
physical-world phenomenon, and then analyzing the abstraction,
doing sorts of manipulations of those abstractions, and in fact
automating the abstraction, that then tells us more about the
physical-world phenomenon that we’re actually modelling.’

algorithms and data structure Time complexity

Computational Thinking Everywhere
Computational biology
Examples:
Machine Learning

Analysis of
Algorithms
20
Algorithm
Input Output

Analysis of Algorithms
Measure Performance
Measure Space
Complexity
Our goal:

22
Running Time
• Most algorithms transform
input objects into output
objects.
• The running time of an
algorithm typically grows
with the input size.
• Average case time is often
difficult to determine.
• We focus on the worst case
running time.
• Easier to analyze
• Crucial to applications such
as games, finance and
robotics
0
20
40
60
80
100
120
Running
Time
1000 2000 3000 4000
Input Size
best case
average case
worst case

Why discarding average case,
And choose worst case instead?
• An algorithm may run faster on some inputs than it does
on others of the same size. Thus, we may wish to express
the running time of an algorithm as the function of the
input size
• Average-case analysis is typically quite challenging. It
requires us to define a probability distribution on the set of
inputs, which is often a difficult task.
• An average-case analysis usually requires that we
calculate expected running times based on a given input
distribution, which usually involves sophisticated
probability theory. Therefore we will characterize running
times in terms of the worst case, as a function of the input
size, n, of the algorithm.
• Worst-case analysis is much easier than average-case
analysis, as it requires only the ability to identify the worst-
case input, which is often simple.

24
Experimental Studies
• Write a program
implementing the
algorithm
• Run the program with
inputs of varying size and
composition, noting the
time needed:
• Plot the results
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
0 50 100
Input Size
Time
(
ms)

25
Limitations of Experiments
• It is necessary to implement the whole algorithm
before conducting any experiment, which may be
difficult.
• Results may not be indicative of the running time on
other inputs not included in the experiment.
• In order to compare two algorithms, the same
hardware and software environments must be
used

So we need another way to measure the
performance of the algorithms
• So we need to learn about Theoretical analysis or
Asymptotic analysis.
• Uses a high-level description of the algorithm
instead of an implementation (Pseudo code).
• Characterizes running time as a function of the
input size, n.
• Takes into account all possible inputs.
• Allows us to evaluate the speed of an algorithm
independent of the hardware/software
environment.
Pseudo code
• High-level description of an
algorithm.
• More structured than English
prose.
• Less detailed than a program.
• Preferred notation for
describing algorithms.
• Hides program design issues.

27
Big-Oh Notation
• Given functions f(n) and
g(n), we say that f(n) is
O(g(n)) if there are positive
constants
c and n0 such that
f(n)  cg(n) for n  n0
• Example: 2n + 10 is O(n)
• 2n + 10  cn
• (c  2) n  10
• n  10/(c  2)
• Pick c = 3 and n0 = 10

28
Big-Oh and Growth Rate
• The big-Oh notation gives an upper bound on the
growth rate of a function
• The statement “f(n) is O(g(n))” means that the
growth rate of f(n) is no more than the growth rate
of g(n)
• We can use the big-Oh notation to rank functions
according to their growth rate
f(n) is O(g(n)) g(n) is O(f(n))
g(n) grows more Yes No
f(n) grows more No Yes
Same growth Yes Yes

29
Relatives of Big-Oh
big-Omega
 f(n) is (g(n)) if there is a constant c > 0
and an integer constant n0  1 such that
f(n)  c•g(n) for n  n0
big-Theta
 f(n) is (g(n)) if there are constants c’ > 0 and c’’ >
0 and an integer constant n0  1 such that
c’•g(n)  f(n)  c’’•g(n) for n  n0

Essential Seven functions to estimate algorithms
performance
g(n) = 1
Print(“Hello
Algorithms”)

performance
g(n) = n
for i in range(0,
n):
Print(i)

performance
g(n) = lg n
Def power_of_2(a):
x = 0
while a > 1:
a = a/2
x = x+1
return x

performance
g(n) = n lg n
for i in range(0,n):
Def power_of_2(a):
x = 0
while a > 1:
a = a/2
x = x+1
return x

performance
g(n) = n2
for j in range(0,n):
print(i*j);

performance
g(n) = n3
for i in range(0, k):
for j in range(0,n):
print(i*j);

performance
g(n) = 2n
def F(n):
if n == 0:
return 0
elif n == 1:
return 1
else: return
F(n-1) + F(n-2)

Seven Important Functions
• Seven functions that often appear
in algorithm analysis:
• Constant  1
• Logarithmic  log n
• Linear  n
• N-Log-N  n log n
• Quadratic  n2
• Cubic  n3
• Exponential  2n
• In a log-log chart, the slope of the
line corresponds to the growth rate
Analysis of Algorithms 37

Why Growth Rate Matters Slide by Matt Stallmann included
with permission.
if runtime is... time for n + 1 time for 2 n time for 4 n
c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)
c n c (n + 1) 2c n 4c n
c n lg n
~ c n lg n
+ c n
2c n lg n + 2cn 4c n lg n + 4cn
c n2
~ c n2
+ 2c n 4c n2
16c n2
c n3
~ c n3
+ 3c n2
8c n3
64c n3
c 2n
c 2 n+1
c 2 2n
c 2 4n
runtime
quadruples
when
problem
size doubles

Comparison of Two Algorithms
39
insertion sort is
n2
/ 4
merge sort is
2 n lg n
sort a million items?
insertion sort takes
roughly 70 hours
while
merge sort takes
roughly 40 seconds
This is a slow machine, but if
100 x as fast then it’s 40 minutes
versus less than 0.5 seconds

How to calculate the algorithm’s complexity
 We may not be able to predict to the nanosecond
how long a Python program will take, but do know
some things about timing:
 This loop takes time k*n, for some constants k.
k : How long it takes to go through the loop once
n : The number of times through the loop
(we can use this as the “size” of the problem)
 The total time k*n is linear in n
for i in range(0, n):
print(i);

41
Constant time
• Constant time means there is some
constant k such that this operation
always takes k nanoseconds
• A Java statement takes constant time
if:
• It does not include a loop
• It does not include calling a
method whose time is
unknown or is not a constant
• If a statement involves a choice (if or
switch) among operations, each of
which takes constant time, we
consider the statement to take
constant time
• This is consistent with worst-case analysis

42
Prefix Averages (Quadratic)
The following algorithm computes prefix averages
in quadratic time by applying the definition

43
Prefix Averages 2 (Looks
Better)
The following algorithm uses an internal Python
function to simplify the code
Algorithm prefixAverage2 still runs in O(n2
) time!

44
Prefix Averages 3 (Linear
Time)
The following algorithm computes prefix averages
in linear time by keeping a running sum
Algorithm prefixAverage3 runs in O(n) time

algorithms and data structure Time complexity

Recommended

More Related Content

Similar to algorithms and data structure Time complexity (20)

Recently uploaded (20)

algorithms and data structure Time complexity

Editor's Notes