Introduction to Algorithm for computer engineering students

2
2
What is an algorithm?
What is an algorithm?
An algorithm is “a finite set of precise
An algorithm is “a finite set of precise
instructions for performing a computation or for
instructions for performing a computation or for
solving a problem”
solving a problem”

A program is one type of algorithm
A program is one type of algorithm
All programs are algorithms
All programs are algorithms
Not all algorithms are programs!
Not all algorithms are programs!

Directions to somebody’s house is an algorithm
Directions to somebody’s house is an algorithm

A recipe for cooking a cake is an algorithm
A recipe for cooking a cake is an algorithm

The steps to compute the cosine of 90
The steps to compute the cosine of 90°
° is an
is an
algorithm
algorithm

3
3
Some algorithms are harder than
Some algorithms are harder than
others
others
Some algorithms are easy
Some algorithms are easy

Finding the largest (or smallest) value in a list
Finding the largest (or smallest) value in a list

Finding a specific value in a list
Finding a specific value in a list
Some algorithms are a bit harder
Some algorithms are a bit harder

Sorting a list
Sorting a list
Some algorithms are very hard
Some algorithms are very hard

Finding the shortest path between Miami and Seattle
Finding the shortest path between Miami and Seattle
Some algorithms are essentially impossible
Some algorithms are essentially impossible

Factoring large composite numbers
Factoring large composite numbers
In section 2.2, we’ll see how to rate how “hard”
In section 2.2, we’ll see how to rate how “hard”
algorithms are
algorithms are

4
4
Algorithm 1: Maximum element
Given a list, how do we find the maximum
Given a list, how do we find the maximum
element in the list?
element in the list?
To express the algorithm, we’ll use
To express the algorithm, we’ll use
pseudocode
pseudocode

Pseudocode is kinda like a programming
Pseudocode is kinda like a programming
language, but not really
language, but not really

5
5
Algorithm for finding the maximum element
Algorithm for finding the maximum element
in a list:
in a list:
procedure
procedure max (
max (a
a1
1,
, a
a2
2, …,
, …, a
an
n: integers)
: integers)
max
max :=
:= a
a1
1
for
for i := 2
i := 2 to
to n
n
if
if max <
max < a
ai
i then
then max
max :=
:= a
ai
i
{
{max
max is the largest element}
is the largest element}

6
6
procedure
procedure max (
max (a
a1
1,
, a
a2
2, …,
, …, a
an
n: integers)
: integers)
max
max :=
:= a
a1
1
for
for i
i := 2
:= 2 to
to n
n
if
if max <
max < a
ai
i then
then max
max :=
:= a
ai
i
max
max :=
:= a
a1
1
for
for i
i := 2
:= 2 to
to n
n
if
if max <
max < a
ai
i then
then max
max :=
:= a
ai
i
4
4 1
1 7
7 0
0 5
5 2
2 9
9 3
3 6
6 8
8
a
a1
1 a
a2
2 a
a3
3 a
a4
4 a
a5
5 a
a6
6 a
a7
7 a
a8
8 a
a9
9 a
a10
10
max
max
i
i 2
3
4
5
6
7
8
9
10
4
7
9

7
7
Maximum element running time
Maximum element running time
How long does this take?
If the list has
If the list has n
n elements, worst case
elements, worst case
scenario is that it takes
scenario is that it takes n
n “steps”
“steps”

Here, a step is considered a single step
through the list
through the list

8
8
Properties of algorithms
Properties of algorithms
Algorithms generally share a set of properties:
Algorithms generally share a set of properties:

Input: what the algorithm takes in as input
Input: what the algorithm takes in as input

Output: what the algorithm produces as output
Output: what the algorithm produces as output

Definiteness: the steps are defined precisely
Definiteness: the steps are defined precisely

Correctness: should produce the correct output
Correctness: should produce the correct output

Finiteness: the steps required should be finite
Finiteness: the steps required should be finite

Effectiveness: each step must be able to be
Effectiveness: each step must be able to be
performed in a finite amount of time
performed in a finite amount of time

Generality: the algorithm
Generality: the algorithm should
should be applicable to all
be applicable to all
problems of a similar form
problems of a similar form

9
9
Searching algorithms
Searching algorithms
Given a list, find a specific element in the
Given a list, find a specific element in the
list
list
We will see two types

Linear search
Linear search
a.k.a. sequential search
a.k.a. sequential search

Binary search
Binary search

10
10
Algorithm 2: Linear search
Algorithm 2: Linear search
Given a list, find a specific element in the list

List does NOT have to be sorted!
List does NOT have to be sorted!
procedure
procedure linear_search (
linear_search (x
x: integer;
: integer; a
a1
1,
, a
a2
2, …,
, …, a
an
n:
:
integers)
integers)
i
i := 1
:= 1
while
while (
( i
i ≤
≤ n
n and
and x
x ≠
≠ a
ai
i )
)
i
i :=
:= i
i + 1
+ 1
if
if i
i ≤
≤ n
n then
then location
location := i
:= i
else
else location
location := 0
:= 0
{
{location
location is the subscript of the term that equals x, or it is
is the subscript of the term that equals x, or it is
0 if x is not found}
0 if x is not found}

11
11
procedure
linear_search (x
x: integer;
: integer; a
a1
1,
, a
a2
2, …,
, …, a
an
n: integers)
: integers)
i
i := 1
:= 1
while
while (
( i
i ≤
≤ n
n and
and x
x ≠
≠ a
ai
i )
)
i
i :=
:= i
i + 1
+ 1
if
if i
i ≤
≤ n
n then
then location
location := i
:= i
else
else location
location := 0
:= 0
i
i := 1
:= 1
while
while (
( i
i ≤
≤ n
n and
and x
x ≠
≠ a
ai
i )
)
i
i :=
:= i
i + 1
+ 1
if
if i
i ≤
≤ n
n then
then location
location := i
:= i
else
else location
location := 0
:= 0
Algorithm 2: Linear search, take 1
4
4 1
1 7
7 0
0 5
5 2
2 9
9 3
3 6
6 8
8
a
a1
1 a
a2
2 a
a3
3 a
a4
4 a
a5
5 a
a6
6 a
a7
7 a
a8
8 a
a9
9 a
a10
10
i
i 2
3
4
5
6
7
8
1
x
x 3
location
location 8

12
12
procedure
linear_search (x
x: integer;
: integer; a
a1
1,
, a
a2
2, …,
, …, a
an
n: integers)
: integers)
i
i := 1
:= 1
while
while (
( i
i ≤
≤ n
n and
and x
x ≠
≠ a
ai
i )
)
i
i :=
:= i
i + 1
+ 1
if
if i
i ≤
≤ n
n then
then location
location := i
:= i
else
else location
location := 0
:= 0
i
i := 1
:= 1
while
while (
( i
i ≤
≤ n
n and
and x
x ≠
≠ a
ai
i )
)
i
i :=
:= i
i + 1
+ 1
if
if i
i ≤
≤ n
n then
then location
location := i
:= i
else
else location
location := 0
:= 0
4
4 1
1 7
7 0
0 5
5 2
2 9
9 3
3 6
6 8
8
a
a1
1 a
a2
2 a
a3
3 a
a4
4 a
a5
5 a
a6
6 a
a7
7 a
a8
8 a
a9
9 a
a10
10
i
i 2
3
4
5
6
7
8
9
10
1
x
x 11
location
location 0
11

13
13
Linear search running time
Linear search running time
If the list has
If the list has n
n elements, worst case
elements, worst case
scenario is that it takes
scenario is that it takes n
n “steps”
“steps”

through the list
through the list

14
14
Algorithm 3: Binary search

List MUST be sorted!
List MUST be sorted!
Each time it iterates through, it cuts the list in half
Each time it iterates through, it cuts the list in half
procedure
procedure binary_search (
binary_search (x
x: integer;
: integer; a
a1
1,
, a
a2
2, …,
, …, a
an
n: increasing integers)
: increasing integers)
i
i := 1
:= 1 {
{ i
i is left endpoint of search interval }
is left endpoint of search interval }
j
j :=
:= n
n {
{ j
j is right endpoint of search interval }
is right endpoint of search interval }
while
while i
i <
< j
j
begin
begin
m :=
m := 
(
(i
i+
+j
j)/2
)/2
 {
{ m
m is the point in the middle }
is the point in the middle }
if
if x >
x > a
am
m then
then i
i :=
:= m
m+1
+1
else
else j
j :=
:= m
m
end
end
if
if x
x =
= a
ai
i then
then location
location :=
:= i
i
else
else location
location := 0
:= 0
{
{location
location is the subscript of the term that equals x, or it is 0 if x is not found}
is the subscript of the term that equals x, or it is 0 if x is not found}

15
15
Algorithm 3: Binary search, take 1
2
2 4
4 6
6 8
8 10
10 12
12 14
14 16
16 18
18 20
20
a
a1
1 a
a2
2 a
a3
3 a
a4
4 a
a5
5 a
a6
6 a
a7
7 a
a8
8 a
a9
9 a
a10
10
i
i j
j
m
m
i
i := 1
:= 1
j
j :=
:= n
n
procedure
binary_search (x
x: integer;
: integer; a
a1
1,
, a
a2
2, …,
, …, a
an
while
while i
i <
< j
j
begin
begin
m :=
m := 
(
(i
i+
+j
j)/2
)/2

if
if x >
x > a
am
m then
then i
i :=
:= m
m+1
+1
else
else j
j :=
:= m
m
end
end
if
if x
x =
= a
ai
i then
then location
location :=
:= i
i
else
else location
location := 0
:= 0
i
i := 1
:= 1
j
j :=
:= n
n
while
while i
i <
< j
j
begin
begin
m :=
m := 
(
(i
i+
+j
j)/2
)/2

if
if x >
x > a
am
m then
then i
i :=
:= m
m+1
+1
else
else j
j :=
:= m
m
end
end
if
if x
x =
= a
ai
i then
then location
location :=
:= i
i
1
x
x 14
10
5
6 8 8
7 7
6
7
location
location 7

16
16
2
2 4
4 6
6 8
8 10
10 12
12 14
14 16
16 18
18 20
20
a
a1
1 a
a2
2 a
a3
3 a
a4
4 a
a5
5 a
a6
6 a
a7
7 a
a8
8 a
a9
9 a
a10
10
i
i j
j
m
m
i
i := 1
:= 1
j
j :=
:= n
n
procedure
binary_search (x
x: integer;
: integer; a
a1
1,
, a
a2
2, …,
, …, a
an
while
while i
i <
< j
j
begin
begin
m :=
m := 
(
(i
i+
+j
j)/2
)/2

if
if x >
x > a
am
m then
then i
i :=
:= m
m+1
+1
else
else j
j :=
:= m
m
end
end
if
if x
x =
= a
ai
i then
then location
location :=
:= i
i
else
else location
location := 0
:= 0
i
i := 1
:= 1
j
j :=
:= n
n
while
while i
i <
< j
j
begin
begin
m :=
m := 
(
(i
i+
+j
j)/2
)/2

if
if x >
x > a
am
m then
then i
i :=
:= m
m+1
+1
else
else j
j :=
:= m
m
end
end
if
if x
x =
= a
ai
i then
then location
location :=
:= I
I
else
else location
location := 0
:= 0
1
x
x 15
10
5
6 8 8
7
location
location 0
8

17
17
A somewhat alternative view of what a
A somewhat alternative view of what a
binary search does…
binary search does…

18
18
Binary search running time
Binary search running time
How long does this take (worst case)?
How long does this take (worst case)?
If the list has 8 elements

It takes 3 steps
It takes 3 steps

It takes 4 steps
It takes 4 steps

It takes 6 steps
It takes 6 steps
If the list has
If the list has n
n elements
elements

It takes log
It takes log2
2 n
n steps
steps

19
19
Sorting algorithms
Sorting algorithms
Given a list, put it into some order
Given a list, put it into some order

Numerical, lexicographic, etc.
Numerical, lexicographic, etc.

Bubble sort
Bubble sort

Insertion sort
Insertion sort

20
20
Algorithm 4: Bubble sort
One of the most simple sorting algorithms
One of the most simple sorting algorithms

Also one of the least efficient
Also one of the least efficient
It takes successive elements and “bubbles” them
It takes successive elements and “bubbles” them
up the list
up the list
procedure
procedure bubble_sort (
bubble_sort (a
a1
1,
, a
a2
2, …,
, …, a
an
n)
)
for
for i
i := 1
:= 1 to
to n
n-1
-1
for
for j
j := 1
:= 1 to
to n
n-
-i
i
if
if a
aj
j >
> a
aj
j+1
+1
then
then interchange
interchange a
aj
j and
and a
aj
j+1
+1
{
{ a
a1
1, …,
, …, a
an
n are in increasing order }
are in increasing order }

22
22
An example using physical objects…
An example using physical objects…

23
23
Bubble sort running time
Bubble sort running time
Bubble sort algorithm:
Bubble sort algorithm:
for
for i
i := 1
:= 1 to
to n
n-1
-1
for
for j
j := 1
:= 1 to
to n
n-
-i
i
if
if a
aj
j >
> a
aj
j+1
+1
then
then interchange
interchange a
aj
j and
and a
aj
j+1
+1
Outer for loop does n-1 iterations
Outer for loop does n-1 iterations
Inner for loop does
Inner for loop does

n
n-1 iterations the first time
-1 iterations the first time

n
n-2 iterations the second time
-2 iterations the second time

…
…

1 iteration the last time
1 iteration the last time
Total: (
Total: (n
n-1) + (
-1) + (n
n-2) + (
-2) + (n
n-3) + … + 2 + 1 = (
-3) + … + 2 + 1 = (n
n2
2
-
-n
n)/2
)/2

We can say that’s “about”
We can say that’s “about” n
n2
2
time
time

25
25
Algorithm 5: Insertion sort
Algorithm 5: Insertion sort
Another simple (and inefficient) algorithm
Another simple (and inefficient) algorithm
It starts with a list with one element, and inserts new elements into
It starts with a list with one element, and inserts new elements into
their proper place in the sorted part of the list
their proper place in the sorted part of the list
procedure
procedure insertion_sort (
insertion_sort (a
a1
1,
, a
a2
2, …,
, …, a
an
n)
)
for
for j
j := 2
:= 2 to
to n
n
begin
begin
i
i := 1
:= 1
while
while a
aj
j >
> a
ai
i
i
i :=
:= i
i +1
+1
m
m :=
:= a
aj
j
for
for k
k := 0
:= 0 to
to j
j-
-i
i-1
-1
a
aj
j-
-k
k :=
:= a
aj
j-
-k
k-1
-1
a
ai
i :=
:= m
m
end
end {
{ a
a1
1,
, a
a2
2, …,
, …, a
an
n are sorted }
are sorted }
take successive elements in the list
find where that element should be
in the sorted portion of the list
move all elements in the sorted
portion of the list that are greater
than the current element up by one
put the current element into it’s proper place in the sorted portion of the list

26
26
Insertion sort running time
Insertion sort running time
for
for j
j := 2
:= 2 to
to n
n begin
begin
i
i := 1
:= 1
while
while a
aj
j >
> a
ai
i
i
i :=
:= i
i +1
+1
m
m :=
:= a
aj
j
for
for k
k := 0
:= 0 to
to j
j-
-i
i-1
-1
a
aj
j-
-k
k :=
:= a
aj
j-
-k
k-1
-1
a
ai
i :=
:= m
m
end
end {
{ a
a1
1,
, a
a2
2, …,
, …, a
an
n are sorted }
are sorted }
Outer for loop runs
Outer for loop runs n
n-1 times
-1 times
In the inner for loop:
In the inner for loop:

Worst case is when the while keeps
Worst case is when the while keeps i
i at 1, and the for loop runs lots of
at 1, and the for loop runs lots of
times
times

If
If i
i is 1, the inner for loop runs 1 time (
is 1, the inner for loop runs 1 time (k
k goes from 0 to 0) on the first
goes from 0 to 0) on the first
iteration, 1 time on the second, up to
iteration, 1 time on the second, up to n
n-2 times on the last iteration
-2 times on the last iteration
Total is 1 + 2 + … +
Total is 1 + 2 + … + n
n-2 = (
-2 = (n
n-1)(
-1)(n
n-2)/2
-2)/2

We can say that’s “about”
We can say that’s “about” n
n2
2
time
time

27
27
Comparison of running times
Comparison of running times
Searches
Searches

Linear:
Linear: n
n steps
steps

Binary: log
Binary: log2
2 n
n steps
steps

Binary search is about as fast as you can get
Binary search is about as fast as you can get
Sorts
Sorts

Bubble:
Bubble: n
n2
2
steps
steps

Insertion:
Insertion: n
n2
2
steps
steps

There are other, more efficient, sorting techniques
There are other, more efficient, sorting techniques
In principle, the fastest are heap sort, quick sort, and merge
In principle, the fastest are heap sort, quick sort, and merge
sort
sort
These each take take
These each take take n
n * log
* log2
2 n
n steps
steps
In practice, quick sort is the fastest, followed by merge sort
In practice, quick sort is the fastest, followed by merge sort

Introduction to Algorithm for computer engineering students

Recommended

More Related Content

Similar to Introduction to Algorithm for computer engineering students (20)

Recently uploaded (20)

Introduction to Algorithm for computer engineering students