CSC2105: Algorithms

Greedy Graph Algorithm

Mashiour Rahman
[email protected]
American International University Bangladesh
 Spanning Tree
• A spanning tree of G is a subgraph which
– is a tree
– contains all vertices of G

 How many edges

are there in a
spanning tree, if
there are V
vertices?

CSC4226: Algorithms Mashiour Rahman Graph Algorithm2

 Minimum Spanning Trees
• Undirected, connected graph
G = (V,E)
• Weight function W: E  
(assigning cost or length or other
values to edges)

 Spanning tree: tree that connects all vertices

 Minimum spanning tree (MST): spanning tree
T that minimizes w(T )  
( u , v )T
w(u , v)

CSC4226: Algorithms Mashiour Rahman Graph Algorithm3

 Optimal Substructure
MST(G ) = T MST(G’ ) = T – (u,v )
u ”u +v”
v

• Rationale:
– If G’ would have a cheaper ST T’, then we would get
a cheaper ST of G: T’ + (u, v)

CSC4226: Algorithms Mashiour Rahman Graph Algorithm4

 Idea for an Algorithm
• We have to make V–1 choices (edges of the
MST) to arrive at the optimization goal.
• After each choice we have a sub-problem that
is one vertex smaller than the original problem.
– A dynamic programming algorithm would consider all
possible choices (edges) at each vertex.
– Goal: at each vertex cheaply determine an edge that
definitely belongs to an MST.

CSC4226: Algorithms Mashiour Rahman Graph Algorithm5

 Greedy Choice
• Greedy choice property: locally optimal
(greedy) choice yields a globally optimal
solution.
• Theorem on Greedy choice for MST
– Let G=(V, E) and S V
– S is a cut of G (it splits G into parts S and V-S)
– (u,v) is a light edge if it is a min-weight edge of G
that connects S and V-S
– Then (u,v) belongs to a MST T of G.

CSC4226: Algorithms Mashiour Rahman Graph Algorithm6

 ...Greedy Choice
• Proof
– Suppose (u,v) is light but (u,v)  any MST
– look at path from u to v in some MST T
– Let (x, y) be the first edge on a path from u to v in T that crosses
from S to V–S. Swap (x, y) with (u,v) in T.
– this improves cost of T  contradiction (T is supposed to be an
MST)

V-S
S
y
x
u v

CSC4226: Algorithms Mashiour Rahman Graph Algorithm7

 Generic MST Algorithm
Generic-MST
Generic-MST(G, (G, w)w)
11 AA := 
:= // Contains edges
// Contains edges that
that belong
belong to
to aa MST
MST
22 while A does not form a spanning
while A does not form a spanning tree do tree do
33 Find
Find an an edge
edge (u,v)
(u,v) that
that is
is safe
safe for
for AA
44 := AA 
AA := {(u,v)}
{(u,v)}
55 return
return A A

A safe edge is an edge that does not destroy A’s property.

MoreSpecific-MST(G,
MoreSpecific-MST(G, w) w)
:= //
11 AA := // Contains
Contains edges
edges that
that belong
belong toto aa MST
MST
22 while
while AA does
does not
not form
form aa spanning
spanning tree
tree do
do
3.1
3.1 Make
Make aa cut
cut (S,
(S, V-S)
V-S) of
of GG that
that respects
respects AA
3.2
3.2 Take
Take the
the min-weight
min-weight edge
edge (u,v)
(u,v) connecting
connecting SS toto V-S
V-S

44 := AA {(u,v)}
AA := {(u,v)}
55 return
return AA

CSC4226: Algorithms Mashiour Rahman Graph Algorithm8

 Prim-Jarnik Algorithm
• Vertex based algorithm
• Grows a single MST T one vertex at a time
• The set A covers the portion of T that was already
computed
• Annotate all vertices v outside of the set A with v.key the
minimum weight of an edge that connects v to a vertex
in A
(v.key =  if no such edge exists)

CSC4226: Algorithms Mashiour Rahman Graph Algorithm9

 ...Prim-Jarnik Algorithm
MST-Prim(G, s)
01 for each vertex u  G.V
02 u.key := 
03 u.pred := NIL
04 s.key := 0
05 init(Q, G.V) // Q is a priority queue
06 while not isEmpty(Q)
07 u := extractMin(Q) // add u to T
08 for each v  u.adj do
09 if v  Q and w(u,v) < v.key then
10 v.key := w(u,v)
11 modifyKey(Q,v)
12 v.pred := u updating
keys

CSC4226: Algorithms Mashiour Rahman Graph Algorithm10

 Prim-Jarnik’s Example
Q
Node Pred key

6
8 C D
4 B A NIL 0
3 4 13 9
A 12 B NIL 
6 I
8 H 10 C NIL 
F E
5 3 D NIL 
MST-Prim(Graph,A) 1
G E NIL 

F NIL 
A Node
G NIL 
Pred

Key H NIL 

I NIL 

CSC4226: Algorithms Mashiour Rahman Graph Algorithm11

 ...Prim-Jarnik’s Example
Q
Node Pred key

6
8 C D
4 B B A 4
3 4 13 9
A 12 H A 8
6 I
8 H 10 C NIL 
F E
5 3
D NIL 
1
G
E NIL 

F NIL 
A Node A

Pred NIL G NIL 

Key 0
I NIL 

CSC4226: Algorithms Mashiour Rahman Graph Algorithm12

 ...Prim-Jarnik’s Example
Q
Node Pred key

6
8 C D
4 B H A 8
3 4 13 9
A 12 C B 8
6 I
8 H 10 F NIL 
F E
5 3
E NIL 
1
G
F NIL 

G NIL 
A Node A B

Pred NIL A I NIL 

Key 0 4

CSC4226: Algorithms Mashiour Rahman Graph Algorithm13

 ...Prim-Jarnik’s Example
Q
Node Pred key

6
8 C D
4 B G H 1
3 4 13 9
A 12 I H 6
6 I
8 H 10 C B 8
F E
5 3
D NIL 
1
G
E NIL 

F NIL 
A Node A B H

Pred NIL A A

Key 0 4 8

CSC4226: Algorithms Mashiour Rahman Graph Algorithm14

 ...Prim-Jarnik’s Example
Q
Node Pred key

6
8 C D
4 B F G 3
3 4 13 9
A 12 I G 5
6 I
8 H 10 C B 8
F E
5 3
D NIL 
1
G
E NIL 

A Node A B H G

Pred NIL A A H

Key 0 4 8 1

CSC4226: Algorithms Mashiour Rahman Graph Algorithm15

 ...Prim-Jarnik’s Example
Q
Node Pred key

6
8 C D
4 B C F 4
3 4 13 9
A 12 I G 5
6 I
8 H 10 E F 10
F E
5 3
D F 13
1
G

A Node A B H G F

Pred NIL A A H G

Key 0 4 8 1 3

CSC4226: Algorithms Mashiour Rahman Graph Algorithm16

 ...Prim-Jarnik’s Example
Q
Node Pred key

6
8 C D
4 B I C 3
3 4 13 9
A 12 D C 6
6 I
8 H 10 E F 10
F E
5 3
1
G

A Node A B H G F C

Pred NIL A A H G F

Key 0 4 8 1 3 4

CSC4226: Algorithms Mashiour Rahman Graph Algorithm17

 ...Prim-Jarnik’s Example
Q
Node Pred key

6
8 C D
4 B D C 6
3 4 13 9
A 12 E F 10
6 I
8 H 10
F E
5 3
1
G

A Node A B H G F C I

Pred NIL A A H G F C

Key 0 4 8 1 3 4 3

CSC4226: Algorithms Mashiour Rahman Graph Algorithm18

 ...Prim-Jarnik’s Example
Q
Node Pred key

6
8 C D
4 B E D 9
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

A Node A B H G F C I D

Pred NIL A A H G F C C

Key 0 4 8 1 3 4 3 6

CSC4226: Algorithms Mashiour Rahman Graph Algorithm19

 ...Prim-Jarnik’s Example
Q is empty
Node Pred key

6
8 C D
4 B
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

A Node A B H G F C I D E

Pred NIL A A H G F C C D

Key 0 4 8 1 3 4 3 6 9

CSC4226: Algorithms Mashiour Rahman Graph Algorithm20

 ...Prim-Jarnik’s Example
Q is empty
Node Pred Key

6
C D
4 B
3 4 9
A
I
8 H
F E
3
1
G

A Node A B H G F C I D E Total
Cost
Pred NIL A A H G F C C D

Key 0 4 8 1 3 4 3 6 9 38

CSC4226: Algorithms Mashiour Rahman Graph Algorithm21

 Implementation Issues
MST-Prim(G,r)
01 for each vertex u  G.V
02 u.key := 
03 u.parent := NIL
04 r.key := 0
05 init(Q, G.V) // Q is a priority queue
06 while not isEmpty(Q)
07 u := extractMin(Q) // add u to T
08 for each v  u.adj do
09 if v  Q and w(u,v) < v.key then
10 v.key := w(u,v)
11 modifyKey(Q,v)
12 v.parent := u

CSC4226: Algorithms Mashiour Rahman Graph Algorithm22

 Priority Queues
• A priority queue maintains a set S of elements, each with
an associated key value.
• We need PQ to support the following operations
– init( Q: PriorityQueue, S: VertexSet)
– extractMin( Q: PriorityQueue): Vertex
– modifyKey( Q: PriorityQueue, v: Vertex)

• To choose how to implement a PQ, we need to count

how many times the operations are performed:
– init is performed just once and runs in O(n)

CSC4226: Algorithms Mashiour Rahman Graph Algorithm23

 Prim-Jarnik’s Running Time
• Time = |V|*T(extractMin) + O(E)*T(modifyKey)

Q T(extractMin) T(modifyKey) Total

array O(V) O(1) O(V2)
binary heap O(log V) O(log V) O(E logV)

• E  V-1, E < V2, E = O(V2)

• Binary heap implementation:
– Time = O(V logV + E logV) = O(V2 logV) = O(E logV)

CSC4226: Algorithms Mashiour Rahman Graph Algorithm24

 Kruskal's Algorithm
• Edge based algorithm
• Add edges one at a time in increasing
weight order.
• The algorithm maintains A: a forest of
trees.
• An edge is accepted if it connects
vertices of distinct trees (the cut
respects A).

CSC4226: Algorithms Mashiour Rahman Graph Algorithm25

 ...Kruskal's Algorithm
• The algorithm keeps adding the cheapest edge
that connects two trees of the forest
MST-Kruskal(G)
01 A := 
02 init(S) // Init disjoint-set
03 for each vertex v  G.V
04 addSingletonSet(S,v)
05 sort the edges of G.E by non-decreasing w(u,v)
06 for each edge (u,v)  E in sorted order do
07 if findSet(S,u)  findSet(S,v) then
08 A := A  {(u,v)}
09 unionSets(S,u,v)
10 return A
CSC4226: Algorithms Mashiour Rahman Graph Algorithm26
 Kruskal’s Example

6
8 C D
4 B
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

S = { A, B, C, D, E, F, G,
G, H,
H, I }
E’ = { HG-1,
HG-1, CI-3, GF-3, CF-4, AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }

CSC4226: Algorithms Mashiour Rahman Graph Algorithm27

 ...Kruskal’s Example

6
8 C D
4 B
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

S = { A, B, C, D, E, F, GH, I }
E’ = { CI-3, GF-3, CF-4, AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }

CSC4226: Algorithms Mashiour Rahman Graph Algorithm28

 ...Kruskal’s Example

6
8 C D
4 B
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

S = { A, B, CI, D, E, F, GH }
E’ = { GF-3, CF-4, AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }

CSC4226: Algorithms Mashiour Rahman Graph Algorithm29

 ...Kruskal’s Example

6
8 C D
4 B
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

S = { A, B, CI, D, E, FGH }
E’ = { CF-4, AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }

CSC4226: Algorithms Mashiour Rahman Graph Algorithm30

 ...Kruskal’s Example

6
8 C D
4 B
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

S = { A, B, CFGHI, D, E }
E’ = { AB-4, HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }

CSC4226: Algorithms Mashiour Rahman Graph Algorithm31

 ...Kruskal’s Example

6
8 C D
4 B
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

S = { AB, CFGHI, D, E }
E’ = { HI-6, CD-6, BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }

CSC4226: Algorithms Mashiour Rahman Graph Algorithm32

 ...Kruskal’s Example

6
8 C D
4 B
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

S = { AB, CDFGHI, E }
E’ = { BC-8, AH-8, DE-9, EF-10, BH-12, DF-13 }

CSC4226: Algorithms Mashiour Rahman Graph Algorithm33

 ...Kruskal’s Example

6
8 C D
4 B
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

S = { ABCDFGHI, E }
E’ = { AH-8, DE-9, EF-10, BH-12, DF-13 }

CSC4226: Algorithms Mashiour Rahman Graph Algorithm34

 ...Kruskal’s Example

6
8 C D
4 B
3 4 13 9
A 12
6 I
8 H 10
F E
5 3
1
G

S = { ABCDEFGHI }
E’ = { EF-10, BH-12, DF-13 }

CSC4226: Algorithms Mashiour Rahman Graph Algorithm35

 ...Kruskal’s Example

6
8 C D
4 B
3 4 9
A
I
H E
3 F
1
G

S = { ABCDEFGHI }
E’ = { HG-1, CI-3, GF-3, CF-4, AB-4, CD-6, BC-8, DE-9 }; Total Cost := 38

CSC4226: Algorithms Mashiour Rahman Graph Algorithm36

 Implementation Factors:
Disjoint Sets
• We need to maintain a disjoint partitioning of a
set, i.e., a collection S of disjoint sets.
Operations:
– addSingeltonSet(S:Set, x:Vertex)
• S := S  {{x}}
– findSet(S:Set, x:Vertex): Set
• returns X  S such that x  X
– unionSets(S:Set, x:Vertex, y:Vertex)
• X := findSet(S:Set, x)
• Y := findSet(S:Set, y)
• S := S – {X, Y}  {X Y}

CSC4226: Algorithms Mashiour Rahman Graph Algorithm37

 Implementation Factors:
Disjoint Sets as Lists
• Each set is a list of elements identified by the first
element, all elements in the list point to the first element
• UnionSets: add a shorter list to a longer one, O(min{|
C(u)|, |C(v)|})
• MakeSet/FindSet: O(1)

1 2 3 4 A B C
 

1 2 3 4 A B C


CSC4226: Algorithms Mashiour Rahman Graph Algorithm38

 Kruskal Running Time
• Initialization O(V) time
• Sorting the edges (E log E) = (E log V)
• O(E) calls to FindSet
• Union costs
– Let t(v) be the number of times v is moved to a new cluster
– Each time a vertex is moved to a new cluster the size of the
cluster containing the vertex at least doubles: t(v) log V
– Total time spent doing Union  t (v)  V log V
vV

• Total time: O(E log V)

CSC4226: Algorithms Mashiour Rahman Graph Algorithm39