Dynamic Programming: Memoization, Introduction to ALgorithms

Introduction
CSE 221: Algorithms
Dynamic Programming
Mumit Khan
Fatema Tuz Zohora
Computer Science and Engineering
BRAC University
References
1 Jon Kleinberg and Éva Tardos, Algorithm Design. Pearson Education, 2006.
2 T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms, Second Edition.
The MIT Press, September 2001.
Last modified: November 27, 2012
This work is licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.
Licensed under CSE 221: Algorithms 1 / 53

Introduction Memoization Dynamic programming Weighted interval sched
Contents
1 Introduction
Memoization
Dynamic programming
Weighted interval scheduling problem
0/1 Knapsack problem
Coin changing problem
What problems can be solved by DP?
Conclusion

Dynamic Programming (DP)
Build up the solution by computing solutions to the
subproblems.

subproblems.
Don’t solve the same subproblem twice, but rather save the
solution so it can be re-used later on.

subproblems.
Often used for a large class to optimization problems.

subproblems.
Unlike Greedy algorithms, implicitly solve all subproblems.

subproblems.
Motivating the case for DP with Memoization – a top-down
technique, and then moving on to Dynamic Programming – a
bottom-up technique.

subproblems.
Motivating the case for DP with Memoization – a top-down
technique, and then moving on to Dynamic Programming – a
bottom-up technique.
Greedy is evil, Dynamic Programming is good. – Prof. Jeff
Erickson, University of Illinois, Urbana-Champaign.

Contents
1 Introduction
Memoization
Dynamic programming
Conclusion

Recursive solution to Fibonacci numbers
Definition (Fibonacci numbers)
The Fibonacci numbers are given by the following sequence:
h0, 1, 1, 2, 3, 5, 8, 21, 34, 55, 89, . . .i

h0, 1, 1, 2, 3, 5, 8, 21, 34, 55, 89, . . .i
and described by the following recurrence.
Fib(n) =
(
n if n = 0 or 1
Fib(n − 1) + Fib(n − 2) if n ≥ 2

h0, 1, 1, 2, 3, 5, 8, 21, 34, 55, 89, . . .i
and described by the following recurrence.
Fib(n) =
(
n if n = 0 or 1
Fib(n − 1) + Fib(n − 2) if n ≥ 2
Straightforward recursive algorithm
Fibonacci(n) n ≥ 0
1 if n = 0 or n = 1
2 then return n
3 else return fibonacci(n − 1) + fibonacci(n − 2)

Recursion tree

Recursion tree
Complexity
This recursive algorithm for Fibonacci numbers has exponential
running time!

Recursion tree
Complexity
This recursive algorithm for Fibonacci numbers has exponential
running time!
To be precise, T(n) = O(ϕn) , where ϕ = 1+
√
5
2 is the golden
ratio.

Redundant computations
Note how fib(n − 2) and fib(n − 3) are each being computed
twice.

In fact, computing fib(n − 2) involves computing a whole
subtree.

Likewise for computing fib(n − 3).

Observations
Spectacular redundancy in computation

Observations
Spectacular redundancy in computation – how many times are we
computing fib(n − 2)?

Observations
computing fib(n − 2)? fib(n − 3)?

Observations
What if we compute and save the result of fib(i) for i = {2, 3, . . , n} the
first time, and then re-use it each time afterward?

Observations
What if we compute and save the result of fib(i) for i = {2, 3, . . , n} the
first time, and then re-use it each time afterward?
Ah, we’ve just (re)discovered Memo(r)ization!

Memoization
Definition (Memoization)
The process of saving solutions to subproblems that can be re-used
later without redundant computations.

Memoization
Basic idea
Typically, the solutions to subproblems (i.e., the intermediate
solutions) are saved in a global array, which are later looked up and
re-used as needed.

Memoization
Basic idea
re-used as needed.
1 At each step of computation, first see if the solution to the
subproblem has already been found and saved.

Memoization
Basic idea
re-used as needed.
2 If so, simply return the solution.

Memoization
Basic idea
re-used as needed.
2 If so, simply return the solution.
3 If not, compute the solution, and save it before returning the
solution.

Memoized recursive algorithm for Fibonacci numbers
M-Fibonacci(n) n ≥ 0, global F = [0 . . n]
1 if n = 0 or n = 1
2 then return n Our base conditions.
3 if F[n] is empty No saved solution found for n.
4 then F[n] ← m-fibonacci(n − 1) + m-fibonacci(n − 2)
5 return F[n]

1 if n = 0 or n = 1
5 return F[n]
Questions
What is this global array F?

1 if n = 0 or n = 1
5 return F[n]
Questions
What is this global array F? It’s used store the values of the
intermediate results, and must be initialized by the caller to
all empty.

1 if n = 0 or n = 1
5 return F[n]
Questions
all empty.
What is an appropriate sentinel to indicate that
F[i], 0 ≤ i ≤ n has not been solved yet (i.e., empty)?

1 if n = 0 or n = 1
5 return F[n]
Questions
all empty.
What is an appropriate sentinel to indicate that
F[i], 0 ≤ i ≤ n has not been solved yet (i.e., empty)? Use −1,
which is guaranteed to be an invalid value.

Memoized . . . Fibonacci numbers (continued)
Allocate an array F[0 . . n] to save results (length[F] = n + 1).
1 for i ← 0 to n
2 do F[i] ← −1 No solution computed for i yet (sentinel)
3 return m-fibonacci(F, n)

1 for i ← 0 to n
M-Fibonacci(F, n) n ≥ 0, F = [0 . . n]
1 if n ≤ 1
2 then return n
3 if F[n] = −1 No saved solution found for n.
4 then F[n] ← m-fibonacci(F, n − 1) + m-fibonacci(F, n − 2)
5 return F[n]

1 for i ← 0 to n
M-Fibonacci(F, n) n ≥ 0, F = [0 . . n]
1 if n ≤ 1
2 then return n
3 if F[n] = −1 No saved solution found for n.
4 then F[n] ← m-fibonacci(F, n − 1) + m-fibonacci(F, n − 2)
5 return F[n]
Running time
Each element F[2] . . . F[n] is filled in just once in Θ(1) time, so
T(n) = Θ(n) .

Memoization highlights
Idea is to re-use saved solutions, trading off space for time.

Any recursive algorithm can be memoized, but only helps if
there is redundancy in computing solutions to subproblems (in
other words, if there are overlapping subproblems).

Any recursive algorithm where redundant solutions are
computed, Memoization is an appropriate solution.

Often called Top-down Dynamic Programming.

Questions to ask (and remember)

What are the drawbacks, if any, of memoization?

Would all recursive algorithms benefit from memoization?

Would all recursive algorithms benefit from memoization?
For example, would the recursive algorithm to compute the
factorial of a number benefit from memoization?

Contents
1 Introduction
Memoization
Dynamic programming
Conclusion

Dynamic programming
Note how the recursive algorithm computes the Fibonacci
number n top down by computing (and saving) solutions for
smaller values.

Dynamic programming
smaller values.
Idea: why not build up the solution bottom-up, starting from
the base case(s) all the way to n?

Dynamic programming
smaller values.
This bottom up construction gives us the first Dynamic
Programming algorithm.

Dynamic programming
smaller values.
Dynamic programming algorithm for fibonacci numbers
1 F[0] ← 0
2 F[1] ← 1
3 for i ← 2 to n
4 do F[i] ← F[i − 1] + F[i − 2]
5 return F[n]

Dynamic programming
smaller values.
Dynamic programming algorithm for fibonacci numbers
1 F[0] ← 0
2 F[1] ← 1
3 for i ← 2 to n
4 do F[i] ← F[i − 1] + F[i − 2]
5 return F[n]
T(n) = Θ(n)

Dynamic programming (continued)
The pattern
1 Formulate the problem recursively.

The pattern
1 Formulate the problem recursively. Write a formula for the
whole problem as a simple combination of of the answers to
smaller subproblems.

The pattern
2 Build solutions to the recurrence from the bottom up.

The pattern
Write an algorithm that starts with the base case, and works
its way up to the final solution by considering the subproblems
in the correct order.

The pattern
Observations
1 Must ensure that the recurrence is correct of course!

The pattern
Observations
2 Need a “place” to store the solutions to subproblems, and
need to look these solutions up when needed.

The pattern
Observations
2 Need a “place” to store the solutions to subproblems, and
need to look these solutions up when needed. Typically, but
not always, a multi-dimensional table is used as storage.

Contents
1 Introduction
Memoization
Dynamic programming
Conclusion

Definition (Weighted interval scheduling problem)
Given a set of schedules I = {Ii }, with associated weights
W = {wi }, find A ⊆ I such that the members of A are
non-conflicting and the total weight
P
i∈A wi is maximized.
Example (an instance of weighted interval problem)
|A| =???,
P
i∈A wi =???.

P
Example (using an optimal strategy)
|A| = 1,
P
i∈A wi = 3.

P
Example (using an optimal strategy)
|A| = 1,
P
i∈A wi = 3.
What now?
First step is to formulate a recursive solution, but first we need to
figure out what the subproblems are.

Developing a recursive solution
Let W be an instance of a weighted interval problem.

As in the greedy approach, we sort the intervals according to
finish times such that fi ≤ fj for i j (“a natural order of the
subproblems”).

subproblems”).
Let ϑ be an optimal solution (even if we have no idea what it
is yet).

subproblems”).
is yet).
All we can say about ϑ is the following: interval n (the last
interval) either belongs to ϑ, or it doesn’t.

subproblems”).
is yet).
If n ∈ ϑ Then clearly all intervals that conflict with n are
not members of ϑ. ϑ then contains n, plus an
optimal solution to all intervals that do not
conflict with n. We now need to have a quick
way of computing list of conflicting intervals for
n.

subproblems”).
is yet).
If n ∈ ϑ Then clearly all intervals that conflict with n are
not members of ϑ. ϑ then contains n, plus an
optimal solution to all intervals that do not
conflict with n. We now need to have a quick
way of computing list of conflicting intervals for
n.
If n /
∈ ϑ Then ϑ contains an optimal solution for the
intervals {i1, i2, . . , in−1}.

Developing a recursive solution (continued)
Example (an instance of a weighted interval problem)
For each interval i, compute p(i), the rightmost interval among
the non-conflicting preceding intervals of i. Define p(j) = 0 if no
request i j is disjoint from j.

For a given interval i, p(i) means that intervals
{p(i) + 1, p(i) + 2, . . . , i − 1} overlap with it. For example,
p(6) = 3, which means that intervals {4, 5} overlap interval 6.

Alternatively, intervals {1, 2, . . , p(i)} do not overlap interval i.
For example, p(6) = 3 means that intervals {1, 2, 3} do not
overlap interval 6.

If n ∈ ϑ, then ϑ must include, in addition to interval n, an
optimal solution to the subproblem consisting of intervals
{1, 2, . . . , p(n)}.

{1, 2, . . . , p(n)}. If ϑ(n) is an optimal solution to the
subproblem for intervals {1, 2, . . . , n}, then:
ϑ(n) = wn + ϑ(p(n))

ϑ(n) = wn + ϑ(p(n))
If n /
∈ ϑ, then ϑ simply contains an optimal solution to the
subproblem consisting of the intervals {1, 2, . . . , n − 1}.

ϑ(n) = wn + ϑ(p(n))
If n /
ϑ(n) = ϑ(n − 1)

ϑ(n) = wn + ϑ(p(n))
If n /
ϑ(n) = ϑ(n − 1)
Since an optimal solution must maximize the sum of the
weights in the intervals it contains, we accept the larger of the
two.

ϑ(n) = wn + ϑ(p(n))
If n /
ϑ(n) = ϑ(n − 1)
two.
ϑ(n) = max(wn + ϑ(p(n)), ϑ(n − 1))

Recursive algorithm for an optimal value
If OPT(j) is an optimal solution to the subproblem for intervals
{1, 2, . . . , j}, for any j ∈ {1, 2, . . . , n}, then:
OPT(j) = max(wj + OPT(p(j)), OPT(j − 1))

{1, 2, . . . , j}, for any j ∈ {1, 2, . . . , n}, then:
Extracting the intervals in an optimal solution
The interval j is in an optimal solution OPT(j) if and only if the
first of the two options is larger than the second.

{1, 2, . . . , j}, for any j ∈ {1, 2, . . . , n}, then:
Extracting the intervals in an optimal solution
The interval j is in an optimal solution OPT(j) if and only if the
Interval j belongs to an optimal solution on the set {1, 2, . . . , j} if
and only if
wj + OPT(p(j)) ≥ OPT(j − 1)

A recursive algorithm
WIS(j)
1 if j = 0
2 then return 0
3 else return max(wj + WIS(p(j)),
WIS(j − 1))

WIS(j)
1 if j = 0
2 then return 0
WIS(j − 1))
The initial call is WIS(n) for intervals {1, 2, . . . , n} sorted in
non-decreasing order of the finishing times.

WIS(j)
1 if j = 0
2 then return 0
WIS(j − 1))
The tree grows very rapidly, leading to exponential running
time. The tree when p(j) = j − 2 for all j shows how quickly
it grows.

WIS(j)
1 if j = 0
2 then return 0
WIS(j − 1))
time. The tree when p(j) = j − 2 for all j shows how quickly
it grows.
There are many overlapping subproblems, so the obvious
choice is to memoize the recursion.

Memoizing the recursion
M-WIS(j)
1 if j = 0
2 then return 0
3 elseif M[j] is empty
4 then M[j] ← max(wj + M-WIS(p(j)),
M-WIS(j − 1))
5 return M[j]

M-WIS(j)
1 if j = 0
2 then return 0
M-WIS(j − 1))
5 return M[j]
Each entry in M[j] gets filled in only once at Θ(1) time, and
there are n + 1 entries, so M-WIS(n) takes Θ(n) time.

M-WIS(j)
1 if j = 0
2 then return 0
M-WIS(j − 1))
5 return M[j]
Of course, sorting the intervals by the finish times takes
Θ(n lg n) time.

M-WIS(j)
1 if j = 0
2 then return 0
M-WIS(j − 1))
5 return M[j]
Of course, sorting the intervals by the finish times takes
Θ(n lg n) time.
This memoized algorithm plus sorting the intervals takes
Θ(n lg n) + Θ(n) = Θ(n lg n) time.

Computing a solution in addition to its values
The memoized algorithm only computes the optimal value,
but does not extract the intervals that make up the solution.
The key to extracting the solution is to note that item j is in
ϑ if and only if wj + M[p(j)] ≥ M[j − 1]. This provides two
ways of extracting the intervals in the optimal solution:
1 Trace back from M[n] and extract the solution by checking
which choice was made – j − 1 or p(j) – when M[j] was
included in the optimal set of intervals.
2 Whenever a choice is made between two options, save in
pred[j], the predecessor pointer, the choice that was made
between j − 1 and p(j).

Computing a solution in addition to its values (continued)
The first way recursively extracts an optimal set of intervals
for a problem size of 1 ≤ j ≤ n.
Calling WIS-find-solution(n) extracts all the intervals in
the optimal solution.

The first way recursively extracts an optimal set of intervals
for a problem size of 1 ≤ j ≤ n.
Calling WIS-find-solution(n) extracts all the intervals in
the optimal solution.
WIS-find-solution(j)
1 if j = 0
2 then Output nothing
3 else
4 if wj + M[p(j)] ≥ M[j − 1]
5 then Output j
6 WIS-find-solution(p(j))
7 else WIS-find-solution(j − 1)

The second way requires that M-WIS use an auxiliary array
pred[0 . . n] to save the predecessor of each interval in the
solution.
Initialize pred[j] = 0 for all 0 ≤ j ≤ n.

The second way requires that M-WIS use an auxiliary array
pred[0 . . n] to save the predecessor of each interval in the
solution.
Initialize pred[j] = 0 for all 0 ≤ j ≤ n.
M-WIS(j)
1 if j = 0
2 then return 0
4 then if wj + M-WIS(p(j)) M-WIS(j − 1))
5 then M[j] ← wj + M-WIS(p(j)
6 pred[j] ← p(j)
7 else M[j] ← M-WIS(j − 1)
8 pred[j] ← j − 1
9 return M[j]

Now that we have pred[j] filled in, we start from M[n] and work
backwards.
1 If pred[j] = p(j), then we did add the jth interval in the final
solution, and we continue with pred[j] ← p(j).
2 if pred[j] 6= p(j), then we did not add the jth interval in the
final solution, and we continue with pred[j] ← j − 1.

backwards.
1 if j = 0
3 else
4 if pred[j] = p(j)
5 then Output j

backwards.
1 if j = 0
3 else
4 if pred[j] = p(j)
5 then Output j
Can you come up with an iterative version?

Developing a Dynamic Programming algorithm
The value of an optimal solution OPT(j) for any
j ∈ {1, 2, 3, . . . , n} depends on the values of OPT(p(j)) and
OPT(j − 1).

OPT(j − 1).
We can build the table M[j] bottom-up, starting from the
base case of j = 0, up to n by using the memoized recursive
formulation: M[j] = max(wj + M[p(j)], M[j − 1]).

OPT(j − 1).
Dynamic programming algorithm
WIS(n)
1 M[0] ← 0
2 for j ← 1 to n
3 do M[j] = max(wj + M[p(j)], M[j − 1])
4 return M[n]

OPT(j − 1).
Dynamic programming algorithm
WIS(n)
1 M[0] ← 0
2 for j ← 1 to n
3 do M[j] = max(wj + M[p(j)], M[j − 1])
4 return M[n]
T(n) = Θ(n)

WIS(n)
1 M[0] ← 0
2 for j ← 1 to n
3 do if wj + M[p(j)] M[j − 1]
4 then M[j] = wj + M[p(j)]
5 pred[j] = p(j)
6 else M[j] = M[j − 1]
7 pred[j] = j − 1
8 return M[n]

WIS(n)
1 M[0] ← 0
2 for j ← 1 to n
3 do if wj + M[p(j)] M[j − 1]
4 then M[j] = wj + M[p(j)]
5 pred[j] = p(j)
6 else M[j] = M[j − 1]
7 pred[j] = j − 1
8 return M[n]
1 j ← n
2 while j 0
3 do if pred[j] = p(j)
4 then Output j
5 j ← pred[j]

Weighted Interval Scheduling DP algorithm in action

Optimal value: 8
Optimal solution: {5, 3, 1}

Optimal value: 8
Optimal solution: {1, 3, 5}

So, you think you understand Dynamic Programming now?
Answer the following questions
1 Instead of sorting the intervals by finish time, what if we
sorted the requests by start time?

2 What if we didn’t sort the requests at all? Would it still work?

3 If all the weights are the same, what does this problem
become?

3 If all the weights are the same, what does this problem
become? Can you solve it using DP?

Contents
1 Introduction
Memoization
Dynamic programming
Conclusion

0/1 knapsack problem
Definition (0/1 knapsack problem)
Given a set S of n items, such that each item i has a positive
benefit vi and a positive weight wi , the goal is to find the
maximum-benefit subset that does not exceed a given weight W .

Formally, we wish to determine a subset of S that maximizes
P
i∈S vi , subject to
P
i∈S wi ≤ W .

P
P
i∈S wi ≤ W .
Maximum weight: W = 4 kg

P
P
i∈S wi ≤ W .
Maximum weight: W = 4 kg
Optimal solution: items B and C Benefit: 370

Let S be an instance of a 0/1 Knapsack problem, and ϑ be an
optimal solution (even if we have no idea what it is yet).

Note that the presence of an item i in ϑ does not preclude
any other item j 6= i in ϑ.

If item n weighs more than the maximum allowed weight, it
will not be in ϑ.

will not be in ϑ.
Otherwise, all we can say about ϑ is the following: item n
(the last one) either belongs to ϑ, or it doesn’t.

will not be in ϑ.
If n ∈ ϑ Then the optimal solution contains n, plus an
optimal solution for the other n − 1 items, but
with a reduced maximum weight of W − wn.

will not be in ϑ.
If n /
∈ ϑ Then ϑ simply contains an optimal solution for
the first n − 1 items, with the maximum allowed
weight W remaining unchanged.

will not be in ϑ.
If n /
∈ ϑ Then ϑ simply contains an optimal solution for
the first n − 1 items, with the maximum allowed
weight W remaining unchanged.
We have two parameters for each subproblem – the items S,
and the maximum allowed weight W .

wn W =⇒ n /
∈ ϑ.
ϑ(n, W ) = ϑ(n − 1, W )

wn W =⇒ n /
∈ ϑ.
ϑ(n, W ) = ϑ(n − 1, W )
Otherwise, n is either ∈ ϑ or /
∈ ϑ.
If n ∈ ϑ, then ϑ(n, W ) is an optimal solution to the
subproblem for items {1, 2, . . . , n}:
ϑ(n, W ) = vn + ϑ(n − 1, W − wn)

wn W =⇒ n /
∈ ϑ.
ϑ(n, W ) = ϑ(n − 1, W )
∈ ϑ.
ϑ(n, W ) = vn + ϑ(n − 1, W − wn)
If n /
∈ ϑ, then ϑ(n, W ) simply contains an optimal solution to
the subproblem consisting of the intervals {1, 2, . . . , n − 1}:
ϑ(n, W ) = ϑ(n − 1, W )

wn W =⇒ n /
∈ ϑ.
ϑ(n, W ) = ϑ(n − 1, W )
∈ ϑ.
ϑ(n, W ) = vn + ϑ(n − 1, W − wn)
If n /
ϑ(n, W ) = ϑ(n − 1, W )
two.

wn W =⇒ n /
∈ ϑ.
ϑ(n, W ) = ϑ(n − 1, W )
∈ ϑ.
ϑ(n, W ) = vn + ϑ(n − 1, W − wn)
If n /
ϑ(n, W ) = ϑ(n − 1, W )
two.
ϑ(n, W ) = max(vn + ϑ(n − 1, W − wn), ϑ(n − 1, W ))

If OPT(j, w) is an optimal solution to the subproblem for items
{1, 2, . . . , j}, for any j ∈ {1, 2, . . . , n}, and with a maximum
allowed weight of w, then:
OPT(j, w) =





OPT(j − 1, w) if wj w,
max(vj + OPT(j − 1, w − wj),
OPT(j − 1, w)) otherwise.

If OPT(j, w) is an optimal solution to the subproblem for items
{1, 2, . . . , j}, for any j ∈ {1, 2, . . . , n}, and with a maximum
allowed weight of w, then:
OPT(j, w) =





OPT(j − 1, w) if wj w,
max(vj + OPT(j − 1, w − wj),
OPT(j − 1, w)) otherwise.
Extracting the items in an optimal solution
The item j is in an optimal solution OPT(j, w) if and only if the
vj + OPT(j − 1, w − wj) ≥ OPT(j − 1, w)

Knapsack(j, w)
1 if j = 0 or w = 0
2 then return 0
3 elseif wj w
4 then return Knapsack(j − 1, w))
5 else return max(vj + Knapsack(j − 1, w − wj),
Knapsack(j − 1, w))

Knapsack(j, w)
1 if j = 0 or w = 0
2 then return 0
3 elseif wj w
The initial call is Knapsack(n, W ).

Knapsack(j, w)
1 if j = 0 or w = 0
2 then return 0
3 elseif wj w
time.

M-Knapsack(j, w)
1 if j = 0 or w = 0
2 then return 0
3 elseif M[j, w] is empty
4 then M[j, w] ← max(vj + M-Knapsack(j − 1, w − wj),
M-Knapsack(j − 1, w))
5 return M[j, w]

M-Knapsack(j, w)
1 if j = 0 or w = 0
2 then return 0
5 return M[j, w]
Each entry in M[j, w] gets filled in only once at Θ(1) time,
and there are n + 1 × W + 1 entries, so M-Knapsack(n, W )
takes Θ(nW ) time.

M-Knapsack(j, w)
1 if j = 0 or w = 0
2 then return 0
5 return M[j, w]
takes Θ(nW ) time.
Is this a linear-time algorithm?

M-Knapsack(j, w)
1 if j = 0 or w = 0
2 then return 0
5 return M[j, w]
takes Θ(nW ) time.
Is this a linear-time algorithm?
This is an example of a pseudo-polynomial problem, since it
depends on another parameter W that is independent of the
problem size.

Knapsack(n, W )
1 for i ← 0 to n no remaining capacity
2 do M[i, 0] ← 0
3 for w ← 0 to W no item to choose from
4 do M[0, w] ← 0
5 for j ← 1 to n
6 do for w ← 1 to W
7 do if wj w //we cannot take object j
8 then M[j, w] = M[j − 1, w]
9 else M[j, w] ← max(vj + M[j − 1, w − wj],
M[j − 1, w])
10 return M[n, W ]

0/1 Knapsack recursive algorithm in action
Given the following (from M. H. Alsuwaiyel, ex. 7.6):
W = 9
wi = {2, 3, 4, 5}
vi = {3, 4, 5, 7}

0/1 Knapsack DP algorithm in action
Given the following (from M. H. Alsuwaiyel, ex. 7.6):
W = 9
wi = {2, 3, 4, 5}
vi = {3, 4, 5, 7}

Related problem: Subset Sums problem
Definition (Subset Sums problem)
weight wi , the goal is to find the maximum-weight subset that
does not exceed a given weight W .

P
i∈S wi , subject to
P
i∈S wi ≤ W .

P
P
i∈S wi ≤ W .
How is this similar to the 0/1 Knapsack problem?

P
P
i∈S wi ≤ W .
How is this similar to the 0/1 Knapsack problem?
Can you solve this using the same algorithm?

Contents
1 Introduction
Memoization
Dynamic programming
Conclusion

Definition
Given coin denominations in C = {ci }, make change for a given
amount A with the minimum number of coins.

Definition
Example
Coin denominations, C = {12, 5, 1} Amount to change, A = 15

Definition
Example
1 Choose 0 12 coins, so remaining is 15

Definition
Example
2 Choose 3 5 coins, so remaining is 15 − 3 ∗ 5 = 0

Definition
Example
Solution: 3 coins.

Definition
Example
Solution: 3 coins.
Questions
What is the natural search space? Does this problem have a
Dynamic Programming solution? If so, how do we develop it?

The best combination of coins for 15 paisa must be one of the
following:

following:
1 Best combination for 15 − 12 = 3 paisa, plus a 12 paisa coin.

following:

following:
Since we’re minimizing the number of coins, the best
combination would be the minimum of these three choices.

following:
By recursively solving for the best combination, this can be
generalized to |C| denominations to make change for any
amount A.

following:
By recursively solving for the best combination, this can be
generalized to |C| denominations to make change for any
amount A.
What are the subproblems?

If OPT(p) is the minimum number of coins needed to make change
for amount p with denominations C = {c1, c2, . . . , ck}, then:

The coin ci chosen at any step must be smaller than p, the
amount left at that point.

Once we choose ci ≤ p, OPT(p) = 1 + OPT(p − ci ), since we
have to find the best combination for the remaining amount
(picking a coin smaller than the amount at each step).

Since we don’t know which coin would be chosen, we have to
search all |C| denominations and find the minimum.

The number of coins for 0 amount is 0.

The number of coins for 0 amount is 0.
Recurrence
OPT(p) =
(
0 if p = 0
mini:ci ≤p{1 + OPT(p − ci )} if p 0

Change(n, C)
1 if n = 0
2 then return 0
3 else min ← ∞
4 for i ← 1 to |C|
5 do if ci ≤ n and 1 + Change(n − ci , C) min
6 then min ← 1 + Change(n − ci , C)

Change(n, C)
1 if n = 0
2 then return 0
3 else min ← ∞
4 for i ← 1 to |C|
The initial call is Change(A, C).

Change(n, C)
1 if n = 0
2 then return 0
3 else min ← ∞
4 for i ← 1 to |C|
time.

M-Change(n, C)
1 if n = 0
2 then return 0
3 else if M[n] is empty
4 then min ← ∞
5 for i ← 1 to |C|
6 do if ci ≤ n and
1 + M-Change(n − ci , C) min
7 then min ← 1 + M-Change(n − ci , C)
8 M[n] ← min
9 return M[n]

M-Change(n, C)
1 if n = 0
2 then return 0
4 then min ← ∞
5 for i ← 1 to |C|
8 M[n] ← min
9 return M[n]
Each entry in M[n] gets filled in only once at Θ(|C|) time,
and there are n + 1 entries, so M-Change(n) takes
Θ(n|C|) time.

M-Change(n, C)
1 if n = 0
2 then return 0
4 then min ← ∞
5 for i ← 1 to |C|
8 M[n] ← min
9 return M[n]
Each entry in M[n] gets filled in only once at Θ(|C|) time,
and there are n + 1 entries, so M-Change(n) takes
Θ(n|C|) time.
Another pseudo-polynomial problem!

Change(n, C)
M = [0 . . n], S = [0 . . n]
1 M[0] ← 0 no amount to change
2 for p ← 1 to n
3 do min ← ∞
4 for i ← 1 to |C|
5 do if ci ≤ p and 1 + M[p − ci ] min
6 then min ← 1 + M[p − ci ]
7 coin ← i
8 M[p] ← min
9 S[p] ← coin
10 return M and S

Change(n, C)
M = [0 . . n], S = [0 . . n]
1 M[0] ← 0 no amount to change
2 for p ← 1 to n
3 do min ← ∞
4 for i ← 1 to |C|
5 do if ci ≤ p and 1 + M[p − ci ] min
6 then min ← 1 + M[p − ci ]
7 coin ← i
8 M[p] ← min
9 S[p] ← coin
10 return M and S
M[p] for all 0 ≤ p ≤ n – minimum number of coins needed to
change for p paisa.
S[p] for all 0 ≤ p ≤ n – the first coin chosen in computing an
optimal solution for making change for p paise.

The S array in the algorithm “remembers” the first coin we
use when computing an optimal value for a given amount.
We go backwards using S[n] until n = 0 and find the coin that
was added at each step.

The S array in the algorithm “remembers” the first coin we
use when computing an optimal value for a given amount.
We go backwards using S[n] until n = 0 and find the coin that
was added at each step.
Coins(S, C, n)
1 while n 0
2 do Output S[n]
3 n ← n − CS[n]

Contents
1 Introduction
Memoization
Dynamic programming
Conclusion

Problem types solved by Dynamic Programming
The most important part of DP is to set up the subproblem
structure.

structure.
DP is not applicable to all optimization problems.

structure.
If a problem has the following properties, then it’s likely to
have a dynamic programming solution.

structure.
Polynomially many subproblems The total number of
subproblems should be a polynomial, or else DP
may not provide an efficient solution.

structure.
Polynomially many subproblems The total number of
subproblems should be a polynomial, or else DP
may not provide an efficient solution.
Subproblem optimality If the optimal solution to the entire
problem contain optimal solution to the
subproblems, then it has the subproblem
optimality property. Also called the principle of
optimality.

Dynamic Programming highlights
Dynamic Programming, just like Memoization, avoids
computing solutions to overlapping subproblems by saving
intermediate results, and thus both require space for the
“table”.

“table”.
Dynamic Programming is a bottom-up techniques, and finds
the solution by starting from the base case(s) and works its
way upwards.

“table”.
way upwards.
Developing a Dynamic Programming solution often requires
some thought into the subproblems, especially how to find the
natural order in which to solve the subproblems.

“table”.
way upwards.
Unlike Memoization, which solves only the needed
subproblems, DP solves all the subproblems, because it does it
bottom-up.

“table”.
way upwards.
Unlike Memoization, which solves only the needed
subproblems, DP solves all the subproblems, because it does it
bottom-up.
Dynamic Programming on the other hand may be much more
efficient because its iterative, whereas Memoization must pay
for the (often significant) overhead due to recursion.

Conclusion
Memoization is the top-down technique, and dynamic
programming is a bottom-up technique.
The key to Dynamic programming is in “intelligent” recursion
(the hard part), not in filling up the table (the easy part).
Dynamic Programming has the potential to transform
exponential-time brute-force solutions into polynomial-time
algorithms.
Greed does not pay, Dynamic Programming does!

Dynamic Programming: Memoization, Introduction to ALgorithms

Recommended

More Related Content

Similar to Dynamic Programming: Memoization, Introduction to ALgorithms (20)

Recently uploaded (20)

Dynamic Programming: Memoization, Introduction to ALgorithms