Maximizing Submodular Function over the Integer Lattice

Maximizing Submodular Function
over the Integer Lattice
Tasuku Soma
(Univ. Tokyo)
Joint work with:
Yuichi Yoshida (NII, Tokyo)
1 / 22

1 Monotone Submodular Function Maximization on ZS
+
2 Algorithms
3 DR-Submodular Cover
4 Summary
2 / 22

+
2 Algorithms
4 Summary
3 / 22

Monotone Submodular Func Maximization
E: ﬁnite set, f : 2E
→ R ... monotone submodular
maximize f (X) subject to X ∈ F
4 / 22

Monotone Submodular Func Maximization
E: ﬁnite set, f : 2E
→ R ... monotone submodular
maximize f (X) subject to X ∈ F
|X| ≤ k, i∈X w(i) ≤ 1, etc
• NP-hard in general
• O(1)-appriximable for various constraints (cardinality,
knapsack, matroid, etc) and eﬃcient algorithms
• Powerful model for machine learning
4 / 22

Limitation of Set Function
Some real scenarios cannot be captured by a set function.
5 / 22

Budget Allocation [Alon–Gamzu–Tennenholtz ’13,
S.–Kakimura–Inaba–Kawarabayashi ’14]:
We want to decide how much budget set aside for each
ad source.
5 / 22

ad source.
Generalized Sensor Placement:
We can put more than one sensors in each spot.
5 / 22

ad source.
Generalized Sensor Placement:
We can put more than one sensors in each spot.
Can we generalize these set function
models?
5 / 22

Deﬁnitions of Submodularity on {0, 1}E
E: ﬁnite set
f : {0, 1}E
→ R is submodular
f (X) + f (Y) ≥ f (X ∪ Y) + f (X ∩ Y) (∀X, Y ⊆ E)
6 / 22

Deﬁnitions of Submodularity on {0, 1}E
E: ﬁnite set
f : {0, 1}E
→ R is submodular
f (X) + f (Y) ≥ f (X ∪ Y) + f (X ∩ Y) (∀X, Y ⊆ E)
Diminishing Return
f (X ∪ e) − f (X) ≥ f (Y ∪ e) − f (Y)
(X ⊆ Y ⊆ E, e ∈ E Y)
6 / 22

Deﬁnitions of Submodularity on ZE
f : ZE
→ R is lattice submodular:
f (x) + f (y) ≥ f (x ∨ y) + f (x ∧ y) (∀x, y ∈ ZE
)
coord-wise max coord-wise min
7 / 22

f : ZE
f (x) + f (y) ≥ f (x ∨ y) + f (x ∧ y) (∀x, y ∈ ZE
)
f is diminishing return submodular (DR-submodular):
f (x + ei ) − f (x) ≥ f (y + ei ) − f (y)
(x ≤ y ∈ ZE
, i ∈ E)
7 / 22

f : ZE
f (x) + f (y) ≥ f (x ∨ y) + f (x ∧ y) (∀x, y ∈ ZE
)
⇑ ⇓
f is diminishing return submodular (DR-submodular):
f (x + ei ) − f (x) ≥ f (y + ei ) − f (y)
(x ≤ y ∈ ZE
, i ∈ E)
7 / 22

Monotone Submod Func Maximization on ZE
+
f : ZE
+ → R+ ... monotone lattice/DR-submodular func
(with f (0) = 0), r ∈ Z+
Maximize f (x)
subject to 0 ≤ x ≤ r1, x ∈ ZE
+ ∩ F
• F = {x : x(E) ≤ k} (cardinality)
• F = P(+)(ρ) (polymatroid)
• F = {x : w x ≤ 1} (knapsack)
8 / 22

Can We Reduce it to Set Function?
YES, if f is DR-submodular.
E

E
r copies

E
r copies
←→

2
4
0

9 / 22

E
r copies
←→

2
4
0

Drawback:
• The new ground set has r|E| size, pseudopoly
• Does not work for lattice-submodular function
9 / 22

Our Results
Theorem (S. and Yoshida ’15)
For any > 0, (1 − 1/e − )-appriximate polytime
algorithms for various constaints.
DR-submodular lattice submodular
cardinality (deterministic) (deterministic)
polymatroid (random) open
knapsack (random) only pseudopoly
10 / 22

+
2 Algorithms
4 Summary
11 / 22

Algorithms
Naive Approach:
Choosing the best coordinate and step size in every
iteration?
k∗
, i∗
∈ argmax
k,i
f (kei | x)
k
,
where f (kei | x) = f (kei + x) − f (x).
12 / 22

Algorithms
Naive Approach:
Choosing the best coordinate and step size in every
iteration?
k∗
, i∗
∈ argmax
k,i
f (kei | x)
k
,
where f (kei | x) = f (kei + x) − f (x).
Idea:
• Use “Decreasing Threshold Greedy”
[Badanidiyuru–Vondrák ’14] to determine step size
12 / 22

Cardinality/DR-Submodular
DecresingThresholdGreedy
1: x := 0, d := maxi∈E f (ei ), θ := d
2: while θ ≥ d
r
:
3: for each i ∈ E :
4: Find the largest k s.t.
f (kei | x)
k
≥ θ and x + kei
feasible.
5: x := x + kei
6: θ := (1 − )θ
7: return x
13 / 22

Cardinality/DR-Submodular
f is concave along each coordinate.
i
f (· | x)
step size k
slope
θ
Such k can be found in O(log r) time with binary search.
14 / 22

Cardinality/Lattice-Submodular
Idea: Devide the range into polynomially many regions.
i
15 / 22

i
fmax
(1 − )fmax
(1 − )2
fmax
15 / 22

i
fmax
(1 − )fmax
(1 − )2
fmax
k
k
Lemma
If there exists k with f (kei | x) ≥ kθ, we can ﬁnd k with
f (k ei | x) ≥ (1 − )k θ.
15 / 22

Polymatroid/DR-Submodular
Idea: Mimic Continuous Greedy Algorithm
Multilinear Extension of f : 2E
→ R
F(x) =
X⊆E
f (X)
i∈X
x(i)
i X
(1 − x(i)) (x ∈ [0, 1]E
)
Key Facts
• F is monotone if f is monotone
• F is concave along positive direction if f is submodular
16 / 22

Idea: Gluing the multilinear extensions on each
hypercube.
17 / 22

hypercube.
ﬁll by the multilinear ext of
˜f (X) = f (1 + 1X )
17 / 22

hypercube.
ﬁll by the multilinear ext of
˜f (X) = f (1 + 1X )
The resulting extension shares the same property?
→ YES, if f is DR-submodular.
17 / 22

+
2 Algorithms
4 Summary
18 / 22

Submodular Cover [Wolsey ’82]
Somewhat “dual” problem of maximization
f, c : 2S
→ R+ monotone submodular, α > 0
minimize c(X) subject to f (X) ≥ α
c : cost, f : quality, α : worst guarantee
19 / 22

f, c : 2S
Examples in ML:
• Eﬃcient Sensor Placement
[Krause&Guestrin ’05, Krause&Leskovec ’08]
• Text Summarization [Lin & Bilmes ’10]
• Object Finding [Song et al. ’14, Chen et al. ’14]
19 / 22

f, c : 2S
Algorithmic Results:
• For c(X) = |X|, O(log d/β)-approx [Wolsey ’82]
• For integral f, c, O(ρ log d)-approx [Wan et al. ’09]
d = maxs f (s), ρ: curvature of c,
β := min{f (s | X) : s ∈ S, X ⊆ S, f (s | X) > 0}
19 / 22

DR-Submodular Cover
f, c : ZS
→ R+ monotone DR-submodular, α > 0, r ∈ Z+
minimize c(x)
subject to f (x) ≥ α
0 ≤ x ≤ r1
20 / 22

DR-Submodular Cover
f, c : ZS
→ R+ monotone DR-submodular, α > 0, r ∈ Z+
minimize c(x)
subject to f (x) ≥ α
0 ≤ x ≤ r1
Theorem (S.–Yoshida, to appear in NIPS ’15)
An algorithm for ﬁnding a (nearly) feasible solution of
O(ρ log d/β) approx in O (n log nr log r) time.
20 / 22

+
2 Algorithms
4 Summary
21 / 22

Summary
Our Results
• Useful genealizations of monotone submodular func
maximization and submodular cover
• Various polytime approximation algorithms
Recent Work
• Online monotone submodular func maximization on
ZE
+ [Avigdor-Elgrabli et al. ’15]
• Nonmonotone submodular func maximization on ZE
+
[Gottschalk–Peis ’15]
22 / 22

Maximizing Submodular Function over the Integer Lattice

More Related Content

What's hot (20)

Viewers also liked (7)

Similar to Maximizing Submodular Function over the Integer Lattice (20)

Recently uploaded (20)

Maximizing Submodular Function over the Integer Lattice