[DL輪読会]陰関数微分を用いた深層学習

1
 
Deep Learning with Implicit Gradients
Shohei Taniguchi, Matsuo Lab (M1)

•
•
• 2
- Meta-Learning with Implicit Gradients
‣ MAML inner update  
iMAML
- RNNs Evolving on an Equilibrium Manifold: A Panacea for Vanishing  
and Exploding Gradients?
‣ ERNN
2

Outline
1.
-
-
2.
- 1
‣ Implicit Reparameterization Gradients
3. Meta-Learning with Implicit Gradients
4. RNNs Evolving on an Equilibrium Manifold: A Panacea for Vanishing and
Exploding Gradients?
3

• (e.g. 2 )
-
-
- NN
• (e.g. )
-
-
-  
( )
y = f (x) y = ax2
+ bx + c
f (x, y) = 0 x2
+ y2
= r2
x y 5

•
•
, ,
 
f(x, y) = 0
dy
dx
= −
∂f/∂x
∂f/∂y
= −
fx
fy
f(x, y) = 0 (x0, y0) fy (x0, y0)
x0 ∈ U y0 ∈ V g : U → V
{(x, g(x))|x ∈ U} = {(x, y) ∈ U × V| f(x, y) = 0}
6

• 1
-
- A
- B  
( )
• 2 Jacobian
f(x, y) = 0 (x0, y0)
fy (x0, y0)
x2
+ y2
− r2
= 0
y = r2
− x2
fy (r,0) = 2 × 0 = 0
y = ± r2
− x2
fy 7

( )
1.
- ( )
- iMAML
2.
-
- ERNN
8

( )
1.
- ( )
- iMAML
2.
-
- ERNN
9

Implicit Reparameterization Gradients
10

• NeurIPS 2018 accepted
•
- Michael Figurnov, Shakir Mohamed, Andriy Mnih
- DeepMind
• reparameterization
trick
•
iMAML ERNN
11

Reparameterization Trick
• VAE
•
reparameterization trick
•
𝔼q(z; ϕ) [log p (x|z)]−KL (q (z; ϕ)||p (z))
q ϵ = f (z; ϕ) =
z − μϕ
σϕ
ϵ ∼ 𝒩 (0,1)
ϕ ϵ
∇ϕ 𝔼q(z; ϕ) [log p (x|z)] = 𝔼p(ϵ)
[
∇ϕlog p (x|z)
z=f−1
(ϵ; ϕ)]
f f
12

• 1 →
-
-
-
f
ϵ ∼ U (0,1) ϕ
z = f−1
(ϵ; ϕ)
∇ϕ 𝔼q(z; ϕ) [log p (x|z)] = 𝔼p(ϵ) [∇ϕlog p (x|z)]
= 𝔼p(ϵ) [∇zlog p (x|z)∇ϕz]
∇ϕz
13

•
-
-
•
ϵ = f (z; ϕ) ⇔ f (z; ϕ) − ϵ = 0 z ϕ
∇ϕz = −
∇ϕ f (z; ϕ)
∇z f (z; ϕ)
= −
∇ϕ f (z; ϕ)
q (z; ϕ)
z q (z; ϕ) f−1
14

Meta-Learning with Implicit Gradients
15

• NeurIPS 2019 accepted
•
- Aravind Rajeswaran, Chelsea Finn, Sham Kakade, Sergey Levine
- MAML
• MAML
16

Model-Agnostic Meta-Learning (MAML)
•
-
- 1 (one-step adaptation)
θ*ML
:= argmin
θ∈Θ
F(θ), where F(θ) =
1
M
M
∑
i=1
ℒ ( 𝒜lgi (θ), 𝒟test
i )
𝒜lgi (θ) = θ − α∇θℒ (θ, 𝒟tr
i )
17

MAML
•
-
• MAML
• 1 FOMAML
- FOMAML  
https://www.slideshare.net/DeepLearningJP2016/dl1maml
• iMAML
∇θF (θ) 𝒜lgi (θ)
18

Inner Loop
•
•
𝒜lg⋆
(θ) = argmin
ϕ′∈Φ
Gi (ϕ′, θ)
Gi (ϕ′, θ) = ̂ℒ (ϕ′)+
λ
2
ϕ′− θ
2
19

Outer Loop
• MAML outer loop
• inner loop
➡
θ ← θ − ηdθF(θ)
= θ − η
1
M
M
∑
i=1
d𝒜lgi(θ)
dθ
∇ϕℒi ( 𝒜lgi(θ))
(ϕ = 𝒜lgi(θ))
d𝒜lgi(θ)
dθ
20

Outer Loop
• inner loop
•
•
- adapt
ϕi ≡ 𝒜lg⋆
i (θ) = argmin
ϕ′∈Φ
Gi (ϕ′, θ)
∇ϕ′Gi (ϕ′, θ)
ϕ′=ϕi
= 0
∇ ̂ℒ(ϕi) + λ(𝒜lg⋆
i (θ) − θ) = 0
θ 𝒜lg⋆
(θ)
d𝒜lg⋆
(θ)
dθ
=
(
I +
1
λ
∇2 ̂ℒ (ϕi))
−1
ϕi 21

Outer Loop
• 2
① inner loop adapt
(SGD )
② 3
•
(
I +
1
λ
∇2 ̂ℒ (ϕi))
−1
ϕi
(
I +
1
λ
∇2 ̂ℒ (ϕi))
−1
∇ϕℒi ( 𝒜lgi(θ))
22

(CG )
•
•
Ax = b ⋯(1)
(1) f(x) =
1
2
xT
Ax − bT
x
x0 = 0,r0 = b − Ax0, p0 = r0
αk =
rT
k pk
pT
k Apk
xk+1 = xk + αk pk
rk+1 = rk − αkApk
pk+1 = rk+1 +
rT
k+1rk+1
rT
k rk
pk
23

(CG )
•
•
( 5 )
- (p22 ① )
‣ Appendix E
gi =
(
I +
1
λ
∇2 ̂ℒ (ϕi))
−1
∇ϕℒi ( 𝒜lgi(θ)) gi
(
I +
1
λ
∇2 ̂ℒ (ϕi))
gi = ∇ϕℒi ( 𝒜lgi(θ))
rk
𝒜lgi(θ)
24

iMAML
• inner loop
➡adapt
• outer loop inner loop
➡inner loop
‣ MAML 1
‣ iMAML Hessian-Free 2
adapt
25

•
- iMAML inner loop ( )
- FOMAML (CG )
- MAML  
(FOMAML ??)
O(1)
26

• Omniglot
- inner loop Hessian-Free iMAML
- iMAML way ( )
- FOMAML
27

• Mini-ImageNet
- Reptile (FOMAML )
-
??
28

iMAML
•
iMAML
• MAML
•
•
•
-
29

( )
1.
- ( )
- iMAML
2.
-
- ERNN
30

RNNs Evolving on an Equilibrium Manifold:
A Panacea for Vanishing and Exploding Gradients?
31

•
- Anil Kag, Ziming Zhang, Venkatesh Saligrama
- , MERL
• NeurIPS 2019 reject
•
RNN
•
•
32

RNN /
• RNN
- sigmoid tanh
• RNN /
- LSTM GRU
hk = ϕ (Uhk−1 + Wxk + b)
ϕ
∂hm
∂hn
=
∏
m≥k>n
∂hk
∂hk−1
=
∏
m≥k>n
∇ϕ (Uhk−1 + Wxk + b) U
33

RNN ODE
• RNN skip connection
(ODE)
• Neural ODE
-  
https://www.slideshare.net/DeepLearningJP2016/dlneural-ordinary-
differential-equations
dh(t)
dt
≜ h′(t) = ϕ (Uh(t) + Wxk + b)
⟹ hk = hk−1 + ηϕ (Uhk−1 + Wxk + b)
34

ODE
• ODE
• 1
➡ ( )
• ERNN
dh
dt
= f (h, x) f (h, x) = 0 ⋯(1)
(1) h x (h0, x0)
fh (h0, x0) (1)
h = g (x)
(h0, x0)
35

ERNN
• ERNN  
 
ODE
•  
 
➡  
h′(t) = ϕ (U (h(t) + hk−1) + Wxk + b) − γ (h(t) + hk−1)
h′(t) = 0 hk
hk f (hk−1, h) = ϕ (U (h + hk−1) + Wxk + b) − γ (h + hk−1) = 0
∂h
∂hk−1
= −
∂f/∂hk−1
∂f/∂h
= − I
∂f/∂h
36

∂f/∂h
•
1. (sigmoid tanh OK)
2.
-  
( )
∂f
∂h
= ∇ϕ (U (h + hk−1) + Wxk + b) U
ϕ
U
37

•
• 5
•
-
h(0)
k
= 0
h(i+1)
k
= h(i)
k
+ η(i)
k [
ϕ
(
U (h(i)
k
+ hk−1) + Wxk + b
)
− γ (h(i)
k
+ hk−1)]
η(i)
k
38

HAR-2 RNN ERNN (log scale)
• RNN
• ERNN 1
∂hT
∂h1
39

ERNN
• NN
1
RNN
•
• SoTA
• RNN
• accept
42

[DL輪読会]陰関数微分を用いた深層学習

More Related Content

What's hot (20)

Similar to [DL輪読会]陰関数微分を用いた深層学習 (20)

More from Deep Learning JP (20)

Recently uploaded (20)

[DL輪読会]陰関数微分を用いた深層学習