![Preliminary
• O: Zero Matrix/Vector（oi,j=0）
• U: Ones Matrix/Vector (ui,j=1)
• E: Unit Matrix（ei,j=1 ; i=j, otherwise ei,j=0）
• Matrix Product: D = B・C
• Matrix/Vector Decomposition: B = [B1, B2] = [B1, O] + [O, B2]
• Hadamard Product◎：B◎C = E・B・(E・C)
• Graph Representation
• Adjacency Matrix A: ai,j=1 if node-i and node-j is connected
• Baseline Graph G = f(A◎W*X): Mask W by A(=Edge-Pruning Flags)
• Keep W for Next Training](https://image.slidesharecdn.com/hpagnn-190719095556/85/How-Powerful-are-Graph-Networks-6-320.jpg)
How Powerful are Graph Networks?
- 1. How Powerful are Graph Neural Networks? ~Low-Pass Filterを添えて~ NaN 2019/07/18
- 2. Presentation of Amateur, by Amateur, for Amateur Outline • Introduction to Graph Neural Networks • GUNDAM: General Universal Network for Dynamic Active Memory • My Perspective for Graph Neural Networks • What is operation on Graph Neural Networks After All?
- 3. Conclusion Use Case: • NODE & GRAPH Classifications • Drug Discovery, Web Analytics,…, All About Graph Problems (DNN also?) • Could not understand how operate such the classification on GNN “Less Powerful But Interesting GNNs” @Section-5 Title !? …“How Powerful are Graph Neural Networks?”… • “Revisiting Graph Neural Networks: All We Have is Low-Pass Filters” • Claim：Features are in Low-Frequency→GNN outputs such that →Low-Pass Filter!!! • Adjacency Matrix A = I – L (L: Laplacian) • Caused by the ”L”？
- 4. Graph Neural Networks NeighborhoodsExample Graph Node’s Feature Edge’s Feature - Undirected/Directed - Weighted/No-Weighted 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 Adjacent Matrix: O(N2) = Complete Network Representation
- 5. Graph Neural Networks Step-1 (k=1) 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 Adjacent Matrix: O(N2) a a aa b b b bb c c cde Step-2 (k=2) b b bb c c d de cd c Step-3 (k=3) c c cc d d dd e e e Step-4 (k=4) c c cc e e e e c d Weights e
- 6. Preliminary • O: Zero Matrix/Vector（oi,j=0） • U: Ones Matrix/Vector (ui,j=1) • E: Unit Matrix（ei,j=1 ; i=j, otherwise ei,j=0） • Matrix Product: D = B・C • Matrix/Vector Decomposition: B = [B1, B2] = [B1, O] + [O, B2] • Hadamard Product◎：B◎C = E・B・(E・C) • Graph Representation • Adjacency Matrix A: ai,j=1 if node-i and node-j is connected • Baseline Graph G = f(A◎W*X): Mask W by A(=Edge-Pruning Flags) • Keep W for Next Training
- 7. Cheat Sheet f( ) f( ) f( ) = ・ OO OO OO W(1) W(2) W(3) Feedforward Network f(・): Activation Function f( ) f( ) f( ) = ・ OO O O OO E Concat E O f(X)=X f( ) OO Sum U = ・ f(X)=X f( ) f( ) f( ) = ・ OO O O OO E Residual E O f(X)=X f( ) OO Mean-Pool U = ・ f(X)=X/|U| f( ) Max-Pool = ・OOE f(X)=argmax(X) Readout Injection
- 8. Graph Neural Networks =f ・Aav huW◎ COMBINE AGGREGATE =f avhvhv
- 9. Collorary8&9 (Fig.3) 0 1 1 1 0 0 1 0 0 0 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 Mean = (■+■)/(1+1) Max-Pool = (■, ■)=2 Mean = (■+■+■+■)/(1+1+1+1) Max-Pool = (■, ■)=2 Adjacency Matrix Adjacency Matrix isomorphic ?
- 10. Conclusion Use Case: • NODE & GRAPH Classifications • Drug Discovery, Web Analytics,…, All About Graph Problems (DNN also?) • Could not understand how operate such the classification on GNN “Less Powerful But Interesting GNNs” @Section-5 Title !? …“How Powerful are Graph Neural Networks?”… • “Revisiting Graph Neural Networks: All We Have is Low-Pass Filters” • Claim：Features are in Low-Frequency→GNN outputs such that →Low-Pass Filter!!! • Adjacency Matrix A = I – L (L: Laplacian) • Caused by the ”L”？