SlideShare a Scribd company logo

Using Local Spectral Methods to Robustify Graph-Based Learning

D
David Gleich

The document discusses using local spectral methods to enhance the robustness of graph-based learning, highlighting the importance of data graph construction in learning outcomes. It identifies various types of noise in graphs and proposes techniques to improve the performance of semi-supervised learning methods on graphs. Key contributions include a scalable, localized algorithm for diffusion processes and a new ranking-based approach for label assignment that improves robustness in predictions.

Technology
1 of 30
Downloaded 30 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Using Local Spectral Methods to Robustify Graph-Based Learning David F. Gleich! Purdue University! Joint work with Michael Mahoney @ Berkeley supported by " NSF CAREER CCF-1149756 Code www.cs.purdue.edu/homes/dgleich/codes/robust-diffusions! KDD2015
The graph-based data analysis pipeline 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 " Raw data! •  Relationships •  Images •  Text records •  Etc. " Convert to a graph! •  Nearest neighs •  Kernels •  2-mode to 1-mode •  Etc. " Algorithm/Learning! •  Important nodes •  Infer features •  Clustering •  Etc. KDD2015 David Gleich · Purdue 2
“Noise” in the initial data modeling decisions " Explicit graphs! are those that are given to a data analyst. “A social network” •  Known spam accounts included? •  Users not logged in for a year? •  Etc. A type of noise " Constructed graphs! are built based on some other primary data." “nearest neighbor graphs” •  K-NN or ε-NN •  Thresholding correlations to zero Often made for computational convenience! (Graph too big.) A different type a noise! " Labeled graphs! occur in information diffusion/propagation “function prediction” •  Labeled nodes •  Labeled edges •  Some are wrong A direct type of noise! Do these decisions matter? Our experience Yes! Dramatically so! KDD2015 David Gleich · Purdue 3
The graph-based data analysis pipeline 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 " Raw data! •  Relationships •  Images •  Text records •  Etc. " Convert to a graph! •  Nearest neighs •  Kernels •  2-mode to 1-mode •  Etc. " Algorithm/Learning! •  Important nodes •  Infer features •  Clustering •  Etc. Most algorithmic and statistical research happens here The potential downstream signal is determined by this step KDD2015 David Gleich · Purdue 4
Our goal: towards an integrative analysis 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 ! •  How does the graph creation process affect the outcomes of graph-based learning? •  Is there anything we can do to make this process more robust? KDD2015 David Gleich · Purdue 5
Graph-based learning is usually only one component of a big pipeline KDD2015 David Gleich ¡ Purdue 6 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 Many databases over" genes with survival rates" for various cancers List of possible genes" responsible for survival Cluster analysis Reinteration of data THIS STEP SHOULD! BE ROBUST TO ! VARIATIONS ABOVE!
Scalable graph analytics Local methods are one of the most successful classes of scalable graph analytics They don’t even look at the entire graph. •  Andersen Chung Lang (ACL) Push method Conjecture" Local methods regularize some variant of the original algorithm or problem. Justification" For ACL and a few relatives this is exact! Impact?! Improved robustness to noise? KDD2015 David Gleich · Purdue 7 c For instance, to " answer “what function” is shared by the started node, we’d only look at the circled region.
Our contributions We study these issues in the case of " semi-supervised learning (SSL) on graphs 1.  We illustrate a common mincut framework for a variety of SSL methods 2.  Show how to “localize” one (and make it scalable!) 3.  Provide a more robust SSL labeling method 4.  Identify a weakness in SSL methods: they cannot use extra edges! We find one useful way to do so. KDD2015 David Gleich · Purdue 8
Semi-supervised " graph-based learning KDD2015 David Gleich · Purdue 9 Given a graph, and a few labeled nodes, predict the labels on the rest of the graph. Algorithm 1.  Run a diffusion for each label (possibly with neg. info from other classes) 2.  Assign new labels based on the value of each diffusion
Semi-supervised " graph-based learning KDD2015 David Gleich · Purdue 10 Given a graph, and a few labeled nodes, predict the labels on the rest of the graph. Algorithm 1.  Run a diffusion for each label (possibly with neg. info from other classes) 2.  Assign new labels based on the value of each diffusion
Semi-supervised " graph-based learning KDD2015 David Gleich · Purdue 11 Given a graph, and a few labeled nodes, predict the labels on the rest of the graph. Algorithm 1.  Run a diffusion for each label (possibly with neg. info from other classes) 2.  Assign new labels based on the value of each diffusion
The diffusions proposed for semi- supervised learning are s,t-cut minorants KDD2015 David Gleich · Purdue 12 1 3 2 6 4 5 7 8 9 10 t s In the unweighted case, " solve via max-flow. In the weighted case, solve via network simplex or industrial LP. minimize qP ij2E Ci,j |xi xj |2 subject to xs = 1, xt = 0. minimize P ij2E Ci,j |xi xj | subject to xs = 1, xt = 0. MINCUT LP Spectral minorant – lin. sys.
Representative cut problems KDD2015 David Gleich · Purdue 13 ∞ ∞ ∞ ∞ s t ZGL α α 4α 3α 4α 6α 3α 3α 5α 5α 5α 2α 5α 4α 5α s t Zhou et al. Positive label Neg. label Unlabeled Andersen-Lang weighting " variation too Joachims has a variation too. Zhou et al. NIPS 2003; Zhu et al., ICML 2003; Andersen Lang, SODA 2008; Joachims, ICML 2003 These help our intuition about the solutions All spectral minorants are linear systems.
Implicit regularization views on the Zhou et al. diffusion KDD2015 David Gleich · Purdue 14 α α 4α 3α 4α 6α 3α 3α 5α 5α 5α 2α 5α 4α 5α s t Zhou et al. RESULT! The spectral minorant of Zhou is equivalent to the weakly-local MOV solution. PROOF! The two linear systems are the same (after working out a few equivalences). IMPORTANCE! We’d expect Zhou to be “more robust” minimize qP ij2E Ci,j |xi xj |2 subject to xs = 1, xt = 0. The Mahoney-Orecchia-Vishnoi (MOV) vector is a localized variation on the Fiedler vector to find a small conductance set nearby a seed set.
A scalable, localized algorithm for Zhou et al’s diffusion. KDD2015 David Gleich · Purdue 15 RESULT! We can use a variation on coordinate descent methods related to the Andersen-Chung-Lang PUSH procedure to solve Zhou’s diffusion in a scalable manner. PROOF. See Gleich-Mahoney ICML ‘14 IMPORTANCE (1)! We should be able to make Zhou et al. scale. IMPORTANCE (2)! Using this algorithm adds another implicit regularization term that should further improve robustness! minimize qP ij2E Ci,j |xi xj |2 subject to xs = 1, xt = 0. minimize P ij2E Ci,j |xi xj |2 + ⌧ P i2V di xi subject to xs = 1, xt = 0, xi 0.
Semi-supervised " graph-based learning KDD2015 David Gleich · Purdue 16 Given a graph, and a few labeled nodes, predict the labels on the rest of the graph. Algorithm 1.  Run a diffusion for each label (possibly with neg. info from other classes) 2.  Assign new labels based on the value of each diffusion
Traditional rounding methods for SSL are value-based KDD2015 David Gleich · Purdue 17 Class 1 Class 2Class 3 Class 1 Class 2 Class 3 CLASS 1 CLASS 2 CLASS 3 VALUE-BASED Use the largest value of the diffusion to pick the label. Zhou’s diffusion
But value based rounding doesn’t work for all diffusions KDD2015 David Gleich · Purdue 18 Class 1 Class 2Class 3 Class 1 Class 2 Class 3 Class 1 Class 2Class 3 Class 1 Class 2Class 3 Class 1 Class 2Class 3 (b) Zhou et al., l = 3 (c) Andersen-Lang, l = 3 (d) Joachims, l = 3 (e) ZGL, l = 3 Class 1 Class 2Class 3 Class 1 Class 2Class 3 Class 1 Class 2Class 3 Class 1 Class 2Class 3 (f) Zhou et al., l = 15 (g) Andersen-Lang, l = 15 (h) Joachims, l = 15 (i) ZGL, l = 15 CLASS 1 CLASS 3 CLASS 2 VALUE-BASED rounding fails for most of these diffusions BUT There is still a signal there! Adding more labels doesn’t help either, see the paper for those details
Rank-based rounding is far more robust. KDD2015 David Gleich · Purdue 19 Class 1 Class 2Class 3 NEW IDEA! Look at the RANK of the item in each diffusion instead of it’s VALUE. JUSTIFICATION! Based on the idea of sweep-cut rounding in spectral methods (use the order induced by the eigenvector, not its values) IMPACT! Much more robust rounding to labels
Rank-based rounding has a big impact on a real-study. KDD2015 David Gleich · Purdue 20 2 4 6 8 10 0 0.2 0.4 0.6 0.8 errorrate average training samples per class Zhou Zhou+Push 2 4 6 8 10 0 0.2 0.4 0.6 0.8 1 errorrate average training samples per class Zhou Zhou+Push We used the digit prediction task out of Zhou’s paper and added just a bit of noise" as label errors and switched parameters. VALUE-BASED RANK-BASED
Main empirical results 1.  Zhou’s diffusion seems to work best for sparse graphs whereas the ZGL diffusion works best for dense 2.  On the digit’s dataset, dense graph constructions yield higher error rates 3.  Densifying the a super-sparse graph construction on the digits dataset yields lower error. 4.  And a similar fact holds on an Amazon co- purchasing network. KDD2015 David Gleich · Purdue 21
5 10 0 50 100 150 number of labels numberofmistakes An illustrative synthetic problem shows the differences. Two-class block-model, 150 nodes each" between prob = 0.02, " withinprob = 0.35 (dense) or 0.06 (sparse) Reveal labels for k nodes (varied) and we have different error rates (sparse 0/10% low/high) and dense (20%/60% for low-high) KDD2015 David Gleich ¡ Purdue 22 5 10 0 50 100 150 number of labels numberofmistakes Joachims Zhou ZGL Real-world scenario" sparse graph, high error Sparse graph," low error 20 40 60 0 20 40 60 80 100 number of labels numberofmistakes 20 40 60 0 20 40 60 80 100 number of labels numberofmistakes Dense graph," low error rate Dense graph," high error rate
Varying density in an SSL construction. KDD2015 David Gleich · Purdue 23 Ai,j = exp ✓ kdi dj k2 2 2 2 ◆ di dj = 2.5 = 1.25 We use the digits experiment from Zhou et al. 2003. 10 digits and a few label errors. We vary density either by the num. of nearest neighbors or by the kernel width.
As density increases, the results just get worse KDD2015 David Gleich · Purdue 24 1 1.5 2 2.5 0 0.1 0.2 0.3 0.4 errorrate σ 0.8 1.2 1.5 1.8 2.1 2.5 Zhou Zhou+Push 10 2 0 0.1 0.2 0.3 0.4 errorrate nearest neighbors 5 10 25 50 100 150 200 250 Zhou Zhou+Push Varying kernel width Varying nearest neighors •  Adding “more” edges seems to only hurt. (Unless there is no signal). •  Zhou+Push seems to be slightly more robust (Maybe).
Some observations and a question. Adding “more data” yields “worse results” for this procedure (in a simple setting). Suppose I have a real-world system that can work with up to E edges on some graph. Is there a way I can create new edges? KDD2015 David Gleich · Purdue 25
Densifying the graph with path expansions KDD2015 David Gleich ¡ Purdue 26 Ak = kX `=1 A` If A is the adjacency matrix, then this counts the total weight on all paths up to length k. We now repeat the nearest neighbor computation, but with paired parameters such that we have the same average degree. Zhou Zhou w. Push Avg. Deg k = 1 k 1 k = 1 k 1 19 0.163 0.114 0.156 0.117 41 0.156 0.132 0.158 0.113 53 0.183 0.142 0.179 0.136 104 0.193 0.145 0.178 0.144 138 0.216 0.102 0.204 0.101 k=4, nn = 3
The same result holds for Amazon’s co-purchasing network KDD2015 David Gleich · Purdue 27 mean F1 Confidence intervals k Zhou Zhou w. Push Zhou Zhou w. Push 1 0.173 0.229 [0.15 0.19] [0.21 0.25] 2 0.197 0.231 [0.18 0.22] [0.21 0.25] 3 0.221 0.238 [0.17 0.27] [0.19 0.28] Amazon’s co-purchasing network (on Snap) is effectively a highly sparse nearest-neighbor network from their (denser) co-purchasing graph. We attempt to predict the items in a product category based on a small sample and study the F1 score for the predictions. Some small details missing – see the full paper. Ak = kX `=1 A`
(a) K2 sparse (b) K2 dense (c) RK2 Figure 2: We artificially densify this graph to Ak based on a cons and dense di↵usions and regularization. The color indicates the circled nodes. The unavoidable errors are caused by a mislabeled regularizing di↵usions on dense graphs produces only a small Towards some theory, i.e. why are densified sparse graphs better? KDD2015 David Gleich · Purdue 28 How do sparsity, density, and regularization of a diffusion play into the results in a controlled setting? THE ERROR Labels
(a) K2 sparse (b) K2 dense (c) RK2 Figure 2: We artificially densify this graph to Ak based on a cons and dense di↵usions and regularization. The color indicates the circled nodes. The unavoidable errors are caused by a mislabeled regularizing di↵usions on dense graphs produces only a small Towards some theory, i.e. why are densified sparse graphs better? KDD2015 David Gleich · Purdue 29 THE ERROR Labels Using Push algorithm P5 `=1 A` P5 `=1 A` Regularization Dense Dense
Recap, discussion, future work Contributions 1.  Flow-setup for SSL diffusions 2.  New robust rounding rule for class selection 3.  Localized Zhou’s diffusion 4.  Empirical insights on density of graph constructions Observations •  Many of these insights translate to directed, weighted graphs with fuzzy labels and/or some parallel architectures. •  Weakness mainly empirical results on the density. •  We need a theoretical basis for the densification theory! KDD2015 David Gleich · Purdue 30 Supported by NSF, ARO, DARPA CODE www.cs.purdue.edu/homes/dgleich/codes/robust-diffusions

More Related Content

PDF
Personalized PageRank based community detection
David Gleich
 
PDF
PageRank Centrality of dynamic graph structures
David Gleich
 
PDF
Localized methods for diffusions in large graphs
David Gleich
 
PDF
Localized methods in graph mining
David Gleich
 
PDF
Spacey random walks and higher order Markov chains
David Gleich
 
PDF
Engineering Data Science Objectives for Social Network Analysis
David Gleich
 
PDF
Anti-differentiating Approximation Algorithms: PageRank and MinCut
David Gleich
 
PDF
Non-exhaustive, Overlapping K-means
David Gleich
 
Personalized PageRank based community detection
David Gleich
 
PageRank Centrality of dynamic graph structures
David Gleich
 
Localized methods for diffusions in large graphs
David Gleich
 
Localized methods in graph mining
David Gleich
 
Spacey random walks and higher order Markov chains
David Gleich
 
Engineering Data Science Objectives for Social Network Analysis
David Gleich
 
Anti-differentiating Approximation Algorithms: PageRank and MinCut
David Gleich
 
Non-exhaustive, Overlapping K-means
David Gleich
 

What's hot (20)

PDF
Anti-differentiating approximation algorithms: A case study with min-cuts, sp...
David Gleich
 
PDF
Big data matrix factorizations and Overlapping community detection in graphs
David Gleich
 
PDF
Higher-order organization of complex networks
David Gleich
 
PDF
Spacey random walks and higher-order data analysis
David Gleich
 
PDF
Spectral clustering with motifs and higher-order structures
David Gleich
 
PDF
Iterative methods with special structures
David Gleich
 
PDF
A dynamical system for PageRank with time-dependent teleportation
David Gleich
 
PDF
Correlation clustering and community detection in graphs and networks
David Gleich
 
PDF
What is Critical in GAN Training?
Hao-Wen (Herman) Dong
 
PDF
Lecture 5 backpropagation
ParveenMalik18
 
PDF
PR 103: t-SNE
Taeoh Kim
 
PDF
Numerical Approximation of Filtration Processes through Porous Media
Raheel Ahmed
 
PDF
Gaps between the theory and practice of large-scale matrix-based network comp...
David Gleich
 
PDF
Backpropagation: Understanding How to Update ANNs Weights Step-by-Step
Ahmed Gad
 
PPTX
Contactless Calipper - English
Giga Khizanishvili
 
PDF
PR 113: The Perception Distortion Tradeoff
Taeoh Kim
 
PDF
B0560508
IOSR Journals
 
PDF
Lesson 26: Integration by Substitution (handout)
Matthew Leingang
 
PDF
Cs36565569
IJERA Editor
 
PDF
Multimodal Residual Learning for Visual Question-Answering
NAVER D2
 
Anti-differentiating approximation algorithms: A case study with min-cuts, sp...
David Gleich
 
Big data matrix factorizations and Overlapping community detection in graphs
David Gleich
 
Higher-order organization of complex networks
David Gleich
 
Spacey random walks and higher-order data analysis
David Gleich
 
Spectral clustering with motifs and higher-order structures
David Gleich
 
Iterative methods with special structures
David Gleich
 
A dynamical system for PageRank with time-dependent teleportation
David Gleich
 
Correlation clustering and community detection in graphs and networks
David Gleich
 
What is Critical in GAN Training?
Hao-Wen (Herman) Dong
 
Lecture 5 backpropagation
ParveenMalik18
 
PR 103: t-SNE
Taeoh Kim
 
Numerical Approximation of Filtration Processes through Porous Media
Raheel Ahmed
 
Gaps between the theory and practice of large-scale matrix-based network comp...
David Gleich
 
Backpropagation: Understanding How to Update ANNs Weights Step-by-Step
Ahmed Gad
 
Contactless Calipper - English
Giga Khizanishvili
 
PR 113: The Perception Distortion Tradeoff
Taeoh Kim
 
B0560508
IOSR Journals
 
Lesson 26: Integration by Substitution (handout)
Matthew Leingang
 
Cs36565569
IJERA Editor
 
Multimodal Residual Learning for Visual Question-Answering
NAVER D2
 
Ad

Viewers also liked (18)

PDF
How does Google Google: A journey into the wondrous mathematics behind your f...
David Gleich
 
PDF
Sparse matrix computations in MapReduce
David Gleich
 
PDF
Graph libraries in Matlab: MatlabBGL and gaimc
David Gleich
 
PDF
Overlapping clusters for distributed computation
David Gleich
 
PDF
Fast matrix primitives for ranking, link-prediction and more
David Gleich
 
PPTX
Graph based Semi Supervised Learning V1
Neeta Pande
 
PDF
Direct tall-and-skinny QR factorizations in MapReduce architectures
David Gleich
 
PDF
Iterative methods for network alignment
David Gleich
 
PDF
What you can do with a tall-and-skinny QR factorization in Hadoop: Principal ...
David Gleich
 
PDF
A history of PageRank from the numerical computing perspective
David Gleich
 
PDF
A multithreaded method for network alignment
David Gleich
 
PDF
The power and Arnoldi methods in an algebra of circulants
David Gleich
 
PDF
Tall and Skinny QRs in MapReduce
David Gleich
 
PDF
MapReduce Tall-and-skinny QR and applications
David Gleich
 
PDF
Relaxation methods for the matrix exponential on large networks
David Gleich
 
PDF
Tall-and-skinny QR factorizations in MapReduce architectures
David Gleich
 
PDF
Fast relaxation methods for the matrix exponential
David Gleich
 
PDF
Vertex neighborhoods, low conductance cuts, and good seeds for local communit...
David Gleich
 
How does Google Google: A journey into the wondrous mathematics behind your f...
David Gleich
 
Sparse matrix computations in MapReduce
David Gleich
 
Graph libraries in Matlab: MatlabBGL and gaimc
David Gleich
 
Overlapping clusters for distributed computation
David Gleich
 
Fast matrix primitives for ranking, link-prediction and more
David Gleich
 
Graph based Semi Supervised Learning V1
Neeta Pande
 
Direct tall-and-skinny QR factorizations in MapReduce architectures
David Gleich
 
Iterative methods for network alignment
David Gleich
 
What you can do with a tall-and-skinny QR factorization in Hadoop: Principal ...
David Gleich
 
A history of PageRank from the numerical computing perspective
David Gleich
 
A multithreaded method for network alignment
David Gleich
 
The power and Arnoldi methods in an algebra of circulants
David Gleich
 
Tall and Skinny QRs in MapReduce
David Gleich
 
MapReduce Tall-and-skinny QR and applications
David Gleich
 
Relaxation methods for the matrix exponential on large networks
David Gleich
 
Tall-and-skinny QR factorizations in MapReduce architectures
David Gleich
 
Fast relaxation methods for the matrix exponential
David Gleich
 
Vertex neighborhoods, low conductance cuts, and good seeds for local communit...
David Gleich
 
Ad

Similar to Using Local Spectral Methods to Robustify Graph-Based Learning (20)

PPTX
Spectral clustering Tutorial
Zitao Liu
 
PDF
GraphSignalProcessingFinalPaper
Chiraz Nafouki
 
PPTX
Unit ii-ppt
Aravindharamanan S
 
PPTX
Optimization algorithms for solving computer vision problems
Krzysztof Wegner
 
PDF
An Introduction to Spectral Graph Theory
joisino
 
PDF
Machine Learning With MapReduce, K-Means, MLE
Jason J Pulikkottil
 
PPT
. An introduction to machine learning and probabilistic ...
butest
 
PPTX
community detection
kamal berahmand
 
PDF
Graph Signal Processing for Machine Learning A Review and New Perspectives - ...
lauratoni4
 
PDF
Statistical inference of generative network models - Tiago P. Peixoto
Lake Como School of Advanced Studies
 
PDF
IJCAI13 Paper review: Large-scale spectral clustering on graphs
Akisato Kimura
 
PDF
Spectral Clustering Report
Miaolan Xie
 
PDF
Graph Analysis Beyond Linear Algebra
Jason Riedy
 
PDF
Graph Signal Processing for Machine Learning A Review and New Perspectives - ...
lauratoni4
 
PPT
Cristopher M. Bishop's tutorial on graphical models
butest
 
PPT
Cristopher M. Bishop's tutorial on graphical models
butest
 
PPT
Cristopher M. Bishop's tutorial on graphical models
butest
 
PPT
Cristopher M. Bishop's tutorial on graphical models
butest
 
PPT
Cristopher M. Bishop's tutorial on graphical models
butest
 
PPTX
Data Mining Lecture_13.pptx
Subrata Kumer Paul
 
Spectral clustering Tutorial
Zitao Liu
 
GraphSignalProcessingFinalPaper
Chiraz Nafouki
 
Unit ii-ppt
Aravindharamanan S
 
Optimization algorithms for solving computer vision problems
Krzysztof Wegner
 
An Introduction to Spectral Graph Theory
joisino
 
Machine Learning With MapReduce, K-Means, MLE
Jason J Pulikkottil
 
. An introduction to machine learning and probabilistic ...
butest
 
community detection
kamal berahmand
 
Graph Signal Processing for Machine Learning A Review and New Perspectives - ...
lauratoni4
 
Statistical inference of generative network models - Tiago P. Peixoto
Lake Como School of Advanced Studies
 
IJCAI13 Paper review: Large-scale spectral clustering on graphs
Akisato Kimura
 
Spectral Clustering Report
Miaolan Xie
 
Graph Analysis Beyond Linear Algebra
Jason Riedy
 
Graph Signal Processing for Machine Learning A Review and New Perspectives - ...
lauratoni4
 
Cristopher M. Bishop's tutorial on graphical models
butest
 
Cristopher M. Bishop's tutorial on graphical models
butest
 
Cristopher M. Bishop's tutorial on graphical models
butest
 
Cristopher M. Bishop's tutorial on graphical models
butest
 
Cristopher M. Bishop's tutorial on graphical models
butest
 
Data Mining Lecture_13.pptx
Subrata Kumer Paul
 

Recently uploaded (20)

PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
PPTX
breach-and-attack-simulation-cybersecurity-india-chennai-defenderrabbit-2025....
defencerabbit Team
 
PDF
Per capita expenditure prediction using model stacking based on satellite ima...
IAESIJAI
 
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
PDF
Chapter 3 Spatial Domain Image Processing.pdf
Getnet Tigabie Askale -(GM)
 
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
DatVoNgoc
 
PDF
Modernizing your data center with Dell and AMD
Principled Technologies
 
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
Mark Billinghurst
 
PDF
KodekX | Application Modernization Development
KodekX
 
PDF
NewMind AI Weekly Chronicles - August'25 Week I
NewMind AI
 
PPTX
Cloud computing and distributed systems.
kkkkkhan
 
PDF
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
ReZa AdineH
 
PPT
Teaching material agriculture food technology
LiaRayya
 
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
PDF
Advanced IT Governance
University of Hertfordshire
 
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
PDF
madgavkar20181017ppt McKinsey Presentation.pdf
georgschmitzdoerner
 
PPTX
Big Data Technologies - Introduction.pptx
kuthubussaman1
 
PDF
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
IAESIJAI
 
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
breach-and-attack-simulation-cybersecurity-india-chennai-defenderrabbit-2025....
defencerabbit Team
 
Per capita expenditure prediction using model stacking based on satellite ima...
IAESIJAI
 
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
Chapter 3 Spatial Domain Image Processing.pdf
Getnet Tigabie Askale -(GM)
 
20250228 LYD VKU AI Blended-Learning.pptx
DatVoNgoc
 
Modernizing your data center with Dell and AMD
Principled Technologies
 
Reach Out and Touch Someone: Haptics and Empathic Computing
Mark Billinghurst
 
KodekX | Application Modernization Development
KodekX
 
NewMind AI Weekly Chronicles - August'25 Week I
NewMind AI
 
Cloud computing and distributed systems.
kkkkkhan
 
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
ReZa AdineH
 
Teaching material agriculture food technology
LiaRayya
 
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
Advanced IT Governance
University of Hertfordshire
 
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
madgavkar20181017ppt McKinsey Presentation.pdf
georgschmitzdoerner
 
Big Data Technologies - Introduction.pptx
kuthubussaman1
 
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
IAESIJAI
 

Using Local Spectral Methods to Robustify Graph-Based Learning

  • 1. Using Local Spectral Methods to Robustify Graph-Based Learning David F. Gleich! Purdue University! Joint work with Michael Mahoney @ Berkeley supported by " NSF CAREER CCF-1149756 Code www.cs.purdue.edu/homes/dgleich/codes/robust-diffusions! KDD2015
  • 2. The graph-based data analysis pipeline 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 " Raw data! •  Relationships •  Images •  Text records •  Etc. " Convert to a graph! •  Nearest neighs •  Kernels •  2-mode to 1-mode •  Etc. " Algorithm/Learning! •  Important nodes •  Infer features •  Clustering •  Etc. KDD2015 David Gleich ¡ Purdue 2
  • 3. “Noise” in the initial data modeling decisions " Explicit graphs! are those that are given to a data analyst. “A social network” •  Known spam accounts included? •  Users not logged in for a year? •  Etc. A type of noise " Constructed graphs! are built based on some other primary data." “nearest neighbor graphs” •  K-NN or Îľ-NN •  Thresholding correlations to zero Often made for computational convenience! (Graph too big.) A different type a noise! " Labeled graphs! occur in information diffusion/propagation “function prediction” •  Labeled nodes •  Labeled edges •  Some are wrong A direct type of noise! Do these decisions matter? Our experience Yes! Dramatically so! KDD2015 David Gleich ¡ Purdue 3
  • 4. The graph-based data analysis pipeline 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 " Raw data! •  Relationships •  Images •  Text records •  Etc. " Convert to a graph! •  Nearest neighs •  Kernels •  2-mode to 1-mode •  Etc. " Algorithm/Learning! •  Important nodes •  Infer features •  Clustering •  Etc. Most algorithmic and statistical research happens here The potential downstream signal is determined by this step KDD2015 David Gleich ¡ Purdue 4
  • 5. Our goal: towards an integrative analysis 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 ! •  How does the graph creation process affect the outcomes of graph-based learning? •  Is there anything we can do to make this process more robust? KDD2015 David Gleich ¡ Purdue 5
  • 6. Graph-based learning is usually only one component of a big pipeline KDD2015 David Gleich ¡ Purdue 6 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 1 0 0 0 1 0 0 1 0 1 0 1 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 Many databases over" genes with survival rates" for various cancers List of possible genes" responsible for survival Cluster analysis Reinteration of data THIS STEP SHOULD! BE ROBUST TO ! VARIATIONS ABOVE!
  • 7. Scalable graph analytics Local methods are one of the most successful classes of scalable graph analytics They don’t even look at the entire graph. •  Andersen Chung Lang (ACL) Push method Conjecture" Local methods regularize some variant of the original algorithm or problem. Justication" For ACL and a few relatives this is exact! Impact?! Improved robustness to noise? KDD2015 David Gleich ¡ Purdue 7 c For instance, to " answer “what function” is shared by the started node, we’d only look at the circled region.
  • 8. Our contributions We study these issues in the case of " semi-supervised learning (SSL) on graphs 1.  We illustrate a common mincut framework for a variety of SSL methods 2.  Show how to “localize” one (and make it scalable!) 3.  Provide a more robust SSL labeling method 4.  Identify a weakness in SSL methods: they cannot use extra edges! We nd one useful way to do so. KDD2015 David Gleich ¡ Purdue 8
  • 9. Semi-supervised " graph-based learning KDD2015 David Gleich ¡ Purdue 9 Given a graph, and a few labeled nodes, predict the labels on the rest of the graph. Algorithm 1.  Run a diffusion for each label (possibly with neg. info from other classes) 2.  Assign new labels based on the value of each diffusion
  • 10. Semi-supervised " graph-based learning KDD2015 David Gleich ¡ Purdue 10 Given a graph, and a few labeled nodes, predict the labels on the rest of the graph. Algorithm 1.  Run a diffusion for each label (possibly with neg. info from other classes) 2.  Assign new labels based on the value of each diffusion
  • 11. Semi-supervised " graph-based learning KDD2015 David Gleich ¡ Purdue 11 Given a graph, and a few labeled nodes, predict the labels on the rest of the graph. Algorithm 1.  Run a diffusion for each label (possibly with neg. info from other classes) 2.  Assign new labels based on the value of each diffusion
  • 12. The diffusions proposed for semi- supervised learning are s,t-cut minorants KDD2015 David Gleich ¡ Purdue 12 1 3 2 6 4 5 7 8 9 10 t s In the unweighted case, " solve via max-flow. In the weighted case, solve via network simplex or industrial LP. minimize qP ij2E Ci,j |xi xj |2 subject to xs = 1, xt = 0. minimize P ij2E Ci,j |xi xj | subject to xs = 1, xt = 0. MINCUT LP Spectral minorant – lin. sys.
  • 13. Representative cut problems KDD2015 David Gleich ¡ Purdue 13 ∞ ∞ ∞ ∞ s t ZGL Îą Îą 4Îą 3Îą 4Îą 6Îą 3Îą 3Îą 5Îą 5Îą 5Îą 2Îą 5Îą 4Îą 5Îą s t Zhou et al. Positive label Neg. label Unlabeled Andersen-Lang weighting " variation too Joachims has a variation too. Zhou et al. NIPS 2003; Zhu et al., ICML 2003; Andersen Lang, SODA 2008; Joachims, ICML 2003 These help our intuition about the solutions All spectral minorants are linear systems.
  • 14. Implicit regularization views on the Zhou et al. diffusion KDD2015 David Gleich ¡ Purdue 14 Îą Îą 4Îą 3Îą 4Îą 6Îą 3Îą 3Îą 5Îą 5Îą 5Îą 2Îą 5Îą 4Îą 5Îą s t Zhou et al. RESULT! The spectral minorant of Zhou is equivalent to the weakly-local MOV solution. PROOF! The two linear systems are the same (after working out a few equivalences). IMPORTANCE! We’d expect Zhou to be “more robust” minimize qP ij2E Ci,j |xi xj |2 subject to xs = 1, xt = 0. The Mahoney-Orecchia-Vishnoi (MOV) vector is a localized variation on the Fiedler vector to nd a small conductance set nearby a seed set.
  • 15. A scalable, localized algorithm for Zhou et al’s diffusion. KDD2015 David Gleich ¡ Purdue 15 RESULT! We can use a variation on coordinate descent methods related to the Andersen-Chung-Lang PUSH procedure to solve Zhou’s diffusion in a scalable manner. PROOF. See Gleich-Mahoney ICML ‘14 IMPORTANCE (1)! We should be able to make Zhou et al. scale. IMPORTANCE (2)! Using this algorithm adds another implicit regularization term that should further improve robustness! minimize qP ij2E Ci,j |xi xj |2 subject to xs = 1, xt = 0. minimize P ij2E Ci,j |xi xj |2 + ⌧ P i2V di xi subject to xs = 1, xt = 0, xi 0.
  • 16. Semi-supervised " graph-based learning KDD2015 David Gleich ¡ Purdue 16 Given a graph, and a few labeled nodes, predict the labels on the rest of the graph. Algorithm 1.  Run a diffusion for each label (possibly with neg. info from other classes) 2.  Assign new labels based on the value of each diffusion
  • 17. Traditional rounding methods for SSL are value-based KDD2015 David Gleich ¡ Purdue 17 Class 1 Class 2Class 3 Class 1 Class 2 Class 3 CLASS 1 CLASS 2 CLASS 3 VALUE-BASED Use the largest value of the diffusion to pick the label. Zhou’s diffusion
  • 18. But value based rounding doesn’t work for all diffusions KDD2015 David Gleich ¡ Purdue 18 Class 1 Class 2Class 3 Class 1 Class 2 Class 3 Class 1 Class 2Class 3 Class 1 Class 2Class 3 Class 1 Class 2Class 3 (b) Zhou et al., l = 3 (c) Andersen-Lang, l = 3 (d) Joachims, l = 3 (e) ZGL, l = 3 Class 1 Class 2Class 3 Class 1 Class 2Class 3 Class 1 Class 2Class 3 Class 1 Class 2Class 3 (f) Zhou et al., l = 15 (g) Andersen-Lang, l = 15 (h) Joachims, l = 15 (i) ZGL, l = 15 CLASS 1 CLASS 3 CLASS 2 VALUE-BASED rounding fails for most of these diffusions BUT There is still a signal there! Adding more labels doesn’t help either, see the paper for those details
  • 19. Rank-based rounding is far more robust. KDD2015 David Gleich ¡ Purdue 19 Class 1 Class 2Class 3 NEW IDEA! Look at the RANK of the item in each diffusion instead of it’s VALUE. JUSTIFICATION! Based on the idea of sweep-cut rounding in spectral methods (use the order induced by the eigenvector, not its values) IMPACT! Much more robust rounding to labels
  • 20. Rank-based rounding has a big impact on a real-study. KDD2015 David Gleich ¡ Purdue 20 2 4 6 8 10 0 0.2 0.4 0.6 0.8 errorrate average training samples per class Zhou Zhou+Push 2 4 6 8 10 0 0.2 0.4 0.6 0.8 1 errorrate average training samples per class Zhou Zhou+Push We used the digit prediction task out of Zhou’s paper and added just a bit of noise" as label errors and switched parameters. VALUE-BASED RANK-BASED
  • 21. Main empirical results 1.  Zhou’s diffusion seems to work best for sparse graphs whereas the ZGL diffusion works best for dense 2.  On the digit’s dataset, dense graph constructions yield higher error rates 3.  Densifying the a super-sparse graph construction on the digits dataset yields lower error. 4.  And a similar fact holds on an Amazon co- purchasing network. KDD2015 David Gleich ¡ Purdue 21
  • 22. 5 10 0 50 100 150 number of labels numberofmistakes An illustrative synthetic problem shows the differences. Two-class block-model, 150 nodes each" between prob = 0.02, " withinprob = 0.35 (dense) or 0.06 (sparse) Reveal labels for k nodes (varied) and we have different error rates (sparse 0/10% low/high) and dense (20%/60% for low-high) KDD2015 David Gleich ¡ Purdue 22 5 10 0 50 100 150 number of labels numberofmistakes Joachims Zhou ZGL Real-world scenario" sparse graph, high error Sparse graph," low error 20 40 60 0 20 40 60 80 100 number of labels numberofmistakes 20 40 60 0 20 40 60 80 100 number of labels numberofmistakes Dense graph," low error rate Dense graph," high error rate
  • 23. Varying density in an SSL construction. KDD2015 David Gleich ¡ Purdue 23 Ai,j = exp ✓ kdi dj k2 2 2 2 ◆ di dj = 2.5 = 1.25 We use the digits experiment from Zhou et al. 2003. 10 digits and a few label errors. We vary density either by the num. of nearest neighbors or by the kernel width.
  • 24. As density increases, the results just get worse KDD2015 David Gleich ¡ Purdue 24 1 1.5 2 2.5 0 0.1 0.2 0.3 0.4 errorrate σ 0.8 1.2 1.5 1.8 2.1 2.5 Zhou Zhou+Push 10 2 0 0.1 0.2 0.3 0.4 errorrate nearest neighbors 5 10 25 50 100 150 200 250 Zhou Zhou+Push Varying kernel width Varying nearest neighors •  Adding “more” edges seems to only hurt. (Unless there is no signal). •  Zhou+Push seems to be slightly more robust (Maybe).
  • 25. Some observations and a question. Adding “more data” yields “worse results” for this procedure (in a simple setting). Suppose I have a real-world system that can work with up to E edges on some graph. Is there a way I can create new edges? KDD2015 David Gleich ¡ Purdue 25
  • 26. Densifying the graph with path expansions KDD2015 David Gleich ¡ Purdue 26 Ak = kX `=1 A` If A is the adjacency matrix, then this counts the total weight on all paths up to length k. We now repeat the nearest neighbor computation, but with paired parameters such that we have the same average degree. Zhou Zhou w. Push Avg. Deg k = 1 k 1 k = 1 k 1 19 0.163 0.114 0.156 0.117 41 0.156 0.132 0.158 0.113 53 0.183 0.142 0.179 0.136 104 0.193 0.145 0.178 0.144 138 0.216 0.102 0.204 0.101 k=4, nn = 3
  • 27. The same result holds for Amazon’s co-purchasing network KDD2015 David Gleich ¡ Purdue 27 mean F1 Condence intervals k Zhou Zhou w. Push Zhou Zhou w. Push 1 0.173 0.229 [0.15 0.19] [0.21 0.25] 2 0.197 0.231 [0.18 0.22] [0.21 0.25] 3 0.221 0.238 [0.17 0.27] [0.19 0.28] Amazon’s co-purchasing network (on Snap) is effectively a highly sparse nearest-neighbor network from their (denser) co-purchasing graph. We attempt to predict the items in a product category based on a small sample and study the F1 score for the predictions. Some small details missing – see the full paper. Ak = kX `=1 A`
  • 28. (a) K2 sparse (b) K2 dense (c) RK2 Figure 2: We articially densify this graph to Ak based on a cons and dense di↵usions and regularization. The color indicates the circled nodes. The unavoidable errors are caused by a mislabeled regularizing di↵usions on dense graphs produces only a small Towards some theory, i.e. why are densied sparse graphs better? KDD2015 David Gleich ¡ Purdue 28 How do sparsity, density, and regularization of a diffusion play into the results in a controlled setting? THE ERROR Labels
  • 29. (a) K2 sparse (b) K2 dense (c) RK2 Figure 2: We articially densify this graph to Ak based on a cons and dense di↵usions and regularization. The color indicates the circled nodes. The unavoidable errors are caused by a mislabeled regularizing di↵usions on dense graphs produces only a small Towards some theory, i.e. why are densied sparse graphs better? KDD2015 David Gleich ¡ Purdue 29 THE ERROR Labels Using Push algorithm P5 `=1 A` P5 `=1 A` Regularization Dense Dense
  • 30. Recap, discussion, future work Contributions 1.  Flow-setup for SSL diffusions 2.  New robust rounding rule for class selection 3.  Localized Zhou’s diffusion 4.  Empirical insights on density of graph constructions Observations •  Many of these insights translate to directed, weighted graphs with fuzzy labels and/or some parallel architectures. •  Weakness mainly empirical results on the density. •  We need a theoretical basis for the densication theory! KDD2015 David Gleich ¡ Purdue 30 Supported by NSF, ARO, DARPA CODE www.cs.purdue.edu/homes/dgleich/codes/robust-diffusions