Parallel Algorithms for Geometric Graph Problems (at Stanford)
Download as PPTX, PDF2 likes984 views
This document summarizes work on developing parallel algorithms for approximating problems on geometric graphs. Specifically, it presents algorithms for computing a (1+ε)-approximate minimum spanning tree (MST) and earth-mover distance in O(1) rounds of parallel computation using a "solve-and-sketch" framework. The MST algorithm imposes a randomly shifted grid tree and computes MSTs within cells, using only short edges and representative points between cells. This achieves an approximation ratio of 1+O(ε) in O(1) rounds. The framework is also extended to compute a (1+ε)-approximate transportation cost.
![Approximation in Graphs
1930-50s: Given a graph and an optimization
problem…
Transportation Problem:
Tolstoi [1930]
Minimum Cut (RAND):
Harris and Ross [1955] (declassified, 1999)](https://image.slidesharecdn.com/stanford-140328065628-phpapp01/85/Parallel-Algorithms-for-Geometric-Graph-Problems-at-Stanford-5-320.jpg)
![MST: Single Linkage Clustering
• [Zahn’71] Clustering via MST (Single-linkage):
k clusters: remove 𝒌 − 𝟏 longest edges from MST
• Maximizes minimum intercluster distance
[Kleinberg, Tardos]](https://image.slidesharecdn.com/stanford-140328065628-phpapp01/85/Parallel-Algorithms-for-Geometric-Graph-Problems-at-Stanford-12-320.jpg)
![MapReduce-style computations
What I won’t discuss today
• PRAMs (shared memory, multiple processors) (see
e.g. [Karloff, Suri, Vassilvitskii‘10])
– Computing XOR requires Ω(log 𝑛) rounds in CRCW PRAM
– Can be done in 𝑂(log 𝒔 𝑛) rounds of MapReduce
• Pregel-style systems, Distributed Hash Tables (see
e.g. Ashish Goel’s class notes and papers)
• Lower-level implementation details (see e.g.
Rajaraman-Leskovec-Ullman book)](https://image.slidesharecdn.com/stanford-140328065628-phpapp01/85/Parallel-Algorithms-for-Geometric-Graph-Problems-at-Stanford-16-320.jpg)
![Models of parallel computation
• Bulk-Synchronous Parallel Model (BSP) [Valiant,90]
Pro: Most general, generalizes all other models
Con: Many parameters, hard to design algorithms
• Massive Parallel Computation [Feldman-Muthukrishnan-
Sidiropoulos-Stein-Svitkina’07, Karloff-Suri-Vassilvitskii’10,
Goodrich-Sitchinava-Zhang’11, ..., Beame, Koutris, Suciu’13]
Pros:
• Inspired by modern systems (Hadoop, MapReduce, Dryad,
Pregel, … )
• Few parameters, simple to design algorithms
• New algorithmic ideas, robust to the exact model specification
• # Rounds is an information-theoretic measure => can prove
unconditional lower bounds
• Between linear sketching and streaming with sorting](https://image.slidesharecdn.com/stanford-140328065628-phpapp01/85/Parallel-Algorithms-for-Geometric-Graph-Problems-at-Stanford-17-320.jpg)
![Previous work
• Dense graphs vs. sparse graphs
– Dense: 𝑺 ≫ 𝒏 (or 𝑺 ≫ solution size)
“Filtering” (Output fits on a single machine) [Karloff, Suri
Vassilvitskii, SODA’10; Ene, Im, Moseley, KDD’11; Lattanzi,
Moseley, Suri, Vassilvitskii, SPAA’11; Suri, Vassilvitskii,
WWW’11]
– Sparse: 𝑺 ≪ 𝒏 (or 𝑺 ≪ solution size)
Sparse graph problems appear hard (Big open question:
(s,t)-connectivity in o(log 𝑛) rounds?)
VS.](https://image.slidesharecdn.com/stanford-140328065628-phpapp01/85/Parallel-Algorithms-for-Geometric-Graph-Problems-at-Stanford-18-320.jpg)
![Randomly shifted quadtree
• Top cell shifted by a random vector in 0, 𝑳 2
Impose a randomly shifted quadtree (top cell length 𝟐𝚫)
Bottom-up: For each cell in the quadtree
– Compute optimum MSTs in subcells
– Use 𝝐𝑳-net from each cell on the next level
Pay 5 instead of 4
Pr[𝐁𝐚𝐝 𝐂𝐮𝐭] = 𝛀(1)
2
1
𝐁𝐚𝐝 𝐂𝐮𝐭](https://image.slidesharecdn.com/stanford-140328065628-phpapp01/85/Parallel-Algorithms-for-Geometric-Graph-Problems-at-Stanford-22-320.jpg)
![1 + 𝝐 -MST in 𝐑 = 𝑂(log 𝑛) rounds
• Idea: Only use short edges inside the cells
Impose a randomly shifted quadtree (top cell length
𝟐𝚫
𝝐
)
Bottom-up: For each node (cell) in the quadtree
– compute optimum Minimum Spanning Forests in subcells,
using edges of length ≤ 𝝐𝑳
– Use only 𝝐 𝟐
𝑳-net from each cell on the next level
2
1
Pr[𝐁𝐚𝐝 𝐂𝐮𝐭] = 𝑶(𝝐)
𝑳 = 𝛀(
𝟏
𝝐
)](https://image.slidesharecdn.com/stanford-140328065628-phpapp01/85/Parallel-Algorithms-for-Geometric-Graph-Problems-at-Stanford-23-320.jpg)
![1 + 𝝐 -MST in 𝐑 = 𝑂(1) rounds
Theorem: Let 𝒍 = # levels in a random tree P
𝔼 𝑷 𝐀𝐋𝐆 ≤ 1 + 𝑂 𝝐𝒍𝒅 𝐎𝐏𝐓
Proof (sketch):
• 𝚫 𝑷(𝑢, 𝑣) = cell length, which first partitions (𝑢, 𝑣)
• New weights: 𝒘 𝑷 𝑢, 𝑣 = 𝑢 − 𝑣 2
+ 𝝐𝚫 𝑷 𝑢, 𝑣
𝑢 − 𝑣 2
≤ 𝔼 𝑷[𝒘 𝑷 𝑢, 𝑣 ] ≤ 1 + 𝑂 𝝐𝒍𝒅 𝑢 − 𝑣 2
• Our algorithm implements Kruskal for weights 𝒘 𝑷
𝑢 𝑣
𝚫 𝑷 𝑢, 𝑣](https://image.slidesharecdn.com/stanford-140328065628-phpapp01/85/Parallel-Algorithms-for-Geometric-Graph-Problems-at-Stanford-25-320.jpg)
![“Solve-And-Sketch” Framework
(1 + 𝜖)-MST:
– “Load balancing”: partition the tree into parts of
the same size
– Almost linear time: Approximate Nearest
Neighbor data structure [Indyk’99]
– Dependence on dimension d (size of 𝝐-net is
𝑂
𝒅
𝝐
𝒅
)
– Generalizes to bounded doubling dimension
– Basic version is teachable (Jelani Nelson’s ``Big
Data’’ class at Harvard)](https://image.slidesharecdn.com/stanford-140328065628-phpapp01/85/Parallel-Algorithms-for-Geometric-Graph-Problems-at-Stanford-26-320.jpg)
![“Solve-And-Sketch” Framework
(1 + 𝜖)-Earth-Mover Distance, Transportation Cost
• No simple “divide-and-conquer” Arora-Mitchell-style
algorithm (unlike for general matching)
• Only recently sequential 1 + 𝜖 -apprxoimation in
𝑂𝜖 𝒏 log 𝑂 1
𝒏 time [Sharathkumar, Agarwal ‘12]
Our approach (convex sketching):
• Switch to the flow-based version
• In every cell, send the flow to the closest net-point
until we can connect the net points](https://image.slidesharecdn.com/stanford-140328065628-phpapp01/85/Parallel-Algorithms-for-Geometric-Graph-Problems-at-Stanford-27-320.jpg)
Parallel Algorithms for Geometric Graph Problems (at Stanford)
- 1. Parallel Algorithms for Geometric Graph Problems Grigory Yaroslavtsev http://grigory.us To appear in STOC 2014, joint work with Alexandr Andoni, Krzysztof Onak and Aleksandar Nikolov.
- 2. Postdoctoral Fellow icerm.brown.edu By Mike Cohea • Spring 2014: “Network Science and Graph Algorithms” • Fall 2014: “High-dimensional approximation”
- 3. “The Big Data Theory” What should TCS say about big data? • This talk: – Running time: (almost) linear, sublinear, … – Space: linear, sublinear, … – Approximation: 1 + 𝜖 , best possible, … – Randomness: as little as possible, … • Special focus today: roundcomplexity
- 4. Round Complexity Information-theoretic measure of performance • Tools from information theory (Shannon’48) • Unconditional results (lower bounds) Example: • Approximating Geometric Graph Problems
- 5. Approximation in Graphs 1930-50s: Given a graph and an optimization problem… Transportation Problem: Tolstoi [1930] Minimum Cut (RAND): Harris and Ross [1955] (declassified, 1999)
- 6. Approximation in Graphs 1960s: Single processor, main memory (IBM 360)
- 7. Approximation in Graphs 1970s: NP-complete problem – hard to solve exactly in time polynomial in the input size “Black Book”
- 8. Approximation in Graphs Approximate with multiplicative error 𝜶 on the worst- case graph 𝐺: 𝑚𝑎𝑥 𝐺 𝐴𝑙𝑔𝑜𝑟𝑖𝑡ℎ𝑚(𝐺) 𝑂𝑝𝑡𝑖𝑚𝑢𝑚(𝐺) ≤ 𝜶 Generic methods: • Linear programming • Semidefinite programming • Hierarchies of linear and semidefinite programs • Sum-of-squares hierarchies • …
- 9. The New: Approximating Geometric Problems in Parallel Models 1930-70s to 2014
- 10. The New: Approximating Geometric Problems in Parallel Models Geometric graph (implicit): Euclidean distances between n points in ℝ 𝒅 Already have solutions for old NP-hard problems (Traveling Salesman, Steiner Tree, etc.) • Minimum Spanning Tree (clustering, vision) • Minimum Cost Bichromatic Matching (vision)
- 11. Polynomial time (easy) • Minimum Spanning Tree • Earth-Mover Distance = Min Weight Bi-chromatic Matching NP-hard (hard) • Steiner Tree • Traveling Salesman • Clustering (k-medians, facility location, etc.) Geometric Graph Problems Combinatorial problems on graphs in ℝ 𝒅 Arora-Mitchell-style “Divide and Conquer”, easy to implement in Massively Parallel Computational Models Need new theory!
- 12. MST: Single Linkage Clustering • [Zahn’71] Clustering via MST (Single-linkage): k clusters: remove 𝒌 − 𝟏 longest edges from MST • Maximizes minimum intercluster distance [Kleinberg, Tardos]
- 13. Earth-Mover Distance • Computer vision: compare two pictures of moving objects (stars, MRI scans)
- 14. Computational Model • Input: n points in a d-dimensional space (d constant) • 𝑴 machines, space 𝑺 on each (𝑺 = 𝒏 𝛼 , 0 < 𝛼 < 1 ) – Constant overhead in total space: 𝑴 ⋅ 𝑺 = 𝑂(𝒏) • Output: solution to a geometric problem (size O(𝒏)) – Doesn’t fit on a single machine (𝑺 ≪ 𝒏) 𝑴 machines S space 𝐈𝐧𝐩𝐮𝐭: 𝒏 points ⇒ ⇒ 𝐎𝐮𝐭𝐩𝐮𝐭: 𝑠𝑖𝑧𝑒 𝑶(𝒏)
- 15. 𝑴 machines S space Computational Model • Computation/Communication in 𝑹 rounds: – Every machine performs a near-linear time computation => Total running time 𝑂(𝒏 𝟏+𝒐(𝟏) 𝑹) – Every machine sends/receives at most 𝑺 bits of information => Total communication 𝑂(𝒏𝑹). Goal: Minimize 𝑹. Our work: 𝑹 = constant. 𝑶(𝑺 𝟏+𝒐(𝟏)) time ≤ 𝑺 bits
- 16. MapReduce-style computations What I won’t discuss today • PRAMs (shared memory, multiple processors) (see e.g. [Karloff, Suri, Vassilvitskii‘10]) – Computing XOR requires Ω(log 𝑛) rounds in CRCW PRAM – Can be done in 𝑂(log 𝒔 𝑛) rounds of MapReduce • Pregel-style systems, Distributed Hash Tables (see e.g. Ashish Goel’s class notes and papers) • Lower-level implementation details (see e.g. Rajaraman-Leskovec-Ullman book)
- 17. Models of parallel computation • Bulk-Synchronous Parallel Model (BSP) [Valiant,90] Pro: Most general, generalizes all other models Con: Many parameters, hard to design algorithms • Massive Parallel Computation [Feldman-Muthukrishnan- Sidiropoulos-Stein-Svitkina’07, Karloff-Suri-Vassilvitskii’10, Goodrich-Sitchinava-Zhang’11, ..., Beame, Koutris, Suciu’13] Pros: • Inspired by modern systems (Hadoop, MapReduce, Dryad, Pregel, … ) • Few parameters, simple to design algorithms • New algorithmic ideas, robust to the exact model specification • # Rounds is an information-theoretic measure => can prove unconditional lower bounds • Between linear sketching and streaming with sorting
- 18. Previous work • Dense graphs vs. sparse graphs – Dense: 𝑺 ≫ 𝒏 (or 𝑺 ≫ solution size) “Filtering” (Output fits on a single machine) [Karloff, Suri Vassilvitskii, SODA’10; Ene, Im, Moseley, KDD’11; Lattanzi, Moseley, Suri, Vassilvitskii, SPAA’11; Suri, Vassilvitskii, WWW’11] – Sparse: 𝑺 ≪ 𝒏 (or 𝑺 ≪ solution size) Sparse graph problems appear hard (Big open question: (s,t)-connectivity in o(log 𝑛) rounds?) VS.
- 19. Large geometric graphs • Graph algorithms: Dense graphs vs. sparse graphs – Dense: 𝑺 ≫ 𝒏. – Sparse: 𝑺 ≪ 𝒏. • Our setting: – Dense graphs, sparsely represented: O(n) space – Output doesn’t fit on one machine (𝑺 ≪ 𝒏) • Today: (1 + 𝜖)-approximate MST – 𝒅 = 2 (easy to generalize) – 𝑹 = log 𝑺 𝒏 = O(1) rounds (𝑺 = 𝒏 𝛀(𝟏))
- 20. 𝑂(log 𝑛)-MST in 𝑅 = 𝑂(log 𝑛) rounds • Assume points have integer coordinates 0, … , Δ , where Δ = 𝑂 𝒏 𝟐 . Impose an 𝑂(log 𝒏)-depth quadtree Bottom-up: For each cell in the quadtree – compute optimum MSTs in subcells – Use only one representative from each cell on the next level Wrong representative: O(1)-approximation per level
- 21. Wrong representative: O(1)-approximation per level 𝝐𝑳-nets • 𝝐𝑳-net for a cell C with side length 𝑳: Collection S of vertices in C, every vertex is at distance <= 𝝐𝑳 from some vertex in S. (Fact: Can efficiently compute 𝝐-net of size 𝑂 1 𝝐2 ) Bottom-up: For each cell in the quadtree – Compute optimum MSTs in subcells – Use 𝝐𝑳-net from each cell on the next level • Idea: Pay only O(𝝐𝑳) for an edge cut by cell with side 𝑳 • Randomly shift the quadtree: Pr 𝑐𝑢𝑡 𝑒𝑑𝑔𝑒 𝑜𝑓 𝑙𝑒𝑛𝑔𝑡ℎ ℓ 𝑏𝑦 𝑳 ∼ ℓ/𝑳 – charge errors 𝑳 𝑳𝜖𝑳
- 22. Randomly shifted quadtree • Top cell shifted by a random vector in 0, 𝑳 2 Impose a randomly shifted quadtree (top cell length 𝟐𝚫) Bottom-up: For each cell in the quadtree – Compute optimum MSTs in subcells – Use 𝝐𝑳-net from each cell on the next level Pay 5 instead of 4 Pr[𝐁𝐚𝐝 𝐂𝐮𝐭] = 𝛀(1) 2 1 𝐁𝐚𝐝 𝐂𝐮𝐭
- 23. 1 + 𝝐 -MST in 𝐑 = 𝑂(log 𝑛) rounds • Idea: Only use short edges inside the cells Impose a randomly shifted quadtree (top cell length 𝟐𝚫 𝝐 ) Bottom-up: For each node (cell) in the quadtree – compute optimum Minimum Spanning Forests in subcells, using edges of length ≤ 𝝐𝑳 – Use only 𝝐 𝟐 𝑳-net from each cell on the next level 2 1 Pr[𝐁𝐚𝐝 𝐂𝐮𝐭] = 𝑶(𝝐) 𝑳 = 𝛀( 𝟏 𝝐 )
- 24. 1 + 𝝐 -MST in 𝐑 = 𝑂(1) rounds • 𝑂(log 𝒏) rounds => O(log 𝑺 𝒏) = O(1) rounds – Flatten the tree: ( 𝑺 × 𝑺)-grids instead of (2x2) grids at each level. Impose a randomly shifted ( 𝑺 × 𝑺)-tree Bottom-up: For each node (cell) in the tree – compute optimum MSTs in subcells via edges of length ≤ 𝝐𝑳 – Use only 𝝐 𝟐 𝑳-net from each cell on the next level ⇒ 𝑺 = 𝒏Ω(1)
- 25. 1 + 𝝐 -MST in 𝐑 = 𝑂(1) rounds Theorem: Let 𝒍 = # levels in a random tree P 𝔼 𝑷 𝐀𝐋𝐆 ≤ 1 + 𝑂 𝝐𝒍𝒅 𝐎𝐏𝐓 Proof (sketch): • 𝚫 𝑷(𝑢, 𝑣) = cell length, which first partitions (𝑢, 𝑣) • New weights: 𝒘 𝑷 𝑢, 𝑣 = 𝑢 − 𝑣 2 + 𝝐𝚫 𝑷 𝑢, 𝑣 𝑢 − 𝑣 2 ≤ 𝔼 𝑷[𝒘 𝑷 𝑢, 𝑣 ] ≤ 1 + 𝑂 𝝐𝒍𝒅 𝑢 − 𝑣 2 • Our algorithm implements Kruskal for weights 𝒘 𝑷 𝑢 𝑣 𝚫 𝑷 𝑢, 𝑣
- 26. “Solve-And-Sketch” Framework (1 + 𝜖)-MST: – “Load balancing”: partition the tree into parts of the same size – Almost linear time: Approximate Nearest Neighbor data structure [Indyk’99] – Dependence on dimension d (size of 𝝐-net is 𝑂 𝒅 𝝐 𝒅 ) – Generalizes to bounded doubling dimension – Basic version is teachable (Jelani Nelson’s ``Big Data’’ class at Harvard)
- 27. “Solve-And-Sketch” Framework (1 + 𝜖)-Earth-Mover Distance, Transportation Cost • No simple “divide-and-conquer” Arora-Mitchell-style algorithm (unlike for general matching) • Only recently sequential 1 + 𝜖 -apprxoimation in 𝑂𝜖 𝒏 log 𝑂 1 𝒏 time [Sharathkumar, Agarwal ‘12] Our approach (convex sketching): • Switch to the flow-based version • In every cell, send the flow to the closest net-point until we can connect the net points
- 28. “Solve-And-Sketch” Framework Convex sketching the cost function for 𝝉 net points • 𝐹: ℝ 𝝉−1 → ℝ = the cost of routing fixed amounts of flow through the net points • Function 𝐹’ = 𝐹 + “normalization” is monotone, convex and Lipschitz, (1 + 𝝐)- approximates 𝐹 • We can (1 + 𝝐)-sketch it using a lower convex hull
- 29. Thank you! http://grigory.us Open problems • Exetension to high dimensions? – Probably no, reduce from connectivity => conditional lower bound ∶ Ω log 𝑛 rounds for MST in ℓ∞ 𝑛 – The difficult setting is 𝑑 = Ω(log 𝒏) (can do JL) • Streaming alg for EMD and Transporation Cost? • Our work: first near-linear time algorithm for Transportation Cost – Is it possible to reconstruct the solution itself?