Bolt: Building A Distributed ndarray
2 likes664 views
Bolt is a distributed ndarray built on PySpark that conforms to NumPy's API. It can handle large multidimensional datasets exceeding 1 TB in size with up to 1011 elements. By distributing arrays across a cluster, Bolt enables operations on large neuroscience, astronomy, geospatial and climate science datasets that would be impossible on a single machine due to limits of memory and processing power. Key features include indexing, slicing, transpose, and applying functions along axes.
1 of 40
Downloaded 207 times
![data.mean(axis=0)
data[2, 4:10]
data.T](https://image.slidesharecdn.com/2ajasonwittenbach-160614182655/85/Bolt-Building-A-Distributed-ndarray-15-320.jpg)
More Related Content
Viewers also liked (20)
Ad
Similar to Bolt: Building A Distributed ndarray (20)
Ad
More from Jen Aman (20)
Bolt: Building A Distributed ndarray
- 1. Bolt: Building a distributed ndarray Jason Wittenbach Janelia Research Campus (HHMI) Freeman Lab
- 4. t, (x, y, z) time ~ 104 space ~ 107 ~ 1011 elements ~ 1 TB
- 5. (x, y, t) (x, y, z, t) (x, y, z, c, t) (n, k)
- 8. time
- 9. time
- 10. time
- 11. x y
- 14. • a distributed ndarray • built on PySpark • conforms to NumPy API
- 16. (t, x, y, z) (x, y, z, t)
- 17. (v, w, x, y, z) (v,w) y x z
- 18. (v,w,x) (v, w, x, y, z) y z
- 19. (v, w, x, y, z) (v)
- 28. (t x,y,z) (x,y,z t)
- 29. (t x, y, z) (t) x y z
- 30. (t x, y, z) (t) x y z
- 31. (t x, y, z) (t,chunk) (t,chunk) (t,chunk)
- 39. ???
- 40. thanks join us! Jeremy Freeman Nicholas Sofroniew Andrew Osheroff Freeman Lab Ken Carlile Robert Lines Janelia Scientific Computing bolt-project thunder-project GitHub @jsonWittenbach Twitter