![Isomap
For n data points, and a distance matrix D,
i
Dij =
j
...we can construct a m-dimensional space to
preserve inter-point distances by using the top
eigenvectors of D scaled by their eigenvalues.
yi= [ λ1v1i , λ2v2i , ... , λmvmi ]](https://image.slidesharecdn.com/thompsondimensionalityreduction-110404033648-phpapp02/85/Manifold-learning-with-application-to-object-recognition-15-320.jpg)
Manifold learning with application to object recognition
- 1. manifold learning with applications to object recognition Advanced Perception David R. Thompson
- 2. agenda 1. why learn manifolds? 2. Isomap 3. LLE 4. applications
- 3. types of manifolds exhaust Sir Walter manifold Synnot Manifold 1849-1928 low-D surface embedded in high-D space
- 4. manifold learning Find a low-D basis for describing high-D data. X → X' S.T. dim(X') << dim(X) uncovers the intrinsic dimensionality (invertible)
- 5. manifolds in vision plenoptic function / motion / occlusion
- 6. manifolds in vision appearance variation images from hormel corp.
- 7. manifolds in vision deformation images from www.golfswingphotos.com
- 8. why do manifold learning? 1. data compression 2. “curse of dimensionality” 3. de-noising 4. visualization 5. reasonable distance metrics *
- 12. reasonable distance metrics ? linear interpolation
- 13. reasonable distance metrics ? manifold interpolation
- 14. agenda 1. why learn manifolds? 2. Isomap 3. LLE 4. applications
- 15. Isomap For n data points, and a distance matrix D, i Dij = j ...we can construct a m-dimensional space to preserve inter-point distances by using the top eigenvectors of D scaled by their eigenvalues. yi= [ λ1v1i , λ2v2i , ... , λmvmi ]
- 16. Isomap Infer a distance matrix using distances along the manifold.
- 17. Isomap 1. Build a sparse graph with K-nearest neighbors Dg = (distance matrix is sparse)
- 18. Isomap 2. Infer other interpoint distances by finding shortest paths on the graph (Dijkstra's algorithm). Dg =
- 19. Isomap 3. Build a low-D embedded space to best preserve the complete distance matrix. Error function: inner product distances in new inner product coordinate distances in system graph L2 norm Solution – set points Y to top eigenvectors of Dg
- 20. Isomap shortest-distance on a graph is easy to compute
- 22. Isomap: pro and con - preserves global structure - few free parameters - sensitive to noise, noise edges - computationally expensive (dense matrix eigen-reduction)
- 23. Locally Linear Embedding Find a mapping to preserve local linear relationships between neighbors
- 25. LLE: Two key steps 1. Find weight matrix W of linear coefficients: Enforce sum-to-one constraint with the Lagrange Multiplier:
- 26. LLE: Two key steps 2. Find projected vectors Y to minimize reconstruction error must solve for whole dataset simultaneously
- 27. LLE: Two key steps We add constraints to prevent multiple / degenerate solutions:
- 28. LLE: Two key steps cost function becomes: the optimal embedded coordinates are given by bottom m+1 eigenvectors of the matrix M
- 29. LLE: Result preserves local topology PCA LLE
- 30. LLE: pro and con - no local minima, one free parameter - incremental & fast - simple linear algebra operations - can distort global structure
- 31. Others you may encounter ● Laplacian Eigenmaps (Belkin 2001) ● spectral method similar to LLE ● better preserves clusters in data ● Kernel PCA ● Kohonen Self-Organizing Map (Kohonen, 1990) ● iterative algorithm fits a network of pre- defined connectivity ● simple, fast for on-line learning ● local minima ● lacking theoretical justification
- 32. No Free Lunch the “curvier” your manifold, the denser your data must be bad OK!
- 33. conclusions Manifold learning is a key tool in your object recognition toolbox A formal framework for many different ad-hoc object recognition techniques