Feature Matching using SIFT algorithm
8 likes5,026 views
The document discusses a MATLAB procedure aimed at detecting levelling rods using the SIFT algorithm developed by David Lowe, which involves keypoint extraction and matching. It covers the algorithm's ability to identify invariant features across transformations, including scaling and rotation, utilizing methods like RANSAC to filter outliers. The final results indicate the algorithm's effectiveness, while also noting some limitations regarding image types and transformations.
Feature Matching using SIFT algorithm
- 1. DETECTING LEVELLING RODS USING SIFT FEATURE MATCHING GROUP 1 MSc Course 2006-08 25TH June 2007 Sajid Pareeth Sonam Tashi Gabriel Vincent Sanya Michael Mutale PHOTOGRAMMETRY STUDIO
- 2. Objective Introduction SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo To develop a MATLAB procedure for the detection of levelling rods. To carry out matching based on Algorithms proposed by David Lowe Object recognition using invariant features Transformation Examples
- 3. Summary of Steps Images SIFT Key Feature Extraction Keypoint Matching Removing Outliers - RANSAC Transformation Introduction SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples
- 4. Background - SIFT Published by David Lowe et al. 1999-2004 Algorithm to extract features that are invariant to rotation, scaling and partially invariant to changes in illumination and camera viewpoint Resulting features are highly distinctive Consists of (Major stages of computation): Scale-space extrema detection-the Gaussian Widow Keypoint localization- 4x4 samples per window in 8 directions Orientation assignment Keypoint descriptor-feature vector is modified to reduce the effects of illumination change. The vector is normalized to unit length. large gradient magnitudes by thresholding the values in the unit feature vector to each be no larger than 0.2, and then renormalizing to unit length. SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples Introduction
- 5. A keypoint descriptor is created by first computing the gradient magnitude and orientation at each image sample point in a region around the keypoint location, These are weighted by a Gaussian window, indicated by the overlaid circle. These samples are then accumulated into orientation histograms summarizing the contents over 4x4 subregions, The length of each arrow corresponding to the sum of the gradient magnitudes near that direction within the region.
- 6. Scale Invariant Detection Consider regions (e.g. circles) of different sizes around a point Regions of corresponding sizes will look the same in both images SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples SIFT Algorithm Introduction
- 7. The problem: how do we choose corresponding circles independently in each image? Scale Invariant Detection SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples SIFT Algorithm Introduction
- 8. Solution: Design a function on the region (circle), which is “scale invariant” (the same for corresponding regions, even if they are at different scales) Example: average intensity. For corresponding regions (even of different sizes) it will be the same. For a point in one image, we can consider it as a function of region size (circle radius) scale = 1/2 f region size Image 1 f region size Image 2 Scale Invariant Detection SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples SIFT Algorithm Introduction
- 9. Common approach scale = 1/2 f region size Image 1 f region size Image 2 Take a local maximum of this function Observation: region size, for which the maximum is achieved, should be invariant to image scale. s1 s2 Important: this scale invariant region size is found in each image independently! Scale Invariant Detection SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples SIFT Algorithm Introduction
- 10. scale x y ← DoG → ←DoG→ SIFT(Lowe) Maxima and minima of the difference-of-Gaussian images detected by comparing a pixel (marked with X) to its 26 neighbors in 3x3 regions at the current and adjacent scales (marked with circles). SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples SIFT Algorithm Introduction
- 11. Gaussian pyramids Scale First Octave Scale Next Octave Difference of Gaussians Difference of Gaussian(DoG) SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples SIFT Algorithm Introduction
- 12. Keypoint Descriptors Feature Vectors Thresholded image gradients sampled over: 16x16 array of locations in scale space Histogram of 4x4 samples per window in 8 directions Gaussian weighting around center 8 orientations x 4 x 4 histogram array = 128 dimensional feature vector 4x4 Gradient window SIFT Algorithm Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples SIFT-Keypoints Extraction Introduction Gaussian Gradient Window
- 13. Criteria for Selection Prominent Distinguishable-from neighborhood Invariant Stable to disturbances Rare (exceptional) - from other selected points Meaningful - with respect to image interpretation SIFT Algorithm Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples SIFT-Keypoints Extraction Introduction
- 14. SIFT keypoints Detected Keypoints in reference and candidate image Keypoints in both images will be matched Candidate image with keypointsReference image with keypoints SIFT Algorithm Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples SIFT-Keypoints Extraction Introduction
- 15. Keypoint Matching Fundamental aspect in computer vision. Based on Euclidean distance of the feature vectors. Nearest neighbor algorithms. Product between descriptors is calculated. Inverse cosine of the products gives the Euclidean distance Matches with Ratio of vector angles from the nearest to second nearest neighbor less than distRatio value are selected. SIFT Algorithm Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples SIFT-Keypoints Extraction Introduction
- 16. 90% of the false matches are removed. False matches due to ambiguos features or features arise from background clutter. Reliable object recognition with few best Matches Remove outliers Matched points SIFT Algorithm SIFT-Keypoints Extraction Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples Keypoints Matching Introduction
- 17. RANSAC: Algorithm RANdom SAmple Consensus Estimate parameters of a mathematical model from a set of observed data which contains outliers. A model is fitted using hypothetical inliers. If other data fits to the model, added to the inliers. Reestimated the model with the new set of inliers. We used ransac-fit-homography which Robustly fits a homography to a set of matched points. SIFT Algorithm SIFT-Keypoints Extraction Work Flow Advantages/ Limitations Demo Transformation Examples Keypoints Matching RANSAC Introduction
- 18. Ransac RANSAC: Result SIFT Algorithm SIFT-Keypoints Extraction Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples Keypoints Matching Introduction
- 19. Make the shortest image the same height as the other image. Append Images Transformation tform = cp2tform(M1,M2,’affine') SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow Advantages/ Limitations Demo Transformation Examples RANSAC Introduction TStructure
- 20. 2 3 Rod vs. OpenShrubs 1 Matches: 9 2 Rod: 626 keypoints found Image: 28958 keypoints found Matches: 16 …Good Matches SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Examples Transformation Introduction
- 21. 2 Rod Vs Human Occlusion?? 1 Rod: 626 keypoints found Image: 2347 keypoints found Matches: 155 Rod: 626 keypoints found Image: 2347 keypoints found Matches: 144 2 3 …Good Matches SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples Introduction
- 22. Limitations 1 Rod: 626 keypoints found Image: 17927 keypoints found Matches: 16 Matches: 4 2 Rod: 626 keypoints found Image: 30505 keypoints found 3 SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples Introduction
- 23. Advantages Locality: features are local, so robust to occlusion and clutter (no prior segmentation) Distinctiveness: individual features can be matched to a large database of objects Quantity: many features can be generated for even small objects Efficiency: close to real-time performance Extensibility: can easily be extended to wide range of differing feature types, with each adding robustness SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Demo Transformation Examples Introduction
- 24. Problems/Enhancement Only invariant to affine transformations to a certain degree Best performance on highly textured images Use Principal components analysis PCA instead of Gaussian weighting for gradients Demo SIFT Algorithm SIFT-Keypoints Extraction Keypoints Matching Work Flow RANSAC Advantages/ Limitations Transformation Examples Introduction