COMPUTATIONALLY EFFICIENT TWO STAGE SEQUENTIAL FRAMEWORK FOR STEREO MATCHING
0 likes9 views
This document presents a new computationally efficient two-stage sequential framework for stereo matching, addressing the weaknesses of existing algorithms under radiometric variations. The proposed method demonstrates superior performance and reduced execution time compared to traditional stereo matching techniques when tested with various stereo image pairs. Results indicate that the new algorithm effectively maintains accuracy despite changes in illumination and exposure conditions.
![International Journal in Foundations of Computer Science & Technology (IJFCST) Vol.6, No.5, September 2016
DOI:10.5121/ijfcst.2016.6501 1
COMPUTATIONALLY EFFICIENT TWO STAGE
SEQUENTIAL FRAMEWORK FOR STEREO
MATCHING
1
Bindu N.S and 2
H.S.Sheshadri
1
Vidyavardhaka college of Engineering, Dept of E&C, Mysore, India
2
PES College of Engineering, Dept of E&C, Mandya, India
ABSTRACT:
Almost all the existing stereo algorithms fall under a common assumption that corresponding color or
intensity values will be similar like one another. On the other hand, it is also not that true in practice
where the image color or intensity values are regularly affected by different radiometric factors like
illumination direction, change in image device, illuminant color and so on. For this issue, the information
about the raw color of the images which is recorded by the camera should not depend on it totally, and
also the common assumptions on color consistency doesn’t influence good (great) between the stereo
images in real scenario. Therefore, most of the conventional stereo algorithms can be seriously degraded
in terms of performance under radiometric variations. In this work, we intend to develop a new stereo
matching algorithm which will be insensitive to change in radiometric conditions between stereo pairs i.e.
left image as well as right image. Unlike the other stereo algorithms, we propose a computationally
efficient two stage sequential framework for stereo matching which can handle the various radiometric
variations between the stereo pairs.Experimental results proves that the proposed method outperforms
extremely well compare to other state of the art stereo methods under change in various radiometric
conditions for a given stereo pair and it is also found from the results that the execution time is less
compare to existing methods.
KEYWORDS:
Stereo matching, radiometric variation, rank transform, bilateral filter, illumination, exposure.
1. INTRODUCTION
The aim of the stereo matching is to get better 3D information about a scene just by analyzing and
finding the appropriate correspondence amongst the images that have been captured from
different viewpoints.However, finding accurate correspondence in a given image is not going to
be an easy task owedtoo many constraints like occluded regions, discontinuity in object
boundaries, textureless regions, illumination and many other problems during the matching
process. These are some of the constraints which takesa challenging task in research and also the
research is going on since many decades.Effort towards this challenging area have given a great
resultsby taking inputs from the Middlebury dataset [1], [2].
Various factors avoids the two corresponding pixel value of an image having a very same color
pixel value. Radiometric change i.e. illumination and occlusion is the major andpreventing aspect,
which includes the camera device variation for different stereo pair, lighting geometry, change in
illuminant color [3]. A distinct pixel values i.e. color values of an image can be achievedby
viewing the same scene many times as much as possible. The main purpose is that, at all times the
intensity on any point possibly will be determined by the direction and angle in a Lambertian
surface.By considering each and every point one can conclude that under the radiometric](https://image.slidesharecdn.com/6516ijfcst01-241106120356-98fe9dc6/85/COMPUTATIONALLY-EFFICIENT-TWO-STAGE-SEQUENTIAL-FRAMEWORK-FOR-STEREO-MATCHING-1-320.jpg)
![International Journal in Foundations of Computer Science & Technology (IJFCST) Vol.6, No.5, September 2016
2
variations, performance of existing stereo matching algorithms can be ruinedrigorously which can
be seen in Fig.1.
Fig.1. (a) Left image, (b) right image, (c) disparity (SAD+GC)
Keeping all of the above said points, an advancedand efficient stereo matching algorithm is
developed which is capable of taking care of radiometric variations.
2. RELATED WORK
Data cost is the stereo information which plays a vital role on deciding the performance and
quality of the output disparity [4]. Past from many decades it can be observed that most of the
existing correlation based stereo matching algorithms [5] are more efficient under the ideal
conditions but at the same time they are weak at change in radiometric variations. In [6],
evaluation of various data cost function was carried out for better stereo matching by considering
the change in radiometric conditions. On their study, local stereo algorithms like Sum of absolute
difference (SAD), Sum of Squared Differences (SSD), Normalized cross-correlation (NCC), Zero
mean sum of absolute differences (ZSAD), Sum of Hamming Distances (SHD) and many more
algorithms very compared. Hirschmuller also compared Birchfield Tomashi (BT) data cost [7],
Birchfield Tomashi with Laplacian of Gaussian (LOG) [8], Census transform and some of the
semi global algorithms too. One of the most efficient data cost was BT, Where it is very
insensitive due to camera sampling and at the same time it is also sensitive to change in
radiometric conditions as it employs linearly interpolated function of intensity values. Due to the
usage of second order derivatives, LOG filter is very insensitive in term of outliers but it is very
sensitive to change in radiometric variations. One of the popular similarity measures is NCC [9]
which is very much suitable for matching the affine transformed colored values, where it
measures the angle of cosines between the matching vectors. This NCC algorithm as well as most
of the local algorithms suffers from fattening effect which occurs across the object boundaries. In
[10] Heo et al. proposed an algorithm which was based on log chromaticity normalization and
adaptive normalized cross correlation (ANCC). The proposed method was very expensive and it
was very weak due to global variations. In [11], Yong Seok Heo, Kyoung Mu Lee, proposed a
new method which infers accuratedepth maps and color-consistent images between
radiometrically varying stereo images, simultaneously. The proposed method was very accurate
but again it was found that the execution time was more compare to other methods. In [12] Yong
Seok Heo. Proposed a new method which was based on mutual information combined with SIFT
descriptorto find correspondence for images which undergo local as well as global radiometric
variations. Even though the proposed method was quite robust to radiometric variations it failed
to overcome the execution time compare to our proposed method. The major drawback of most of
the existing stereo matching algorithm was due to usage of the raw color information. This paper
proposes a computationally efficient two stage sequential framework for stereo matching which
can handle the various radiometric variations between the stereo pairs.](https://image.slidesharecdn.com/6516ijfcst01-241106120356-98fe9dc6/85/COMPUTATIONALLY-EFFICIENT-TWO-STAGE-SEQUENTIAL-FRAMEWORK-FOR-STEREO-MATCHING-2-320.jpg)
![International Journal in Foundations of Computer Science & Technology (IJFCST) Vol.6, No.5, September 2016
3
3. PROPOSED BLOCK DIAGRAM
The proposed approach is shown in the Fig.2along with their individual constituent blocks. A
stereo pair is taken as input from Middlebury dataset and the pre-processing step is followed and
done. The complete explanation of each individual block is described and explained in the
upcoming sections.
Fig2: Generalized Proposed block diagram
3.1 RANK TRANSFORM (RT)
Almost all the stereo matching algorithms which is existing till today, a transforms which are
said to be non-parametric are used to search and find the possible pixel correspondence
surrounded by the neighbor. The most regularly used transforms is theRT and CT. RT is
susceptible to noise particularly when anarea which consist of no texture is considered, where it
sets a pixel of the matching window as a center and also it takes a pixels in some other square
neighborhood transform window.On the off chance that the middle pixel esteem not as much as
neighborhood pixel, it returns 0 and 1 generally. In this way, the arrangement of pixels in
reference image as well as target image turn into the arrangement of ordered pairs [13].
Let P represents the pixel, I (P) be the intensity pixel, N (P) represents the square neighborhood
which is of the diameter ‘d’ in turn surrounding P and E represents the exact reliance. R (P) is not
an intensity but it’s rather an integer which has the range {0, − 1} as](https://image.slidesharecdn.com/6516ijfcst01-241106120356-98fe9dc6/85/COMPUTATIONALLY-EFFICIENT-TWO-STAGE-SEQUENTIAL-FRAMEWORK-FOR-STEREO-MATCHING-3-320.jpg)
![International Journal in Foundations of Computer Science & Technology (IJFCST) Vol.6, No.5, September 2016
4
The key advantage of using this non-parametric transform called RT in stereo matching is it can
reduce the effects of diverse gains and also bias camera, which is insensible to an image noise,
further helps to improve the robustness to depth discontinuities.
3.2 NCC CORRELATION COMPUTATION
Correlation based techniques usually produces a dense map. This depth map can be obtained by
simply calculating the disparity at every individual pixel with a pre-defined 3x3 neighborhood.
The pre-defined location compared with target image in order to find a so called homologous
pixel in that specified and pre-defined window. The main idea of using this NCC is to search and
measure the pixels that are correlated within the stated range called ‘d’ and this ‘d’ ranging from
(0…dmax). NCC is the correlation algorithm which is very frequently used and this NCC helps
us to measure the so called matching contrast for different images [14, 15].
The generalized equation for an NCC is given in the equation (3).
Where ( , ) represents the left image and ( + , + ) represents the right image.
3.3 DISPARITY
The main aim of disparity is to search and identify the pixels values amongst the rectified images
for the various stereo pair. The general expression to calculate the disparity between two images
is given as
= − (4)
Where and indicates the left and right image at x- coordinate.Once the output of RT and
NCC is obtained, the disparity estimation is done for various data sets.
3.4 BILATERAL FILTER (BF)
Bilateral filtering (BT) aims at image smoothening and also preserves edge. This BF is used in
texture relighting, denoising and texture editing. It has several advantages compare to other
filters. The formulation of BT is very simple where each pixel will be replaced by the weighted
average amongst its neighbors. In this work we intend to make use of this filter for better results
and the general expression for BT is as follows
( ) = ∑ ( ) (‖ ( ) − ( )‖) !(‖ − ‖)
"#∈Ω (5)
Where is the filtered image, I is the original image, Ω represents the window centered at
x, represents the kernel range which helps in smoothening differences amongst the intensities
and ! represents the spatial kernel.](https://image.slidesharecdn.com/6516ijfcst01-241106120356-98fe9dc6/85/COMPUTATIONALLY-EFFICIENT-TWO-STAGE-SEQUENTIAL-FRAMEWORK-FOR-STEREO-MATCHING-4-320.jpg)
![International Journal in Foundations of Computer Science & Technology (IJFCST) Vol.6, No.5, September 2016
5
4. RESULTS AND DISCUSSION
In this evaluation we consider a pair of left and right stereo image as well as its ground-truth
disparity. These stereo pairs are taken from Middlebury stereo datasets [1,2]. The dmaxi.e.,
disparity range is considered as ranging from 0 to 70 pixels. The main contribution of this work
is to evaluate the robustness to change in illumination and exposure.
4.1 CHANGE IN ILLUMINATION
In more or less all the papers, most of the authors have made an assumption saying that the
corresponding values of pixels willhave the similar color values. But at the same time, the above
held statement does not hold good for the case of stereo pair having different color pixel values.
In this work, the exposure index value is set to 1 and vary theillumination index value from 1 to 3
with window size of 9x9. Fig.3 to Fig.6. Shows the output of ANCC, NCC, BT and proposed
algorithm for various stereo pair Aloe, dolls, Moebius and cloth4. Fig.7 and Fig.8 shows the
RMS and BAD pixel calculation.
Fig.3. Results of Aloe image pair by varying illumination. (a) The left image with illumination(1)-
exposure(1). (b) The right image with illumination(2)-exposure(1). (c) ground-truth (d) ANCC disparity (e)
NCC (f) BT (g) Proposed algorithm
Fig4: Results of dolls image pair by varying illumination(a) The left image with illumination (1)-exposure
(1). (b) The right image with illumination (2)-exposure (1). (c) ground-truth (d) ANCC disparity (e) NCC
(f) BT (g) Proposed algorithm](https://image.slidesharecdn.com/6516ijfcst01-241106120356-98fe9dc6/85/COMPUTATIONALLY-EFFICIENT-TWO-STAGE-SEQUENTIAL-FRAMEWORK-FOR-STEREO-MATCHING-5-320.jpg)
![International Journal in Foundations of Computer Science & Technology (IJFCST) Vol.6, No.5, September 2016
10
developed for better result and also selection of suitable window size plays an important role in
deciding the final disparity.
REFERENCE
[1] http://vision.middlebury.edu/stereo/, 2010
[2] D. Scharstein and R. Szeliski, “A Taxonomy and Evaluation of Dense Two-Frame Stereo
Correspondence Algorithms,” Int’l J.Computer Vision, vol. 47, no. 1, pp. 7-42, 2002.
[3] Y.S. Heo, K.M. Lee, and S.U. Lee, “Illumination and Camera Invariant Stereo Matching,” Proc.
IEEE Conf. Computer Vision andPattern Recognition, 2008.
[4] H. Hirschmuller and D. Scharstein, “Evaluation of stereo matching costs on images with radiometric
differences,” IEEETransactions on Pattern Analysis and Machine Intelligence,vol. 31, no. 9, pp.
1582–1599, 2009.
[5] D. Scharstein and R. Szeliski, “A taxonomy and evaluation ofdense two-frame stereo correspondence
algorithms,” Journal of Computer Vision, vol. 47, no. 1-3, pp. 7–42, 2001.
[6] Raghavendra U, KrishnamoorthiMakkithaya, Karunakar A. K,Qualitative and Quantitative
Evaluation of Correlation Based StereoMatching Algorithms, Lecture Notes in Computer Science,
SpringerBerlin/Heidelberg, International Conference on Advanced Computing,Networking and
Security, pp. 242- 252, 16th Dec 2011, NITK,Surathkal.
[7] S. Birchfield and C. Tomasi, “A Pixel Dissimilarity Measure that Is Insensitive to Image Sampling,”
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 4, pp. 401-406, Apr. 1998.
[8] H. Hirschmuller, P. Innocent, and J. Garibaldi, “Real-Time Correlation-Based Stereo Vision with
Reduced Border Errors,” Int’l J. Computer Vision, vol. 47, nos. 1-3, pp. 229-246, 2002.
[9] O. Faugeras, B. Hotz, H. Mathieu, T. Viville, Z. Zhang, P. Fua, E. Thron, and P. Robotvis, “Real
time correlationbased stereo: Algorithm, implementations and applications,” Technical Report RR-
2013 INRIA, 1996.
[10] Y. S. Heo, K. M. Lee, and S. U. Lee, “Robust stereo matchingusing adaptive normalized cross-
correlation”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 4, pp.
807–822, 2011.
[11] Yong Seok Heo, Kyoung Mu Lee, and Sang Uk Lee “Simultaneous Color Consistency and Depth
Map Estimation for Radiometrically Varying Stereo Images”, IEEE 12th International Conference
on computer vision, vol 31, no.9, pp. 1771 – 1778, 2009.
[12] Yong Seok Heo, Kyoung Mu Lee, and Sang Uk Lee “Mutual Information-based Stereo Matching
Combined with SIFT Descriptor in Log-chromaticity Color Space”, IEEE Conference on computer
vision and pattern recognition, vol 31, no.9, pp.445 – 452, 2009
[13] Scharstein, D., & Szeliski, R. (2002). A taxonomy and evaluation of dense two-frame stereo
correspondence algorithms. International Journal of Computer Vision, 47, 7–42.
[14] O. Faugeras, B. Hotz, H. Mathieu, T. Vie´ville, Z. Zhang, P. Fua, E. The´ron, L. Moll, G. Berry, J.
Vuillemin, P. Bertin, and C. Proy, “Real Time Correlation-Based Stereo: Algorithm,
Implementations and Applications,” Technical Report RR-2013, INRIA, 1993.
[15] M. Goesele, N. Snavely, B. Curless, H. Hoppe, and S.M. Seitz, “Multi-View Stereo for Community
Photo Collections,” Proc. IEEE Int’l Conf. Computer Vision, 2007.](https://image.slidesharecdn.com/6516ijfcst01-241106120356-98fe9dc6/85/COMPUTATIONALLY-EFFICIENT-TWO-STAGE-SEQUENTIAL-FRAMEWORK-FOR-STEREO-MATCHING-10-320.jpg)
COMPUTATIONALLY EFFICIENT TWO STAGE SEQUENTIAL FRAMEWORK FOR STEREO MATCHING
- 1. International Journal in Foundations of Computer Science & Technology (IJFCST) Vol.6, No.5, September 2016 DOI:10.5121/ijfcst.2016.6501 1 COMPUTATIONALLY EFFICIENT TWO STAGE SEQUENTIAL FRAMEWORK FOR STEREO MATCHING 1 Bindu N.S and 2 H.S.Sheshadri 1 Vidyavardhaka college of Engineering, Dept of E&C, Mysore, India 2 PES College of Engineering, Dept of E&C, Mandya, India ABSTRACT: Almost all the existing stereo algorithms fall under a common assumption that corresponding color or intensity values will be similar like one another. On the other hand, it is also not that true in practice where the image color or intensity values are regularly affected by different radiometric factors like illumination direction, change in image device, illuminant color and so on. For this issue, the information about the raw color of the images which is recorded by the camera should not depend on it totally, and also the common assumptions on color consistency doesn’t influence good (great) between the stereo images in real scenario. Therefore, most of the conventional stereo algorithms can be seriously degraded in terms of performance under radiometric variations. In this work, we intend to develop a new stereo matching algorithm which will be insensitive to change in radiometric conditions between stereo pairs i.e. left image as well as right image. Unlike the other stereo algorithms, we propose a computationally efficient two stage sequential framework for stereo matching which can handle the various radiometric variations between the stereo pairs.Experimental results proves that the proposed method outperforms extremely well compare to other state of the art stereo methods under change in various radiometric conditions for a given stereo pair and it is also found from the results that the execution time is less compare to existing methods. KEYWORDS: Stereo matching, radiometric variation, rank transform, bilateral filter, illumination, exposure. 1. INTRODUCTION The aim of the stereo matching is to get better 3D information about a scene just by analyzing and finding the appropriate correspondence amongst the images that have been captured from different viewpoints.However, finding accurate correspondence in a given image is not going to be an easy task owedtoo many constraints like occluded regions, discontinuity in object boundaries, textureless regions, illumination and many other problems during the matching process. These are some of the constraints which takesa challenging task in research and also the research is going on since many decades.Effort towards this challenging area have given a great resultsby taking inputs from the Middlebury dataset [1], [2]. Various factors avoids the two corresponding pixel value of an image having a very same color pixel value. Radiometric change i.e. illumination and occlusion is the major andpreventing aspect, which includes the camera device variation for different stereo pair, lighting geometry, change in illuminant color [3]. A distinct pixel values i.e. color values of an image can be achievedby viewing the same scene many times as much as possible. The main purpose is that, at all times the intensity on any point possibly will be determined by the direction and angle in a Lambertian surface.By considering each and every point one can conclude that under the radiometric
- 2. International Journal in Foundations of Computer Science & Technology (IJFCST) Vol.6, No.5, September 2016 2 variations, performance of existing stereo matching algorithms can be ruinedrigorously which can be seen in Fig.1. Fig.1. (a) Left image, (b) right image, (c) disparity (SAD+GC) Keeping all of the above said points, an advancedand efficient stereo matching algorithm is developed which is capable of taking care of radiometric variations. 2. RELATED WORK Data cost is the stereo information which plays a vital role on deciding the performance and quality of the output disparity [4]. Past from many decades it can be observed that most of the existing correlation based stereo matching algorithms [5] are more efficient under the ideal conditions but at the same time they are weak at change in radiometric variations. In [6], evaluation of various data cost function was carried out for better stereo matching by considering the change in radiometric conditions. On their study, local stereo algorithms like Sum of absolute difference (SAD), Sum of Squared Differences (SSD), Normalized cross-correlation (NCC), Zero mean sum of absolute differences (ZSAD), Sum of Hamming Distances (SHD) and many more algorithms very compared. Hirschmuller also compared Birchfield Tomashi (BT) data cost [7], Birchfield Tomashi with Laplacian of Gaussian (LOG) [8], Census transform and some of the semi global algorithms too. One of the most efficient data cost was BT, Where it is very insensitive due to camera sampling and at the same time it is also sensitive to change in radiometric conditions as it employs linearly interpolated function of intensity values. Due to the usage of second order derivatives, LOG filter is very insensitive in term of outliers but it is very sensitive to change in radiometric variations. One of the popular similarity measures is NCC [9] which is very much suitable for matching the affine transformed colored values, where it measures the angle of cosines between the matching vectors. This NCC algorithm as well as most of the local algorithms suffers from fattening effect which occurs across the object boundaries. In [10] Heo et al. proposed an algorithm which was based on log chromaticity normalization and adaptive normalized cross correlation (ANCC). The proposed method was very expensive and it was very weak due to global variations. In [11], Yong Seok Heo, Kyoung Mu Lee, proposed a new method which infers accuratedepth maps and color-consistent images between radiometrically varying stereo images, simultaneously. The proposed method was very accurate but again it was found that the execution time was more compare to other methods. In [12] Yong Seok Heo. Proposed a new method which was based on mutual information combined with SIFT descriptorto find correspondence for images which undergo local as well as global radiometric variations. Even though the proposed method was quite robust to radiometric variations it failed to overcome the execution time compare to our proposed method. The major drawback of most of the existing stereo matching algorithm was due to usage of the raw color information. This paper proposes a computationally efficient two stage sequential framework for stereo matching which can handle the various radiometric variations between the stereo pairs.
- 3. International Journal in Foundations of Computer Science & Technology (IJFCST) Vol.6, No.5, September 2016 3 3. PROPOSED BLOCK DIAGRAM The proposed approach is shown in the Fig.2along with their individual constituent blocks. A stereo pair is taken as input from Middlebury dataset and the pre-processing step is followed and done. The complete explanation of each individual block is described and explained in the upcoming sections. Fig2: Generalized Proposed block diagram 3.1 RANK TRANSFORM (RT) Almost all the stereo matching algorithms which is existing till today, a transforms which are said to be non-parametric are used to search and find the possible pixel correspondence surrounded by the neighbor. The most regularly used transforms is theRT and CT. RT is susceptible to noise particularly when anarea which consist of no texture is considered, where it sets a pixel of the matching window as a center and also it takes a pixels in some other square neighborhood transform window.On the off chance that the middle pixel esteem not as much as neighborhood pixel, it returns 0 and 1 generally. In this way, the arrangement of pixels in reference image as well as target image turn into the arrangement of ordered pairs [13]. Let P represents the pixel, I (P) be the intensity pixel, N (P) represents the square neighborhood which is of the diameter ‘d’ in turn surrounding P and E represents the exact reliance. R (P) is not an intensity but it’s rather an integer which has the range {0, − 1} as
- 4. International Journal in Foundations of Computer Science & Technology (IJFCST) Vol.6, No.5, September 2016 4 The key advantage of using this non-parametric transform called RT in stereo matching is it can reduce the effects of diverse gains and also bias camera, which is insensible to an image noise, further helps to improve the robustness to depth discontinuities. 3.2 NCC CORRELATION COMPUTATION Correlation based techniques usually produces a dense map. This depth map can be obtained by simply calculating the disparity at every individual pixel with a pre-defined 3x3 neighborhood. The pre-defined location compared with target image in order to find a so called homologous pixel in that specified and pre-defined window. The main idea of using this NCC is to search and measure the pixels that are correlated within the stated range called ‘d’ and this ‘d’ ranging from (0…dmax). NCC is the correlation algorithm which is very frequently used and this NCC helps us to measure the so called matching contrast for different images [14, 15]. The generalized equation for an NCC is given in the equation (3). Where ( , ) represents the left image and ( + , + ) represents the right image. 3.3 DISPARITY The main aim of disparity is to search and identify the pixels values amongst the rectified images for the various stereo pair. The general expression to calculate the disparity between two images is given as = − (4) Where and indicates the left and right image at x- coordinate.Once the output of RT and NCC is obtained, the disparity estimation is done for various data sets. 3.4 BILATERAL FILTER (BF) Bilateral filtering (BT) aims at image smoothening and also preserves edge. This BF is used in texture relighting, denoising and texture editing. It has several advantages compare to other filters. The formulation of BT is very simple where each pixel will be replaced by the weighted average amongst its neighbors. In this work we intend to make use of this filter for better results and the general expression for BT is as follows ( ) = ∑ ( ) (‖ ( ) − ( )‖) !(‖ − ‖) "#∈Ω (5) Where is the filtered image, I is the original image, Ω represents the window centered at x, represents the kernel range which helps in smoothening differences amongst the intensities and ! represents the spatial kernel.
- 5. International Journal in Foundations of Computer Science & Technology (IJFCST) Vol.6, No.5, September 2016 5 4. RESULTS AND DISCUSSION In this evaluation we consider a pair of left and right stereo image as well as its ground-truth disparity. These stereo pairs are taken from Middlebury stereo datasets [1,2]. The dmaxi.e., disparity range is considered as ranging from 0 to 70 pixels. The main contribution of this work is to evaluate the robustness to change in illumination and exposure. 4.1 CHANGE IN ILLUMINATION In more or less all the papers, most of the authors have made an assumption saying that the corresponding values of pixels willhave the similar color values. But at the same time, the above held statement does not hold good for the case of stereo pair having different color pixel values. In this work, the exposure index value is set to 1 and vary theillumination index value from 1 to 3 with window size of 9x9. Fig.3 to Fig.6. Shows the output of ANCC, NCC, BT and proposed algorithm for various stereo pair Aloe, dolls, Moebius and cloth4. Fig.7 and Fig.8 shows the RMS and BAD pixel calculation. Fig.3. Results of Aloe image pair by varying illumination. (a) The left image with illumination(1)- exposure(1). (b) The right image with illumination(2)-exposure(1). (c) ground-truth (d) ANCC disparity (e) NCC (f) BT (g) Proposed algorithm Fig4: Results of dolls image pair by varying illumination(a) The left image with illumination (1)-exposure (1). (b) The right image with illumination (2)-exposure (1). (c) ground-truth (d) ANCC disparity (e) NCC (f) BT (g) Proposed algorithm
- 6. International Journal in Foundations of Computer Science & Technology (IJFCST) Vol.6, No.5, September 2016 6 Fig5: Results of Moebius image pair by varying illumination. (a) The left image with illumination (1)- exposure (1). (b) The right image with illumination (2)-exposure (1). (c) ground-truth (d) ANCC disparity (e) NCC (f) BT (g) Proposed algorithm Fig.6: Results of cloth4 image pair by varying illumination. (a) The left image with illumination (1)- exposure (1). (b) The right image with illumination (2)-exposure (1). (c) ground-truth (d) ANCC disparity (e) BT (f) Proposed algorithm (g) NCC (a) (b) (c) (d) (e) Fig.7: RMS calculationfor various stereo pairs with vary in illumination. 0 100 1,1 1,2 1,3 2,2 2,3 3,3 Dolls Proposed BT NCC 0 50 1,1 1,2 1,3 2,2 2,3 3,3 Art Proposed BT NCC ANCC 0 100 1,1 1,2 1,3 2,2 2,3 3,3 Moebius Proposed BT NCC ANCC 0 50 1,1 1,2 1,3 2,2 2,3 3,3 Aloe Proposed BT NCC ANCC 0 50 1,1 1,2 1,3 2,2 2,3 3,3 Cloth4 ANCC BT NCC Proposed
- 7. International Journal in Foundations of Computer Science & Technology (IJFCST) Vol.6, No.5, September 2016 7 (a) (b) (c) (d) (e) Fig.8: Bad pixel calculation for various stereopairs with vary in illumination 4.2 CHANGE IN EXPOSURE In this work, the illumination index value is set to 2 and vary theexposure index value from 0 to 2 with window size of 9x9. Fig.9 to Fig.12. Shows the output of ANCC, NCC, BT and proposed algorithm for various stereo pair Aloe, dolls, Moebius and cloth4. Fig.13 and Fig.14 shows the RMS and BAD pixel calculation. (a) (b) (c) (d) (e) (f) (g) Fig.9: Results of Aloe image pair by varyingexposure (a) The left image with illumination (2)-exposure(1). (b) The right image with illumination(2)-exposure (2). (c) ground-truth (d) ANCC disparity (e) NCC (f) BT (g) Proposed algorithm 0 2 1,1 1,2 1,3 2,2 2,3 3,3 Dolls Proposed ANCC NCC 0 2 1,1 1,2 1,3 2,2 2,3 3,3 Aloe Proposed ANCC NCC BT 0 1 2 1,1 1,2 1,3 2,2 2,3 3,3 Cloth4 Proposed ANCC NCC BT 0 2 4 1,1 1,2 1,3 2,2 2,3 3,3 Art Proposed NCC BT ANCC 0 2 1,1 1,2 1,3 2,2 2,3 3,3 Moebius Proposed ANCC NCC BT
- 8. International Journal in Foundations of Computer Science & Technology (IJFCST) Vol.6, No.5, September 2016 8 Fig.10: Results of doll image pair by varyingexposure. (a) The left image with illumination (2)- exposure(1). (b) The right image with illumination(2)-exposure (2). (c) ground-truth (d) ANCC disparity (e) BT (f) NCC (g) Proposed algorithm Fig.11: Results of Moebius image pair by varyingexposure(a) The left image with illumination (2)- exposure(1). (b) The right image with illumination(2)-exposure (2). (c) ground-truth (d) BT disparity (e) ANCC (f) NCC (g) Proposed algorithm Fig.12: Results of Cloth4 image pair by varyingexposure. (a) The left image with illumination (2)- exposure(1). (b) The right image with illumination(2)-exposure (2). (c) ground truth (d) BT disparity (e) ANCC (f) NCC (g) Proposed algorithm (a) (b) (c) 0 100 1,1 1,2 1,3 2,2 2,3 3,3 Cloth4 ANCC BT NCC 0 50 100 1,1 1,2 1,3 2,2 2,3 3,3 Mobieus ANCC NCC BT Proposed 0 50 100 1,1 1,2 1,3 2,2 2,3 3,3 Dolls ANCC NCC BT Proposed
- 9. International Journal in Foundations of Computer Science & Technology (IJFCST) Vol.6, No.5, September 2016 9 (d) (e) Fig.13: RMS calculation for variousstereo pairs with vary in exposure. (a) (b) (c) (d) (e) Fig.14: Bad pixel calculation for variousstereo pairs with vary in exposure.. CONCLUSION As far as real time applications are considered, the correlation based stereo algorithms play an important roleas it is one of the most popular and very efficient and effective algorithms. In this work we have considered 3 correlation based stereo algorithms namely ANCC, NCC and BT and also analysis has been done considering various parameters. It is then compared with the proposed algorithm by using a window size of 9. The evaluation is done towards the robustness to change in illumination and exposure in our algorithm. It is also very important and necessary to get the better results due to radiometric changes.As a key solution to many of the above stated problems, an efficient hybrid correlation of rank transform, NCC followed by bilateral filter is 0 50 1,1 1,2 1,3 2,2 2,3 3,3 Aloe ANCC NCC BT Proposed 0 100 1,1 1,2 1,3 2,2 2,3 3,3 Art NCC BT Proposed ANCC 0 2 1,1 1,2 1,3 2,2 2,3 3,3 Cloth4 ANCC NCC BT 0 1 2 1,1 1,2 1,3 2,2 2,3 3,3 Mobieus ANCC NCC BT Proposed 0 2 1,1 1,2 1,3 2,2 2,3 3,3 Dolls ANCC NCC BT Proposed 0 2 1,1 1,2 1,3 2,2 2,3 3,3 Aloe ANCC NCC BT 0 5 1,1 1,2 1,3 2,2 2,3 3,3 Art ANCC NCC BT
- 10. International Journal in Foundations of Computer Science & Technology (IJFCST) Vol.6, No.5, September 2016 10 developed for better result and also selection of suitable window size plays an important role in deciding the final disparity. REFERENCE [1] http://vision.middlebury.edu/stereo/, 2010 [2] D. Scharstein and R. Szeliski, “A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms,” Int’l J.Computer Vision, vol. 47, no. 1, pp. 7-42, 2002. [3] Y.S. Heo, K.M. Lee, and S.U. Lee, “Illumination and Camera Invariant Stereo Matching,” Proc. IEEE Conf. Computer Vision andPattern Recognition, 2008. [4] H. Hirschmuller and D. Scharstein, “Evaluation of stereo matching costs on images with radiometric differences,” IEEETransactions on Pattern Analysis and Machine Intelligence,vol. 31, no. 9, pp. 1582–1599, 2009. [5] D. Scharstein and R. Szeliski, “A taxonomy and evaluation ofdense two-frame stereo correspondence algorithms,” Journal of Computer Vision, vol. 47, no. 1-3, pp. 7–42, 2001. [6] Raghavendra U, KrishnamoorthiMakkithaya, Karunakar A. K,Qualitative and Quantitative Evaluation of Correlation Based StereoMatching Algorithms, Lecture Notes in Computer Science, SpringerBerlin/Heidelberg, International Conference on Advanced Computing,Networking and Security, pp. 242- 252, 16th Dec 2011, NITK,Surathkal. [7] S. Birchfield and C. Tomasi, “A Pixel Dissimilarity Measure that Is Insensitive to Image Sampling,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 4, pp. 401-406, Apr. 1998. [8] H. Hirschmuller, P. Innocent, and J. Garibaldi, “Real-Time Correlation-Based Stereo Vision with Reduced Border Errors,” Int’l J. Computer Vision, vol. 47, nos. 1-3, pp. 229-246, 2002. [9] O. Faugeras, B. Hotz, H. Mathieu, T. Viville, Z. Zhang, P. Fua, E. Thron, and P. Robotvis, “Real time correlationbased stereo: Algorithm, implementations and applications,” Technical Report RR- 2013 INRIA, 1996. [10] Y. S. Heo, K. M. Lee, and S. U. Lee, “Robust stereo matchingusing adaptive normalized cross- correlation”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 4, pp. 807–822, 2011. [11] Yong Seok Heo, Kyoung Mu Lee, and Sang Uk Lee “Simultaneous Color Consistency and Depth Map Estimation for Radiometrically Varying Stereo Images”, IEEE 12th International Conference on computer vision, vol 31, no.9, pp. 1771 – 1778, 2009. [12] Yong Seok Heo, Kyoung Mu Lee, and Sang Uk Lee “Mutual Information-based Stereo Matching Combined with SIFT Descriptor in Log-chromaticity Color Space”, IEEE Conference on computer vision and pattern recognition, vol 31, no.9, pp.445 – 452, 2009 [13] Scharstein, D., & Szeliski, R. (2002). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47, 7–42. [14] O. Faugeras, B. Hotz, H. Mathieu, T. Vie´ville, Z. Zhang, P. Fua, E. The´ron, L. Moll, G. Berry, J. Vuillemin, P. Bertin, and C. Proy, “Real Time Correlation-Based Stereo: Algorithm, Implementations and Applications,” Technical Report RR-2013, INRIA, 1993. [15] M. Goesele, N. Snavely, B. Curless, H. Hoppe, and S.M. Seitz, “Multi-View Stereo for Community Photo Collections,” Proc. IEEE Int’l Conf. Computer Vision, 2007.