![Sundarapandian et al. (Eds): CoNeCo,WiMo, NLP, CRYPSIS, ICAIT, ICDIP, ITCSE, CS & IT 07,
pp. 293–296, 2012. © CS & IT-CSCP 2012 DOI : 10.5121/csit.2012.2427
EXTENDED WAVELET TRANSFORM BASED
IMAGE INPAINTING ALGORITHM FOR
NATURAL SCENE IMAGE COMPLETION
K.Sangeetha1
, Dr.P.Sengottuvelan2
and E.Balamurugan3
1
Department of Computer Applications, BIT, Sathyamangalam, India
kavigeeth@gmail.com
2
Department of Information Technology, BIT, Sathyamangalam, India
psengottuvelan@rediffmail.com
3
Department of Computer Applications, BIT, Sathyamangalam, India
sanbala@rediffmail.com
ABSTRACT
This paper proposes an exemplar based image inpainting using extended wavelet transform. The
Image inpainting modifies an image with the available information outside the region to be
inpainted in an undetectable way. The extended wavelet transform is in two dimensions. The
Laplacian pyramid is first used to capture the point discontinuities, and then followed by a
directional filter bank to link point discontinuities into linear structures. The proposed model
effectively captures the edges and contours of natural scene images.
KEYWORDS
Extended Wavelet Transform, Exemplar Based, Image Inpainting
1. INTRODUCTION
The Image Inpainting is the art of modifying an image in a form that is not easily detectable by
an ordinary observer. The aim of algorithms evaluated in this paper is creating a visually pleasing
continuation of data around the hole in such a way that it is not detectable. In order to effectively
retain image data, various researchers have continually proposed various methods of image
inpainting and these works are classified into 2 major categories. One is non exemplar based
method and the other is exemplar based method.
Art repairers use their knowledge of the world and the abilities of the brain to complete missing
parts of something as the connectivity principle [2-3] and allow to "see" the missing parts of an
image and complete it in such way that one could think that nothing was ever missing. In our
model, uses the same approach with the least possible user interaction; only an inpainting area
must be informed by the user, meaning that the selection of the data that will fill the image is done
automatically.
2. RELATED WORK
In 2000 Bertalmio et al. [1] introduced the first aspects of digital inpainting, providing an
efficient way to routinely fill the target area. The technique gathers the information to complete
the inpainting region by applying a dispersal scheme based on partial differential equations
(PDEs) over the boundary of the area to be filled. The core of this method is to transmit](https://image.slidesharecdn.com/csit2427-180205121820/85/EXTENDED-WAVELET-TRANSFORM-BASED-IMAGE-INPAINTING-ALGORITHM-FOR-NATURAL-SCENE-IMAGE-COMPLETION-1-320.jpg)
![294 Computer Science & Information Technology ( CS & IT )
isophotes into the inpainting region, alternating with anisotropic diffusion for directional
smoothing. More recently, Tschumperl´e and Deriche [4] proposed a unified approach for image
restoration, object removal and resolution improvement also based on PDEs. In their paper, the
authors present a powerful mechanism for digital inpainting based on image regularization
through vector fields. Although both methods presented good results for relatively simple regions
they failed to complete larger texturized regions. The need to fill larger inpainting regions
provoked the development of techniques that propagate blocks of pixels per iteration instead of
isolated pixels.
The evolution of Bertalmio's work shown in [5] describes a mechanism that splits the image
information in texture and structure, supported by the methods devised in [5, 6]. The structured
part of the image is processed with an inpainting algorithm, and the texture is synthesized using
Efros' method [8]. Criminisi et al. [9] developed important work in digital inpainting with texture
synthesis. In their work, the authors state that exemplar-based texture synthesis suffices in order
to fill large inpainting regions. Criminisi's method estimates the gradient vector of the image and
a confidence term in order to determine the block that should be processed first at each iteration,
aiming to preserve both image structure and texture.
The wavelet transform has been widely used in the medical image processing [7]. The wavelet
transform is a type of multi-scale analysis that decomposes input signal into high frequency detail
and low frequency approximation components at various resolutions. To enhance features, the
selected detail wavelet coefficients are multiplied by an adaptive gain value. The image is then
enhanced by reconstructing the processed wavelet coefficients.
Do and Vetterli (10) utilized a double filter banks structure to develop the Contourlet transform
and used it for some nonlinear approximation and de-noising experiments and obtained some
hopeful results. In this work, a new approach for retinal image contrast enhancement that is
based on Contourlet transform is proposed. The main reason for the choice of Contourlet is
based on its better performance of representing edges and textures of natural images. The
proposed model achieves better visual results and outperformed the previous methods.
3. THE PROPOSED MODEL
3.1 Contourlet Transform
Contourlet Transform consists of two steps: the sub bands decomposition and the directional
transform. A Laplacian Pyramid is first used to capture point discontinuities, then followed by a
directional filter bank to link point discontinuity into linear structure. The overall result is an
image expansion using basic elements like contour segments.
Contourlet transform is well-adapted to represent images containing edges; it is a good candidate
for edge enhancement in natural images. Contourlet coefficients can be modified via a nonlinear
function ya. Taking noise into consideration, will introduce explicitly a noise standard deviation
σ in the equation.
if σ≤x<2ασ
if x≥t (1)](https://image.slidesharecdn.com/csit2427-180205121820/85/EXTENDED-WAVELET-TRANSFORM-BASED-IMAGE-INPAINTING-ALGORITHM-FOR-NATURAL-SCENE-IMAGE-COMPLETION-2-320.jpg)
![Computer Science & Information Technology ( CS & IT ) 295
Here, t determines the degree of nonlinearity and s introduces a dynamic range compression.
Using a nonzero s will enhance the faintest edges and soften the strongest edges. α is a
normalization parameter. The t parameter is the value under which coefficients are amplified. This value
depends obviously on the pixel values. It can derive the t value from the data.
4. RESULTS
We have experimented with the method [9] and our proposed model on some images comparing
with PSNR. These algorithms are programmed by matlab2008Ra and all experiments are run on
a 2.93GHz PC. The algorithm proposed by [9] succeeds in filling the target region without
implicit or explicit segmentation. The proposed model performs well as previous techniques
designed for the restoration of small scratches, and, in instances in which larger objects are
removed.
Figure 1 - 3 show the results produced by the aforementioned exemplar based Inpainting
algorithms. The images in each figure are arranged as original image, an image with occluded
region, the final result of methods in [9] and proposed model respectively.
Figure 1. Lady occluded region of image. (a) Original Image. (b)Mask image. (c) The output
image by Criminisi et al's algorithm [9]. (d) The output image by the proposed model.
(a) (b) (c) (d)
Figure 2. Man occluded region of image. (a) Original Image. (b)Mask image. (c) The output image
by Criminisi et al's algorithm [9]. (d) The output image by the proposed model
(a) (b) (c) (d)
Figure 3. Bird occluded region of image. (a) Original Image. (b)Mask image. (c) The output image
by Criminisi et al's algorithm [9]. (d) The output image by the proposed model](https://image.slidesharecdn.com/csit2427-180205121820/85/EXTENDED-WAVELET-TRANSFORM-BASED-IMAGE-INPAINTING-ALGORITHM-FOR-NATURAL-SCENE-IMAGE-COMPLETION-3-320.jpg)
![296 Computer Science & Information Technology ( CS & IT )
(a) (b) (c) (d)
Table 1 PSNR value for the two exemplar based image inpainting Results
Image
Lady occluded
image
Man occluded
image
Bird occluded
imageMethod
Criminisi et al's 33.12 33.16 33.22
Proposed model 36.29 35.89 36.53
5. CONCLUSIONS
With the exemplar- based texture synthesis, the proposed image restoration method can restore
structure features and composite textures both for large and thick or long and thin blackened
regions without blurring. As evidenced by the experiments with the Contourlet transform, there
is better preservation of contours than with other methods. The Contourlet can detect the
contours and edges quite adequately. We have planned to work on methods to restore complex
structures such as corners, curves etc.
REFERENCES
[1] Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proceedings of
SIGGRAPH 2000, pp. 417-424. ACM Press, New York (2000)
[2] Kokaram, A., Morris, R., Fitzgerald, W.,Rayner, P.: Interpolation of missing data in image sequences.
IEEE Trans. Image Processing 11, 1509-1519 (1995)
[3] Pessoa, L., Thompson, E., Noe, A.: Finding out about filling-in: A guide to perceptual completion for
visual science and the philosophy of perception. Behavioral and Brain Sciences 21(6), 723-
748(1998)
[4] Tschumperle, D., Deriche, R.: Vector-valued image regularization with PDEs: A common framework
for different applications. IEEE Trans. Pattern Anal. Mach. Intell. 27(4), 506- 517 (2005).
[5] Bertalmio, M., Vese, L., Sapiro, G., Osher, S.: Simultaneous structure and texture image inpainting
IEEE Trans. Image Processing 12, 882-889 (2003). Meyer, Y.: Oscillating Patterns in Image
Processing and Nonlinear Evolution Equations. American Mathematical Society, Boston (2001).
[6] Vese, L.A., Osher, S.J.: Modeling textures with total variation minimization and oscillating patterns
in image processing. J. Sci. Comput. 19(1-3), 553-572 (2003).
[7] Fu, J.C., Chai, J.W., Wong, S.T.C., 2000a. Wavelet-based enhancement for detection of left
ventricular myocardial boundaries in magnetic resonance images. Magn. Reson. Imaging 18 (9),
1135-1141.
[8] Efros, A.A., Leung, T.K.: Texture synthesis by non-parametric sampling. In: IEEE International
Conference on Computer Vision, pp. 1033-1038. Corfu, Greece (1999).
[9] Criminisi, A., Perez, P., Toyama, K.: Region filling and object removal by exemplar based image
inpainting. IEEE Trans. Image Processing 13(9), 1200-1212 (2004)](https://image.slidesharecdn.com/csit2427-180205121820/85/EXTENDED-WAVELET-TRANSFORM-BASED-IMAGE-INPAINTING-ALGORITHM-FOR-NATURAL-SCENE-IMAGE-COMPLETION-4-320.jpg)
EXTENDED WAVELET TRANSFORM BASED IMAGE INPAINTING ALGORITHM FOR NATURAL SCENE IMAGE COMPLETION
- 1. Sundarapandian et al. (Eds): CoNeCo,WiMo, NLP, CRYPSIS, ICAIT, ICDIP, ITCSE, CS & IT 07, pp. 293–296, 2012. © CS & IT-CSCP 2012 DOI : 10.5121/csit.2012.2427 EXTENDED WAVELET TRANSFORM BASED IMAGE INPAINTING ALGORITHM FOR NATURAL SCENE IMAGE COMPLETION K.Sangeetha1 , Dr.P.Sengottuvelan2 and E.Balamurugan3 1 Department of Computer Applications, BIT, Sathyamangalam, India [email protected] 2 Department of Information Technology, BIT, Sathyamangalam, India [email protected] 3 Department of Computer Applications, BIT, Sathyamangalam, India [email protected] ABSTRACT This paper proposes an exemplar based image inpainting using extended wavelet transform. The Image inpainting modifies an image with the available information outside the region to be inpainted in an undetectable way. The extended wavelet transform is in two dimensions. The Laplacian pyramid is first used to capture the point discontinuities, and then followed by a directional filter bank to link point discontinuities into linear structures. The proposed model effectively captures the edges and contours of natural scene images. KEYWORDS Extended Wavelet Transform, Exemplar Based, Image Inpainting 1. INTRODUCTION The Image Inpainting is the art of modifying an image in a form that is not easily detectable by an ordinary observer. The aim of algorithms evaluated in this paper is creating a visually pleasing continuation of data around the hole in such a way that it is not detectable. In order to effectively retain image data, various researchers have continually proposed various methods of image inpainting and these works are classified into 2 major categories. One is non exemplar based method and the other is exemplar based method. Art repairers use their knowledge of the world and the abilities of the brain to complete missing parts of something as the connectivity principle [2-3] and allow to "see" the missing parts of an image and complete it in such way that one could think that nothing was ever missing. In our model, uses the same approach with the least possible user interaction; only an inpainting area must be informed by the user, meaning that the selection of the data that will fill the image is done automatically. 2. RELATED WORK In 2000 Bertalmio et al. [1] introduced the first aspects of digital inpainting, providing an efficient way to routinely fill the target area. The technique gathers the information to complete the inpainting region by applying a dispersal scheme based on partial differential equations (PDEs) over the boundary of the area to be filled. The core of this method is to transmit
- 2. 294 Computer Science & Information Technology ( CS & IT ) isophotes into the inpainting region, alternating with anisotropic diffusion for directional smoothing. More recently, Tschumperl´e and Deriche [4] proposed a unified approach for image restoration, object removal and resolution improvement also based on PDEs. In their paper, the authors present a powerful mechanism for digital inpainting based on image regularization through vector fields. Although both methods presented good results for relatively simple regions they failed to complete larger texturized regions. The need to fill larger inpainting regions provoked the development of techniques that propagate blocks of pixels per iteration instead of isolated pixels. The evolution of Bertalmio's work shown in [5] describes a mechanism that splits the image information in texture and structure, supported by the methods devised in [5, 6]. The structured part of the image is processed with an inpainting algorithm, and the texture is synthesized using Efros' method [8]. Criminisi et al. [9] developed important work in digital inpainting with texture synthesis. In their work, the authors state that exemplar-based texture synthesis suffices in order to fill large inpainting regions. Criminisi's method estimates the gradient vector of the image and a confidence term in order to determine the block that should be processed first at each iteration, aiming to preserve both image structure and texture. The wavelet transform has been widely used in the medical image processing [7]. The wavelet transform is a type of multi-scale analysis that decomposes input signal into high frequency detail and low frequency approximation components at various resolutions. To enhance features, the selected detail wavelet coefficients are multiplied by an adaptive gain value. The image is then enhanced by reconstructing the processed wavelet coefficients. Do and Vetterli (10) utilized a double filter banks structure to develop the Contourlet transform and used it for some nonlinear approximation and de-noising experiments and obtained some hopeful results. In this work, a new approach for retinal image contrast enhancement that is based on Contourlet transform is proposed. The main reason for the choice of Contourlet is based on its better performance of representing edges and textures of natural images. The proposed model achieves better visual results and outperformed the previous methods. 3. THE PROPOSED MODEL 3.1 Contourlet Transform Contourlet Transform consists of two steps: the sub bands decomposition and the directional transform. A Laplacian Pyramid is first used to capture point discontinuities, then followed by a directional filter bank to link point discontinuity into linear structure. The overall result is an image expansion using basic elements like contour segments. Contourlet transform is well-adapted to represent images containing edges; it is a good candidate for edge enhancement in natural images. Contourlet coefficients can be modified via a nonlinear function ya. Taking noise into consideration, will introduce explicitly a noise standard deviation σ in the equation. if σ≤x<2ασ if x≥t (1)
- 3. Computer Science & Information Technology ( CS & IT ) 295 Here, t determines the degree of nonlinearity and s introduces a dynamic range compression. Using a nonzero s will enhance the faintest edges and soften the strongest edges. α is a normalization parameter. The t parameter is the value under which coefficients are amplified. This value depends obviously on the pixel values. It can derive the t value from the data. 4. RESULTS We have experimented with the method [9] and our proposed model on some images comparing with PSNR. These algorithms are programmed by matlab2008Ra and all experiments are run on a 2.93GHz PC. The algorithm proposed by [9] succeeds in filling the target region without implicit or explicit segmentation. The proposed model performs well as previous techniques designed for the restoration of small scratches, and, in instances in which larger objects are removed. Figure 1 - 3 show the results produced by the aforementioned exemplar based Inpainting algorithms. The images in each figure are arranged as original image, an image with occluded region, the final result of methods in [9] and proposed model respectively. Figure 1. Lady occluded region of image. (a) Original Image. (b)Mask image. (c) The output image by Criminisi et al's algorithm [9]. (d) The output image by the proposed model. (a) (b) (c) (d) Figure 2. Man occluded region of image. (a) Original Image. (b)Mask image. (c) The output image by Criminisi et al's algorithm [9]. (d) The output image by the proposed model (a) (b) (c) (d) Figure 3. Bird occluded region of image. (a) Original Image. (b)Mask image. (c) The output image by Criminisi et al's algorithm [9]. (d) The output image by the proposed model
- 4. 296 Computer Science & Information Technology ( CS & IT ) (a) (b) (c) (d) Table 1 PSNR value for the two exemplar based image inpainting Results Image Lady occluded image Man occluded image Bird occluded imageMethod Criminisi et al's 33.12 33.16 33.22 Proposed model 36.29 35.89 36.53 5. CONCLUSIONS With the exemplar- based texture synthesis, the proposed image restoration method can restore structure features and composite textures both for large and thick or long and thin blackened regions without blurring. As evidenced by the experiments with the Contourlet transform, there is better preservation of contours than with other methods. The Contourlet can detect the contours and edges quite adequately. We have planned to work on methods to restore complex structures such as corners, curves etc. REFERENCES [1] Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proceedings of SIGGRAPH 2000, pp. 417-424. ACM Press, New York (2000) [2] Kokaram, A., Morris, R., Fitzgerald, W.,Rayner, P.: Interpolation of missing data in image sequences. IEEE Trans. Image Processing 11, 1509-1519 (1995) [3] Pessoa, L., Thompson, E., Noe, A.: Finding out about filling-in: A guide to perceptual completion for visual science and the philosophy of perception. Behavioral and Brain Sciences 21(6), 723- 748(1998) [4] Tschumperle, D., Deriche, R.: Vector-valued image regularization with PDEs: A common framework for different applications. IEEE Trans. Pattern Anal. Mach. Intell. 27(4), 506- 517 (2005). [5] Bertalmio, M., Vese, L., Sapiro, G., Osher, S.: Simultaneous structure and texture image inpainting IEEE Trans. Image Processing 12, 882-889 (2003). Meyer, Y.: Oscillating Patterns in Image Processing and Nonlinear Evolution Equations. American Mathematical Society, Boston (2001). [6] Vese, L.A., Osher, S.J.: Modeling textures with total variation minimization and oscillating patterns in image processing. J. Sci. Comput. 19(1-3), 553-572 (2003). [7] Fu, J.C., Chai, J.W., Wong, S.T.C., 2000a. Wavelet-based enhancement for detection of left ventricular myocardial boundaries in magnetic resonance images. Magn. Reson. Imaging 18 (9), 1135-1141. [8] Efros, A.A., Leung, T.K.: Texture synthesis by non-parametric sampling. In: IEEE International Conference on Computer Vision, pp. 1033-1038. Corfu, Greece (1999). [9] Criminisi, A., Perez, P., Toyama, K.: Region filling and object removal by exemplar based image inpainting. IEEE Trans. Image Processing 13(9), 1200-1212 (2004)