Local binary pattern
3 likes8,223 views
The document outlines a report on the local binary pattern (LBP) algorithm used in digital image processing, detailing its steps and code for calculating LBP from images. It then discusses the proposed local binary pattern histogram Fourier features (LBP-HF), which provide a rotation-invariant image descriptor that maintains the discriminative nature of LBP histograms for various applications like texture classification and face recognition. The LBP-HF approach asserts advantages over previous methods by computing invariant features globally, preserving the relative distribution of local orientations, and demonstrating effective performance in empirical tests.
Local binary pattern
- 1. Report on Local Binary Pattern Subject Digital Image Processing Submitted To Dr. Ayyaz Class MSCS-F14 Prepared By: Sunawar Khan Mehwish Shabbir
- 2. Algorithm to calculate Local Binary Pattern Steps: 1- Divide the examined window into cells (e.g. 16x16 pixels for each cell). 2- For each pixel in a cell, compare the pixel to each of its 8 neighbors (on its left-top, left- middle, left-bottom, right-top, etc.). Follow the pixels along a circle, i.e. clockwise or counter- clockwise. 3- Where the center pixel's value is greater than the neighbor's value, write "1". Otherwise, write "0". This gives an 8-digit binary number (which is usually converted to decimal for convenience). 4- Compute the histogram, over the cell, of the frequency of each "number" occurring (i.e., each combination of which pixels are smaller and which are greater than the center). 5- Optionally normalize the histogram. 6- Concatenate (normalized) histograms of all cells. This gives the feature vector for the window. Formula/Expression:
- 3. Code Studied for calculation of LBP: function LBP_Im = LBP(Input_Im, R) % %======================================================================= % %======================================================================= % This function computes the LBP transformation of the input image % Input_Im. % The parameters of the LBP operator are (P = 8, R), % where P - the number of sampling points in the region with the radius R. % Radius R is the input parameter of the function. % Possible values for R = 1, 2, 3, etc. % If input image is COLOR, then the grayscale transformation is performed. % %======================================================================= % %======================================================================= if size(Input_Im, 3) == 3 Input_Im = rgb2gray(Input_Im); end; L = 2*R + 1; %% The size of the LBP label C = round(L/2); Input_Im = uint8(Input_Im); row_max = size(Input_Im,1)-L+1; col_max = size(Input_Im,2)-L+1; LBP_Im = zeros(row_max, col_max); for i = 1:row_max for j = 1:col_max A = Input_Im(i:i+L-1, j:j+L-1); A = A+1-A(C,C); A(A>0) = 1; LBP_Im(i,j) = A(C,L) + A(L,L)*2 + A(L,C)*4 + A(L,1)*8 + A(C,1)*16 + A(1,1)*32 + A(1,C)*64 + A(1,L)*128; end; end;
- 4. Report on Paper Studied: Title: Rotation Invariant Image Description with LocalBinary Pattern Histogram Fourier Features Authors: Timo Ahonen, Jiˇr´ı Matas, Chu He, and Matti Pietik¨ainen Technique Used: Rotation Invariant Local Binary Pattern Descriptors Base Technique: The Local Binary Pattern Operator Abstract: In this paper, we propose Local Binary Pattern Histogram Fourier features (LBP-HF), a novel rotation invariant image descriptor computed from discrete Fourier transforms of local binary pattern (LBP) histograms. Unlike most other histogram based invariant texture descriptors which normalize rotation locally, the proposed invariants are constructed globally for the whole region to be described. In addition to being rotation invariant, the LBP-HF features retain the highly discriminative nature of LBP histograms. In the experiments, it is shown that these features outperform non-invariant and earlier version of rotation invariant LBP and the MR8 descriptor in texture classification, material categorization and face recognition tests.
- 5. Introduction Rotation invariant texture analysis is a widely studied problem. It aims at providing with texture features that are invariant to rotation angle of the input texture image. Moreover, these features should typically be robust also to image formation conditions such as illumination changes. Describing the appearance locally, e.g., using co-occurrences of gray values or with filter bank responses and then forming a global description by computing statistics over the image region is a well-established technique in texture analysis. This approach has been extended by several authors to produce rotation invariant features by transforming each local descriptor to a canonical representation invariant to rotations of the input image. The statistics describing the whole region are then computed from these transformed local descriptors. Even though such approaches have produced good results in rotation invariant texture classification, they have some weaknesses. Most importantly, as each local descriptor (e.g., filter bank response) is transformed to canonical representation independently, the relative distribution of different orientations is lost. Furthermore, as the transformation needs to be performed for each text on, it must be computationally simple if the overall computational cost needs to be low. In this paper, we propose novel Local Binary Pattern Histogram Fourier features(LBP-HF), a rotation invariant image descriptor based on uniform Local Binary Patterns (LBP). LBP is an operator for image description that is based on the signs of differences of neighboring pixels. It is fast to compute and invariant to monotonic gray-scale changes of the image. Despite being simple, it is very descriptive, which is attested by the wide variety of different tasks it has been successfully applied to. The LBP histogram has proven to be a widely applicable image feature for, e.g., texture classification, face analysis, video background subtraction, interest region description, etc1.Unlike the earlier local rotation invariant features, the LBP-HF descriptor is formed by first computing a non-invariant LBP histogram over the whole region and then constructing rotationally invariant features from the histogram. This means that rotation invariance is attained globally, and the features are thus invariant to rotations of the whole
- 6. input signal but they still retain information about relative distribution of different orientations of uniform local binary patterns. Technique User: Rotation Invariant Local Binary Pattern Descriptors The proposed rotation invariant local binary pattern histogram Fourier features are based on uniform local binary pattern histograms. First, the LBP methodology is briefly reviewed and the LBP-HF features are then introduced. Foundation: The Local Binary Pattern Operator The local binary pattern operator is a powerful means of texture description. The original version of the operator labels the image pixels by thresholding the3x3-neighborhood of each pixel with the center value and summing the thresholded values weighted by powers of two. The operator can also be extended to use neighborhoods of different sizes. To do this, a circular neighborhood denoted by (P,R) is defined. Here P represents the number of sampling points and R is the radius of the neighborhood. These sampling points around pixel (x, y) lie at coordinates(xp, yp) = (x + Rcos(2πp/P), y − Rsin(2πp/P)). When a sampling point does not fall at integer coordinates, the pixel value is bilinearly interpolated. Now the LBP label for the center pixel (x, y) of image f(x, y) is obtained through
- 7. Rotation Invariant Image Description with LBP-HF Features Three circular neighborhoods: (8,1), (16,2), (24,3). The pixel values are bilin early interpolated whenever the sampling point is not in the center of a pixel. Further extensions to the original operator are so called uniform patterns. A local binary pattern is called uniform if the binary pattern contains at most two bitwise transitions from 0 to 1 or vice versa when the bit pattern is considered circular. In the computation of the LBP histogram, uniform patterns are used so that the histogram has a separate bin for every uniform pattern and all non-uniform patterns are assigned to a single bin. The 58 possible uniform patterns in neighborhood of 8 sampling points. The original rotation invariant LBP operator, denoted here as LBPriu2, is achieved by circularly rotating each bit pattern to the minimum value. For instance, the bit sequences 1000011, 1110000 and 0011100 arise from different rotations of the same local pattern and they all correspond to the normalized sequence 0000111.This means that all the patterns from one row are replaced with a single label. Invariant Descriptors from LBP Histograms Let us denote a specific uniform LBP pattern by UP (n, r). The pair (n, r) specifies an uniform pattern so that n is the number of 1-bits in the pattern (corresponds o row number in Fig. 2) and r is the rotation of the pattern. Now if the neighborhood has P sampling points, n gets values from 0 to P +1,where n = P + 1 is the special label marking all the non-uniform patterns. Furthermore, when 1 ≤ n ≤ P − 1, the rotation of the pattern is in the range0 ≤ r ≤ P − 1.Let Iα ◦ (x, y) denote the rotation of image I(x, y) by α degrees. Under this rotation, point (x, y) is rotated to location (x_, y_). If we place a circular sampling neighborhood on points I(x, y) and Iα ◦ (x_, y_), we observe that it also rotates by α◦. If the rotations are limited to integer multiples of the angle between two sampling points, i.e. α = a360 ◦P , a = 0, 1, . . ., P − 1, this rotates the sampling neighborhood exactly by a discrete steps. Therefore the uniform pattern UP (n, r)at
- 8. point (x, y) is replaced by uniform pattern UP (n, r+a mod P) at point (x_, y_)of the rotated image. Now consider the uniform LBP histograms hI (UP (n, r)). The histogram value hI at bin UP (n, r) is the number of occurrences of uniform pattern UP (n, r) in image I.Let us denote a specific uniform LBP pattern by UP (n, r). The pair (n, r) specifies an uniform pattern so that n is the number of 1-bits in the pattern (corresponds to row number in Fig. 2) and r is the rotation of the pattern . Now if the neighborhood has P sampling points, n gets values from 0 to P +1, where n = P + 1 is the special label marking all the non-uniform patterns. Furthermore, when 1 ≤ n ≤ P − 1, the rotation of the pattern is in the range 0 ≤ r ≤ P − 1. Let Iα◦ (x, y) denote the rotation of image I(x, y) by α degrees. Under this rotation, point (x, y) is rotated to location (x, y). If we place a circular sampling neighborhood on points I(x, y) and Iα ◦ (x, y), we observe that it also rotates by α◦. If the rotations are limited to integer multiples of the angle between two sampling points, i.e. α = a360 ◦P , a = 0, 1, . . ., P − 1, this rotates the sampling neighborhood exactly by a discrete steps. Therefore the uniform pattern UP (n, r) at point (x, y) is replaced by uniform pattern UP (n, r+a mod P) at point (x, y) of the rotated image. Now consider the uniform LBP histograms hI (UP (n, r)). The histogram value hI at bin UP (n, r) is the number of occurrences of uniform pattern UP (n, r) in image I. Further extensions to the original operator are so called uniform patterns. A local binary pattern is called uniform if the binary pattern contains at most two bitwise transitions from 0 to 1 or vice versa when the bit pattern is considered circular. In the computation of the LBP histogram, uniform patterns are used so that the histogram has a separate bin for every uniform pattern and all non-uniform patterns are assigned to a single bin. The 58 possible uniform patterns in neighborhood of 8 sampling points. The original rotation invariant LBP operator, denoted here as LBPriu2, is achieved by circularly rotating each bit pattern to the minimum value. For instance, the bit sequences 1000011, 1110000 and 0011100 arise from different rotations of the same local pattern and they all
- 9. correspond to the normalized sequence 0000111. This means that all the patterns from one row are replaced with a single label. Invariant Descriptors from LBP Histograms Specifies an uniform pattern so that n is the number of 1-bits in the pattern and r is the rotation of the pattern . Now if the neighborhood has P sampling points, n gets values from 0 to P +1, where n = P + 1 is the special label marking all the non-uniform patterns. Furthermore, when 1 ≤ n ≤ P − 1, the rotation of the pattern is in the range0 ≤ r ≤ P − 1. Let Iα(x, y) denote the rotation of image I(x, y) by α degrees. Under this rotation, point (x, y) is rotated to location (x , y ). If we place a circular sampling neighborhood on points I(x, y) and Iα◦ (x , y ), we observe that it also rotates by α◦. If the rotations are limited to integer multiples of the angle between two sampling points, i.e. α = a360◦P , a = 0, 1, . . ., P − 1, this rotates the sampling neighborhood exactly by a discrete steps. Therefore the uniform pattern UP (n, r)at point (x, y) is replaced by uniform pattern UP (n, r+a mod P) at point (x , y )of the rotated image. Now consider the uniform LBP histograms hI (UP (n, r)). The histogram value Face Recognition The third experiment was aimed to further assess whether useful information is lost due to the transformation making the features rotation invariant. For this test, we chose the face recognition problem where the input images have been manually registered, so rotation invariance is not actually needed. The CMU PIE (Pose, Illumination, and Expression) database was used for this experiment. Totally, the database contains 41368 images of 68 subjects taken at different angles, lighting conditions and with varying expression. For our experiments, we selected a set of 23 images of
- 10. each of the 68 subjects. 2 of these are taken with the room lights on and the remaining 21 each with a flash at varying positions. In obtaining a descriptor for the facial image, the procedure of was followed. The faces were first normalized so that the eyes are at fixed positions. The uniform LBP operator at chosen scale was then applied and the resulting label image was cropped to size 128 × 128 pixels. The cropped image was further divided into blocks of size of 16 × 16 pixels and histograms were compute din each block individually. In case of LBP-HF descriptor, the rotation invariant transform was applied to the histogram, and finally the features obtained within each block were concatenated to form the spatially enhanced histogram describing the face. Due to the sparseness of the resulting histograms, Chi square distance was used with histogram type features in this experiments. With LBP-HF descriptor,L1 distance was used as in the previous experiment. Face recognition rates on CMU PIE dataset On each test round, one image per person was used for training and the remaining 22 images for testing. Again, 10000 random selections into training and testing data were used. Results of the face recognition experiment. Surprisingly, the performance of rotation invariant LBP-HF is almost equal to non-invariant LBPu2even though there are no global rotations present in the images.
- 11. Discussion and Conclusion In this paper, we proposed rotation invariant LBP-HF features based on local binary pattern histograms. It was shown that rotations of the input image cause cyclic shifts of the values in the uniform LBP histogram. Relying on this observation we proposed discrete Fourier transform based features that are invariant to cyclic shifts of input vector and, when computed from uniform LBP histograms, hence invariant to rotations of input image. Several other histogram based rotation invariant texture features have been discussed in the literature, e.g. the method proposed here differs from those since LBP-HF features are computed from the histogram representing the whole region, i.e. the invariants are constructed globally instead of computing invariant independently at each pixel location. The major advantage of this approach is that the relative distribution of local orientations is not lost. Another benefit of constructing invariant features globally is that invariant computation needs not to be performed at every pixel location. This allows for using computationally more complex invariant functions still keeping the total computational cost reasonable. In case of LBP-HF descriptor, the computational overhead is negligible. After computing the non-invariant LBP histogram, only P − 1 Fast Fourier Transforms of P points need to be computed to construct the rotation invariant LBP-HF descriptor. In the experiments, it was shown that in addition to being rotation invariant, the proposed features retain the highly discriminative nature of LBP histograms. The LBP-HF descriptor was shown to outperform the MR8 descriptor and the non invariant and earlier version of rotation invariant LBP in texture classification, material categorization and face recognition tests. Final Words: LBP is very useful and widely applicable technique for Pattern Recognition.