Side-Match Vector Quantizers Using Neural Network Based Variance Predictor for Image Coding

Shuangteng Zhang
International Journal of Image Processing (IJIP),Volume (5) : Issue (2) : 2011 119
Side-Match Vector Quantizers Using Neural Network Based
Variance Predictor for Image Coding
Shuangteng Zhang shuangteng.zhang@eku.edu
Department of Computer Science
Eastern Kentucky University
Richmond, KY 40475, U.S.A.
Abstract
Side-match vector quantizer reduces bit-rates in image coding by using smaller-sized state
codebooks generated from a master codebook through exploiting the correlations of neighboring
vectors. This paper presents a new neural network based side-match vector quantization method
for image coding. In this method, based on the variance of a vector which is predicted by a neural
network, a subset of the codewords in the master codebook is selected for the side-matching to
construct the state codebook for the encoding of the vector. This technique generates a lower
encoding bit rate with a higher reconstructed image quality. Experimental results demonstrate
that in terms of PSNR (Peak Signal-to-Noise Ratio) of the reconstructed images, the proposed
method significantly outperforms the regular side-match vector quantizer, especially at lower
coding bit-rates.
Keywords: Vector Quantization, Image Coding, Side Match, Neural Network.
1. INTRODUCTION
Image data compression is very important for various image and video processing applications
which require reduced bit-rate/channel bandwidth. Such applications include digital television,
video conferencing, telemedicine, multimedia, and remote sensing images from satellite and
reconnaissance aircrafts, as well as the storage systems of multispectral image data from space
programs, medical images, finger prints, and facial images.
There are many approaches to data compression. Among them, vector quantization (VQ) [1-3],
which achieves data compression through mapping n-dimensional vectors onto a finite set of
representative vectors called a codebook, is an important and heavily conducted research area.
In VQ image compression [4], the images are usually partitioned into blocks with each block
forming an n-dimensional vector. Each of these vectors is then coded as an index of its best
matching vector in the codebook. The reconstructed images are obtained simply by selecting the
corresponding codeword vectors from the codebook using the indices.
VQ image compression takes advantage of the correlation of the image pixels within a block
(vector) for the coding bit-rate reduction and yields acceptable performance at low bit-rates. To
further improve VQ’s performance at low bit-rates, a side-match vector quantization method
(SMVQ) [5] has been proposed. This method explores not only the redundancy within a vector
but also the strong correlation between the neighboring vectors for high quality image coding at
low bit-rates. In practice, SMVQ assumes the continuity of edges across neighboring image
blocks’ boundaries and generates a smaller-sized state codebook from the master codebook for
each block’s encoding. The selection of each state codebook is a subset of codewords in the
master codebook which are the best matches of the upper and left blocks of the block to be
encoded. In other words, SMVQ reduces the coding bit-rates by predicting the current block using
its upper and left boundary blocks. Therefore, SMVQ’s performance relies on the accuracy of the
prediction. Inaccurate prediction, which occurs often when the block is within an edge area, may
result in reduced quality of reconstructed images.

Shuangteng Zhang
To increase the accuracy of the prediction and therefore further improve the performance of
SMVQ, various SMVQ schemes have been developed. Chang and Chen [6] proposed variable-
rate side-match finite-state vector quantization with a block classifier. Wei et al. [7] reported a
three-sided side match technique which uses not only the upper and left sides, but one of the
bottom and right sides for the prediction. Yang and Tseng [8] developed a smooth side-match
classified vector quantizer which selects state codebook according to the smoothness of the gray
levels between neighboring blocks. Chang proposed a gradient match quantization method [9]
using gradient match error for the selection of the codewords in the state codebook. The
performance of the gradient side match vector quantization method was further improved by
combining the non-iterative fractal block coding technique [10,11]. Some other techniques have
also been reported for the enhancement of the original SMVQ method. Examples include pattern-
based side match VQ [12], side match VQ using gradient based classifier [13], smooth side
match weighted method [14] and side match vector quantizers with variable rates according to
both the coding quality and the bit rates [15].
In this paper, a neural network based side-match vector quantization method for image coding is
presented. In this method, a neural network is designed to predict the variances of the vectors,
which are in turn used to select the codewords in the master codebook for the generation of state
codebooks for the encoding of the vectors. This technique generates a lower encoding bit rate
with a higher reconstructed image quality. Experimental results demonstrate that in terms of
PSNR (Peak Signal-to-Noise Ratio) of the reconstructed images, the proposed method
significantly outperforms the regular side-match vector quantizer at similar low coding bit-rates.
2. SMVQ FOR IMAGE CODING
As mentioned in previous section, SMVQ takes advantage of both the redundancy within a block
and strong correlation between the neighboring blocks for high quality image coding at low bit-
rates. The original SMVQ encodes each image block by a smaller-sized state codebook
generated from a master codebook using a side-match selection function as shown in Figure 1.
Assume that the master codebook has N codewords with each codeword an m x n vector
denoting by Ci, i=1, 2, …, N. Also assume that the image to be encoded is partitioned into blocks
of size m x n. SMVQ encodes the image blocks in an order from left to right and top to bottom.
For each block being encoded, SMVQ uses the side information of its upper and left neighboring
blocks to produce the state codebook. The block is encoded as the index of the codeword in the
state codebook which is the best match to the block.
FIGURE 1: Block diagram of SMVQ encoder
Image to be encoded
Code (index)
Neighboring
block1
Neighboring
block2
Current block
Master Codebook
State CodebookSide Match
Quantizer:
Nearest neighbor
search

Shuangteng Zhang
Let x(i, j) (i=1, 2, …, m and j=1, 2, …, n) be the pixel in the current image block X, and u(i, j) (i=1,
2, …, m and j=1, 2, …, n) and l(i, j) (i=1, 2, …, m and j=1, 2, …, n) be its upper block U and left
block L, respectively. The state codebook is generated and the block is encoded according the
following steps:
(1) For each of the codewords Ci in the master codebook, calculate the side-match error ei,
)1())1,(),(()),1(),((
1
2
1
2
∑∑ ==
−+−=
m
k
i
n
k
ii kcnklkckmue
(2) Select M (M ≤ N) codewords in the master codebook with the smallest side-match errors
as the state codebook codewords Si, i=1, 2, …, M,
(3) For each of the codewords Si in the state codebook, calculate its distortion measure di(Si,
X),
∑∑= =
−=
m
k
n
r
iii rkxrksXSd
1 1
2
)2()),(),((),(
(4) The image block X is encoded as the index j of the codeword Sj which produces the
smallest distortion dj(Sj, X).
For the decoding of each image block, SMVQ first generates the state codebook which was used
to encode it according to step (1) and (2) in the encoding process. Once the state codebook is
generated, the reconstructed block is obtained by simply selecting the corresponding codeword in
the state codebook using the index which is the code of the block generated in the encoding
process. The block diagram of SMVQ decoder is shown in Figure 2.
FIGURE 2: Block diagram of SMVQ decoder
3. PROPOSED NEURAL NETWORK BASED SMVQ
3.1 Image Block Variance for SMVQ
SMVQ image coding achieves low bit-rates through using smaller-sized state codebooks
generated from master codebook for the coding of the image blocks. The quality of the
reconstructed images coded at low bit-rates relies on the accuracy of the prediction of the blocks.
An accurate prediction of a block may keep the codeword, which is the best match of the block
when a full search in the master codebook is conducted, in the state codebook generated by the
side match function. The more accurate prediction makes it possible to select a smaller-sized
state codebook for the coding so that the image can be coded at a lower bit-rate with a quality
Reconstructed Image
Code (index)
Neighboring
block1
Neighboring
block2
Reconstructed block
Master Codebook
Table
Lookup
Side Match State Codebook

Shuangteng Zhang
similar to the one obtained when the master codebook is used. However, the prediction may be
inaccurate, especially when the block is within an edge area. An inaccurate prediction on the
other hand may keep the codeword which is the best match of the block in a full search out of the
generated state codebook when a small size is chosen, and therefore results in a degraded
reconstructed image quality.
Since edge and non-edge blocks may have different variances with an edge block highly likely to
have a higher variance, the best matched codeword for each block in a full search may scatter
around the codeword in the master codebook whose variance is equal to or close to the variance
of the block to be encoded, assume that the master codebook is sorted according to the
variances of the codewords. Figure 3 shows the distribution of the best matched codewords in full
search for the encoding of image Lena using a codebook of size 256 which is sorted by variance
of the codewords. The x-axes is the difference value of the index of codeword whose variance is
equal or close to the variance of the block being encoded and the index of its best matched
codeword in a full search, and the y-axes is the number of images blocks associated with the
same index difference value.
FIGURE 3: Distribution of the best matched codewords in full search
From Figure 3, it can be seen that more than 90% of the best matched codewords are distributed
in the range from 6 codewords before to 54 codewords after the codeword whose variance is
equal to or close to the variance of the block to be coded. In other words, if the 60 codewords
around the codeword whose variance matches the variance of the block being coded are used to
be side-matched to generate the state codebook, then there is a much higher possibility that the
best matched codeword will be in the state codebook even if the size of the state codebook is
selected to be smaller than that selected in the regular SMVQ. This will result in a faster
processing speed since a subset of codewords in the master codebook instead of the whole is
used for side matching. Furthermore, the coding bit rate can be further lowered due to the fact
that smaller size of state codebooks can be used while preserving similar reconstructed image
quality.
Based on the above observation, in our proposed SMVQ method, the variances of the image
blocks, which are usually sensitive to image edges, are combined into the side matching and
state codebook generating process for more accurate prediction and therefore lowered coding bit-
rate with preserved image quality.
-60 -40 -20 0 20 40 60 80 100
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
Position of the Best Matched Codewords in Full Search
NumberofImageBlocks

Shuangteng Zhang
3.2 Neural Network Based Variance Predictor
In the proposed method, the variance of the block being coded is used to select a subset of the
codewords in the master codebook for the side matching and state codebook generation. To
implement the process, the variance of the block must be known. In the proposed algorithm, the
variance is predicted by a three-layered feed-forward neural network [16] as shown in Figure 4.
FIGURE 4: Neural network structure of the variance predictor
This neural network contains an input layer with p external inputs, a hidden layer with q neurons
and an output layer with one neuron. The external inputs of the network consist of the mean value
k of the pixels in the last two rows of the upper block and last two columns of the left block of the
current block and the difference values zi (i=1, 2, …, p-1) between the mean k and each pixel
value in those rows and columns, denoted as I=(I1, I2, …, Ip)=(k, z1, z2, …, zp-1), where p =
2*(m+n)+1, k is calculated as,
∑ ∑∑ ∑ = −=−= =
+
+
=
m
i
n
nj
m
mi
n
j
jiljiu
nm
k
1 11 1
(3))),(),((
)(*2
1
The output of the network is the predicted variance value of the current block, which is formulated
as,
)4())((
11
∑∑ ==
=
p
i
iij
q
j
j IrfwfV
where V is the output of the network, Ii is the i-th element of the input vector I, rij is the weight of
the connection between the i-th neuron in the input layer and the j-th neuron in the hidden layer,
wj is the connection weight between the j-th neuron in the hidden layer and the neuron in the
output layer, and f(.) is the activation function of the neurons, which is
(5)1
1
2
)( −
+
= − x
e
xf λ
where λ > 0 is the neuron activation function coefficient determining the steepness of the function.
The network is trained using back-propagation learning algorithm [16] with sample images. After
trained, it can be used to predict the variance of each block being encoded.
3.3 The Encoding and Decoding Algorithms
Given the information presented in the previous sections, the proposed neural network based
SMVQ encoder and decoder algorithms are described in this section. The encoder consists of
four elements including neural network variance predictor, candidate codeword selector, side
V
H1 H2 ... Hq
...
...
I1 I2 ... Ip
R
W

Shuangteng Zhang
match function, and quantizer as shown in Figure 5. The encoder algorithm can be implemented
as follows:
(1) Sort the master codebook by the variances of the codewords in an ascending order,
(2) Use the neural network variance predictor to predict the variance of the current block,
(3) Select T (predefined) (T < N) codewords in the master codebook around the codeword
whose variance is equal to or is closest to the predicted variance value,
(4) For each of the codewords Ci selected in step (3), calculate the side-match error ei using
equation (1),
(5) Select M (M ≤ T) codewords with the smallest side-match errors from the T codewords
selected in step (3) as the state codebook codewords Si, i=1, 2, …, M,
(6) For each of the codewords Si in the state codebook, calculate its distortion measure di(Si,
X) using equation (2),
(7) If the smallest distortion di(Si, X) is greater than a predefined threshold, the block is
coded using full search. Otherwise, it is encoded as the index j of the codeword Sj which
produces the smallest distortion dj(Sj, X),
(8) Repeat step (2) – (7) for next block encoding.
FIGURE 5: Block diagram of the proposed neural network based SMVQ encoder
The structure of the decoder is same as that of the encoder except that the quantizer element in
the encoder is replaced by the table lookup element. The block diagram of the decoder is shown
as in Figure 6. For the decoding of each image block, the coder first generates the state
codebook which was used to encode it according to step (2) and (5) in the encoding process.
Once the state codebook is generated, the reconstructed block is obtained by simply selecting
the corresponding codeword in the state codebook using the index which is the code of the block
generated in the encoding process.
Predicted variance of
the Current block
Image to be encoded
Code (index)
Neighboring
block1 Neighboring
block2
Current block
Master
Codebook
Quantizer:
Nearest neighbor
search
Neural Network
Variance Predictor
Candidate
Codeword
Selector

Shuangteng Zhang
FIGURE 6: Block diagram of the proposed neural network based SMVQ decoder
4. EXPERIMENTS AND DISCUSSION
In this section, the performance of the proposed algorithm is evaluated by experiments. The
peak signal-to-noise ratio (PSNR) is used as a quantitative measure of the quality of the
reconstructed images. PSNR can be calculated as,
where ),( jiz
∧
and z(i,j) are the (i,j)th pixels in the reconstructed and original images, respectively,
and the images are of size B x B.
In our experiments, the master codebook of size 256 was generated using LBG algorithm [17]
and the neural network based variance predictor was trained using 8-bit gray level image Lena of
size 512 x 512. The trained neural network predictor and coding algorithm were then applied to
code the image Lena as well as some other gray images of different objects at various bit rates.
For comparison purpose, the corresponding images were also coded by using the regular SMVQ
method.
Table 1 shows the PSNRs of the reconstructed images coded around bit rate 0.24bpp (bit per
pixel), 0.31bpp and 0.37bpp by using the regular SMVQ and the proposed method. This table
indicates that the proposed algorithm improves the PSNR of the reconstructed image over the
regular SMVQ average by 1.66dB, 1.06dB and 0.42dB when bit rate is around 0.24bpp, 0.32bpp,
and 0.37bpp, respectively. These results demonstrate that in terms of PSNR, the proposed
algorithm significantly outperforms the regular SMVQ method, especially when the coding bit rate
is low.
(6)
)),(),((
*
1
255
log*10PSNR
1 1
2
2
10
∑∑= =
∧
−
= B
i
B
j
jizjiz
BB
Predicted variance of
the current block
Reconstructed Image
Code (index)
Neighboring
block1 Neighboring
block2
Reconstructed block
Master
Codebook
Neural Network
Variance Predictor
Candidate
Codeword
Selector
Table
Lookup

Shuangteng Zhang
TABLE 1: PSNRs (dB) comparison (Coding Bit Rate around 0.24bpp, 0.31bpp and 0.37bpp)
Figure 7 shows the curves of PSNRs as a function of coding bit-rate for the proposed and regular
SMVQ methods. From the figure, it can be observed that from low to high bit-rates, the PSNRs of
the images coded by using the proposed method approach much faster than the regular SMVQ
method to a constant value, which is the highest PSNR value SMVQ methods can reach and can
be obtained when the images are coded using the master codebook. This also confirms that the
proposed method has a more accurate prediction of the block being coded and therefore allows
to use smaller state codebooks which generate lower coding bit-rates with higher images quality.
FIGURE 7: PSNR as a function of coding bit-rate
The visual quality of the reconstructed images is also examined. Figure 8 shows the original and
corresponding Lena images coded by master codebook at 0.5bpp, the proposed method at
0.30bpp and the regular SMVQ method at 0.31bpp. Magnified face portions of the original and
the reconstructed Lena images are shown in Figure 9 for further comparison. These figures show
that the images coded by the proposed method are at the quality similar to those coded by the
master codebook and they are sharper and visually look much more pleasant than the ones
Bit rate:
around 0.24bpp
Bit rate:
around 0.31bpp
Bit rate:
around 0.37bpp
SMVQ
Proposed
Method
SMVQ
Proposed
Method
SMVQ
Proposed
Method
Lena
Peppers
Elaine
Airplane
Flowers
Kids
28.70
27.87
28.32
26.50
30.31
26.15
31.17
28.86
29.35
28.19
31.75
28.51
30.27
28.66
29.14
27.34
31.67
27.32
31.83
29.20
29.82
28.43
32.53
28.95
31.38
29.06
29.60
28.06
32.38 28.18
31.93
29.25
29.86
28.46
32.65
29.02
Average 27.98 29.64 29.07 30.13 29.78 30.20

Shuangteng Zhang
coded by the regular SMVQ method at the similar coding bit-rate. This is consistent with the
PSNR results shown in Table I.
(a) (b)
(c) (d)
FIGURE 8: (a) Original Lena image; (b) Lena image coded by using the master codebook at 0.5bpp; (c)
Lena image coded by using the proposed method at 0.3bpp; (d) Lena image coded by using
the regular SMVQ at 0.31bpp

Shuangteng Zhang
(a) (b)
(c) (d)
FIGURE 9: Magnified face portion from (a) Original Lena image; (b) Lena image coded by using the
master codebook at 0.5bpp; (c) Lena image coded by using the proposed method at 0.3bpp;
(d) Lena image coded by using the regular SMVQ at 0.31bpp
5. CONCLUSION
We have presented a new side-match vector quantization method for image coding using a
neural network-based variance predictor. In this method, a neural network is used to predict the
variances of the image blocks. The predicted variances are in turn used to select a subset of the
codewords in the master codebook for the side matching to generate the state codebooks for the
encoding of the image blocks. With the involvement of the image block variances, the regular
SMVQ is enhanced with more accurate prediction of the current block by using side information
and therefore generates smaller state codebooks which encode the images at a lower bit rate
with higher image quality. Experimental results demonstrate that in terms of PSNR (Peak Signal-
to-Noise Ratio) of the reconstructed images, the proposed method significantly outperforms the
regular side-match vector quantizer at similar low coding bit-rates.
6. REFERENCES
[1] R. M. Gray, “Vector quantization”, IEEE ASSP Magazine 1, pp. 4-29, 1984.
[2] A. Gersho, R. M. Gray, Vector Quantization and Signal, Compression, Kluwer Academic
Publishers, 1992.
[3] N. M. Nasrabadi, R. A. King, “Image coding using vector quantization: a review”, IEEE Tran.
Communications, vol. 36, no. 8, pp. 957-971, 1988.
[4] K. Sayood, Introduction to data compression, Morgan Kaufmann Publishers, San Francisco,
CA 1996.

Shuangteng Zhang
[5] T. Kim, “Side match and overlap match vector quantizers for images”, IEEE Trans. Image
Process., vol. 1, no. 2, pp. 170 -185, 1992.
[6] R. F. Chang and W. -T. Chen, “Image coding using variable-rate side-match finite-state
vector quantization”, IEEE Tran. Image Processing, vol. 2, no. 1, pp. 104-108, 1993.
[7] H. Wei, P. Tsai and J. Wang, “Three-sided side match vector quantization”, IEEE Trans.
Circuits and Systems for Video Technology, vol. 10, no. 1, pp. 51–58, 2000.
[8] S. B. Yang and L. Y. Tseng, “Smooth side-match classified vector quantizer with variable
block size”, IEEE Tran. Image Processing, vol. 10, no. 5, pp. 677-685, 2001.
[9] H. T. Chang, “Gradient match vector quantizers for images”, Opt. Eng., vol. 39, no. 8,
pp.2046-2057, 2000.
[10] H. T. Chang, “Gradient match and side match fractal vector quantizers for images”, IEEE
Trans. Image Process., vol. 11, no. 1, pp. 1-9, 2002.
[11] H. T. Chang and C. J. Kuo, “Iteration-free fractal image coding based on efficient domain
pool design”, IEEE Tran. Image Processing, vol. 9, pp.329-339, 2000.
[12] C. C. Chang, F. C. Shiue, T. S. Chen, “Pattern-based side match vector quantization for
image compression”, Imaging Science Journal, vol. 48, no. 2, pp. 63-76, 2000.
[13] Z. M. LU, B. Yang, S. H. SUN, “Image compression algorithms based on side-match vector
quantizer with gradient-based classifiers”, IEICE TRAN. Information and Systems, vol. E85-
D, no.9, pp.1409-1415, 2002.
[14] S. B. Yang, “Smooth side-match weighted vector quantiser with variable block size for image
coding”, IEE Proc. Vis. Image Signal Processing, vol. 152, no. 6, pp. 763-770, 2005.
[15] S. B. Yang, “New variable-rate finite state vector quantizer for image coding”, Opt. Eng., vol.
44, no. 6, 067001, 2005.
[16] M. H. Hassoun, Fundamentals of Artificial Neural Network, MIT Press, Cambridge, MA,
1995.
[17] Y. Linde, A. Buzo and R. M. Gray, “An algorithm for vector quantization design”, IEEE Trans.
Communications, vol. 28, pp. 84-95, 1980.

Side-Match Vector Quantizers Using Neural Network Based Variance Predictor for Image Coding

More Related Content

What's hot (16)

Similar to Side-Match Vector Quantizers Using Neural Network Based Variance Predictor for Image Coding (20)

Recently uploaded (20)

Side-Match Vector Quantizers Using Neural Network Based Variance Predictor for Image Coding