Coding and Error Control Lecture 4 Presentation

Coding and Error Control
Lecture 4
G. Noubir
noubir@ccs.neu.edu
Textbook: Chapter 8, “Wireless Communications and Networks”,
William Stallings, Prentice Hall

COM3525: Wireless Networks, Coding and Error Control 2
Coping with Data Transmission
Errors
 Error detection codes
 Detects the presence of an error
 Automatic repeat request (ARQ) protocols
 Block of data with error is discarded
 Transmitter retransmits that block of data
 Error correction codes, or forward correction
codes (FEC)
 Designed to detect and correct errors

Error Detection Probabilities
 Definitions
 Pb : Probability of single bit error (BER)
 P1 : Probability that a frame arrives with no bit errors
 P2 : While using error detection, the probability that a
frame arrives with one or more undetected errors
 P3 : While using error detection, the probability that a
frame arrives with one or more detected bit errors but
no undetected bit errors

Error Detection Probabilities
 With no error detection

F = Number of bits per frame
 
0
1
1
3
1
2
1





P
P
P
P
P
F
b

Error Detection Process
 Transmitter
 For a given frame, an error-detecting code (check bits) is
calculated from data bits
 Check bits are appended to data bits
 Receiver
 Separates incoming frame into data bits and check bits
 Calculates check bits from received data bits
 Compares calculated check bits against received check
bits
 Detected error occurs if mismatch

Parity Check
 Parity bit appended to a block of data
 Even parity
 Added bit ensures an even number of 1s
 Odd parity
 Added bit ensures an odd number of 1s
 Example, 7-bit character [1110001]
 Even parity [11100010]
 Odd parity [11100011]

Cyclic Redundancy Check (CRC)
 Transmitter
 For a k-bit block, transmitter generates an (n-k)-
bit frame check sequence (FCS)
 Resulting frame of n bits is exactly divisible by
predetermined number
 Receiver
 Divides incoming frame by predetermined
number
 If no remainder, assumes no error

CRC using Modulo 2 Arithmetic
 Exclusive-OR (XOR) operation
 Parameters:

T = n-bit frame to be transmitted

D = k-bit block of data; the first k bits of T

F = (n – k)-bit FCS; the last (n – k) bits of T

P = pattern of n–k+1 bits; this is the predetermined
divisor

Q = Quotient

R = Remainder

 For T/P to have no remainder, start with
 Divide 2n-k
D by P gives quotient and
remainder
 Use remainder as FCS
F
D
T k
n

 
2
P
R
Q
P
D
k
n



2
R
D
T k
n

 
2

 Does R cause T/P have no remainder?
 Substituting,
 No remainder, so T is exactly divisible by P
P
R
P
D
P
R
D
P
T k
n
k
n






2
2
Q
P
R
R
Q
P
R
P
R
Q
P
T








CRC using Polynomials
 All values expressed as polynomials
 Dummy variable X with binary coefficients
 
 
   
 
     
X
R
X
D
X
X
T
X
P
X
R
X
Q
X
P
X
D
X
k
n
k
n







CRC using Polynomials
 Widely used versions of P(X)
 CRC–12

X12
+ X11
+ X3
+ X2
+ X + 1
 CRC–16

X16
+ X15
+ X2
+ 1
 CRC – CCITT

X16
+ X12
+ X5
+ 1
 CRC – 32

X32
+ X26
+ X23
+ X22
+ X16
+ X12
+ X11
+ X10
+X8
+ X7
+
X5
+X4
+ X2
+ X + 1

CRC using Digital Logic
 Dividing circuit consisting of:
 XOR gates

Up to n – k XOR gates

Presence of a gate corresponds to the presence of a
term in the divisor polynomial P(X)
 A shift register

String of 1-bit storage devices

Register contains n – k bits, equal to the length of the
FCS

Digital Logic CRC

Wireless Transmission Errors
 Error detection requires retransmission
 Detection inadequate for wireless
applications
 Error rate on wireless link can be high, results in
a large number of retransmissions
 Long propagation delay compared to
transmission time

Block Error Correction Codes
 Transmitter
 Forward error correction (FEC) encoder maps
each k-bit block into an n-bit block codeword
 Codeword is transmitted; analog for wireless
transmission
 Receiver
 Incoming signal is demodulated
 Block passed through an FEC decoder

Forward Error Correction Process

FEC Decoder Outcomes
 No errors present
 Codeword produced by decoder matches original
codeword
 Decoder detects and corrects bit errors
 Decoder detects but cannot correct bit errors;
reports uncorrectable error
 Decoder detects no bit errors, though errors
are present

Block Code Principles
 Hamming distance – for 2 n-bit binary
sequences, the number of different bits
 E.g., v1=011011; v2=110001; d(v1, v2)=3
 Redundancy – ratio of redundant bits to data
bits
 Code rate – ratio of data bits to total bits

Block Codes
 The Hamming distance d of a Block
code is the minimum distance between
two code words
 Error Detection:
 Upto d-1 errors
 Error Correction:
 Upto
 Combine: d2+l+1





 
2
1
d

Coding Gain
 Definition:
 The coding gain is the amount of additional SNR
or Eb/N0 that would be required to provide the
same BER performance for an uncoded signal
 If the code is capable of correcting at most t
errors and PUC is the BER of the channel
without coding, then the probability that a
bit is in error using coding is:
i
n
UC
i
UC
n
t
i
CB P
P
i
n
i
n
P 











  )
1
(
1
1

Hamming Code
 Designed to correct single bit errors
 Family of (n, k) block error-correcting codes
with parameters:
 Block length: n = 2m
– 1
 Number of data bits: k = 2m
– m – 1
 Number of check bits: n – k = m

Minimum distance: dmin = 3
 Single-error-correcting (SEC) code
 SEC double-error-detecting (SEC-DED) code

Example of Binary Block Code (7, 4)
 Any two different code words are different on at least three
different coordinates. This code has Hamming distance 3.
Message block Code word
(0 0 0 0) (0 0 0 0 0 0 0 )
(1 0 0 0) (1 1 0 1 0 0 0)
(0 1 0 0) (0 1 10 1 0 0)
(1 1 0 0) (1 0 1 1 1 0 0)
(0 0 1 0) (1 1 1 0 0 1 0)
(1 0 1 0) (0 0 1 1 0 1 0)
(0 1 1 0) (1 0 0 0 1 1 0)
(1 1 1 0) (0 1 0 1 1 1 0)
(0 0 0 1) (1 0 1 0 0 0 1)
(1 0 0 1) (0 1 1 1 0 0 1)
(0 1 0 1) (1 1 0 0 1 0 1)
(1 1 0 1) (0 0 0 1 1 0 1)
(0 0 1 1) (0 1 0 0 0 1 1)
(1 0 1 1) (1 0 0 1 0 1 1)
(0 1 1 1) (0 0 1 0 1 1 1)
(1 1 1 1) (1 1 1 1 1 1 1)
 Notice that the last 4 bits of the code word are the same as the message
 This is a systematic coding
 The other 3 bits are redundancy bits

Hamming Code Process
 Encoding: k data bits + (n -k) check bits
 Decoding: compares received (n -k) bits with
calculated (n -k) bits using XOR
 Resulting (n -k) bits called syndrome word
 Syndrome range is between 0 and 2(n-k)
-1
 Each bit of syndrome indicates a match (0) or
conflict (1) in that bit position

Linear Block
 A block code of length n and 2k
code words is called a linear (n, k)
code iff its 2k
code words form a k-dimensional subspace of the
vector space of all n-tuples over the field GF(2)
 Informal Meaning: linear combinations of codewords are also
codewords
 The linear code is completely specified by k independent codewords
(G generator matrix)
 The encoder needs only to store the matrix G
 Example: Linear code (7, 4)

To encode u = (1 1 0 1): v = 1g0 + 1g1 + 0g2 + 1g3 












1
0
0
0
1
0
1
0
1
0
0
1
1
1
0
0
1
0
1
1
0
0
0
0
1
0
1
1
G

Parity-Check Matrix (H)
 For every kxn generator matrix G (with k linearly
independent rows), there exists a (n-k)xn matrix H (with
(n-k) linearly independent rows) called parity check
matrix such that: any vector in the row space of G is
orthogonal to any row of H and any vector that is
orthogonal to all the rows of H is in the row space of G:
 The 2n-k
combinations of the rows of H form an (n-k)-
dimensional subspace. It is the null space of C the (n, k)
code generated by G. It is called the dual space of C
and noted Cd.
 For any v in C and w in Cd we have v w = 0
0


 T
vH
uG
v

Systematic Coding
 A linear block code is called systematic iff the
code words can be divided into two parts:
Message part (k bits), and a redundancy checking
part (n-k bits)
 Systematic code generator matrix: G = [P Ik]; H =
[In-k PT
]; GHT
=0
Redundant
Checking part
Message
part
n-k digits k digits




























































 


 






 





 

matrix
identity
matrix
parity
)
(
:
1
,
1
2
,
1
1
,
1
0
,
1
1
,
2
0
,
2
1
,
1
2
,
1
1
,
1
0
,
1
1
,
0
2
,
0
1
,
0
0
,
0
1
1
0
1
.
.
0
0
0
.
.
.
.
.
.
.
.
.
.
.
.
0
.
.
1
0
0
0
.
.
0
1
0
0
.
.
0
0
1
...
.
.
.
.
.
.
.
.
.
.
...
...
.
.
.
kxk
k
n
kx
P
k
n
k
k
k
k
k
n
k
n
k
n
k p
p
p
p
p
p
p
p
p
p
p
p
p
p
g
g
g
G

Hamming Codes (2m
-1, 2m
-m-
1)
 Parameters of Hamming codes (m>2):
 Code length: n = 2m
-1
 Number of information bits: k = 2m
-m-1
 Number of parity check bits: n-k = m
 Minimum distance: 3
 Error-correcting capability: t = (3-1)/2 = 1
 Parity check matrix of a Hamming code: H = [Im Q]
 Im is the identity matrix mxm
 Q consists of 2m
-m-1 columns which are the m-tuples of weight
2 or more
 Example:











1
1
1
0
1
0
0
0
1
1
1
0
1
0
1
1
0
1
0
0
1
H

Minimum Distance of Block
Codes
 The minimum distance dmin of a block code is
the minimum distance between any two
code words:
 dmin = min {d(v, w), s.t. v, w C, v  w}
 dmin = min {w(v + w), s.t. v, w C, v  w}
 dmin = min {w(x), s.t. x C, x  0} (because C is a
vector sub-space)
 Example: Minimum Distance of Hamming
Codes?

Cyclic Block Codes
 Definition:
 An (n, k) linear code C is called a cyclic code if every cyclic
shift of a code vector in C is also a code vector
 Codewords can be represented as polynomials of degree n.
For a cyclic code all codewords are multiple of some
polynomial g(X) modulo Xn
+1 such that g(X) divides Xn
+1.
g(X) is called the generator polynomial.
 Examples:
 Hamming codes, Golay Codes, BCH codes, RS codes
 BCH codes were independently discovered by
Hocquenghem (1959) and by Bose and Chaudhuri (1960)
 Reed-Solomon codes (non-binary BCH codes) were
independently introduced by Reed-Solomon

Cyclic Codes
 Can be encoded and decoded using linear
feedback shift registers (LFSRs)
 For cyclic codes, a valid codeword (c0, c1, …,
cn-1), shifted right one bit, is also a valid
codeword (cn-1, c0, …, cn-2)
 Takes fixed-length input (k) and produces
fixed-length check code (n-k)
 In contrast, CRC error-detecting code accepts
arbitrary length input for fixed-length check code

Cyclic Block Codes
 A cyclic Hamming code of length 2m
-1 with m>2 is
generated by a primitive polynomial p(X) of degree m
 Hamming code (31, 26)
 g(X) = 1 + X2
+ X5
, l = 3
 Golay Code:
 cyclic code (23, 12)
 minimum distance 7
 generator polynomials: either g1(X) or g2(X)
11
9
7
6
5
2
11
10
6
5
4
2
1
1
)
(
1
)
(
X
X
X
X
X
X
X
g
X
X
X
X
X
X
X
g















BCH Codes
 For positive pair of integers m and t, a (n, k)
BCH code has parameters:
– 1
 Number of check bits: n – k  mt
 Minimum distance:dmin  2t + 1
 Correct combinations of t or fewer errors
 Flexibility in choice of parameters
 Block length, code rate

Bose Chaudhuri and
Hocquenghem (BCH) Codes
 For any positive integer m (m>2) and t (t<2m-1
), there exists a BCH code:
 Block length: 2m-1
 Parity bits: n - k  mt
 Minimum distance: dmin  2t + 1

generator polynomial is the lowest-degree polynomial over GF(2) such that:
, 2,  3, …, 2t are its roots.  is the primitive root of unity in the extension field
GF(2n
).
 Such a BCH code is capable of correcting t errors
 The generator polynomial is:
 g(X) = LCM{1(X), 2(X), …, 2t(X)}
 for i = i’2l
, i(X) = i’(X) (because i
and i’
are conjugates), thus
g(X) = LCM{1(X), 3(X), …, 2t-1(X)}
 Since any minimal polynomial has degree m or less, then degree (n - k) of g
is at most mt
 If t = 1, then g is a primitive polynomial and we have a Hamming code

Reed-Solomon Codes
 Subclass of nonbinary BCH codes
 Data processed in chunks of m bits, called symbols
 An (n, k) RS code has parameters:
 Symbol length: m bits per symbol
– 1 symbols = m(2m
– 1) bits
 Data length: k symbols
 Size of check code: n – k = 2t symbols = m(2t) bits
 Minimum distance: dmin = 2t + 1 symbols

Block Interleaving
 Data written to and read from memory in
different orders
 Data bits and corresponding check bits are
interspersed with bits from other blocks
 At receiver, data are deinterleaved to recover
original order
 A burst error that may occur is spread out
over a number of blocks, making error
correction possible

Convolutional Codes
 Generates redundant bits continuously
 Error checking and correcting carried out
continuously
 (n, k, K) code

Input processes k bits at a time

Output produces n bits for every k input bits

K = constraint factor

k and n generally very small
 n-bit output of (n, k, K) code depends on:

Current block of k input bits

Previous K-1 blocks of k input bits

Decoding
 Trellis diagram – expanded encoder diagram
 Viterbi code – error correction algorithm
 Compares received sequence with all possible
transmitted sequences
 Algorithm chooses path through trellis whose
coded sequence differs from received sequence in
the fewest number of places
 Once a valid path is selected as the correct path,
the decoder can recover the input data bits from
the output code bits

Representations of Convolutional
Codes
 Trellis representation:
00
11
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
States
00
10
01
11

Viterbi Decoding Algorithm
 Principle:

Computes a measure of similarity (distance) between
the received signal and all potential trellis paths
 For each state keeps only one “more likely” path
 Algorithm:

For each input

For each state:

Determine the distance/weight of the branches leading to it
 Keep only the best branch
 If all surviving paths at time ti pass through some state S at
time ti-j then commit to S(ti-j)

Viterbi Decoding
 Input: 11011011010
 Codeword: 11 01 01 00 01 01 00 01 01 00 10
 Received: 11 01 01 01 01 01 00 01 01 00 10
00
11
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
00
11
10
01
States
00
10
01
11
11 01 01 01 01 01 00 01 01 00 10
1 1 0 1 1 0 1 1 0 1 0

Automatic Repeat Request
 Mechanism used in data link control and
transport protocols
 Relies on use of an error detection code (such
as CRC)
 Flow Control
 Error Control

Flow Control
 Assures that transmitting entity does not
overwhelm a receiving entity with data
 Protocols with flow control mechanism allow
multiple PDUs in transit at the same time
 PDUs arrive in same order they’re sent
 Sliding-window flow control
 Transmitter maintains list (window) of sequence
numbers allowed to send
 Receiver maintains list allowed to receive

Flow Control
 Reasons for breaking up a block of data
before transmitting:
 Limited buffer size of receiver
 Retransmission of PDU due to error requires
smaller amounts of data to be retransmitted
 On shared medium, larger PDUs occupy medium
for extended period, causing delays at other
sending stations

Error Control
 Mechanisms to detect and correct
transmission errors
 Types of errors:
 Lost PDU : a PDU fails to arrive
 Damaged PDU : PDU arrives with errors

Error Control Requirements
 Error detection
 Receiver detects errors and discards PDUs
 Positive acknowledgement
 Destination returns acknowledgment of received, error-
free PDUs
 Retransmission after timeout
 Source retransmits unacknowledged PDU
 Negative acknowledgement and retransmission
 Destination returns negative acknowledgment to PDUs in
error

Coding and Error Control Lecture 4 Presentation

Recommended

More Related Content

Similar to Coding and Error Control Lecture 4 Presentation (20)

More from DrAdeelAkram2 (14)

Recently uploaded (20)

Coding and Error Control Lecture 4 Presentation

Editor's Notes