Channel Coding/DecodingHamming Method

Channelencoder decoderSourcedecoding

Channeldecoding

Sourcecoding

Channelcoding

Sent

messagessymbolssymbols

Error CorrectionError Correction

Channel CapacityChannel Capacity

CompressionCompression

Source EntropySource Entropy

DecompressionDecompression

Rate vs Distortion Capacity vs Efficiency

Received

messages

INFORMATION TRANSFER ACROSS CHANNELS

source receiver

ChannelSource

decodingSourcecoding

Sent

messages

Informationsource

symbolssymbols

Error CorrectionError Correction

Channel CapacityChannel Capacity

CompressionCompression

Source EntropySource Entropy

DecompressionDecompression

Rate vs Distortion Capacity vs Efficiency

Received

messages

INFORMATION TRANSFER ACROSS CHANNELS

receiverChanneldecoding

Channelcoding

Channel Coding/Decoding

The purpose of channel coding/decoding is to detect and correcterrors in noisy channels

Channel Coding/Decoding

Error detection and correction

In a noisy channel errors may occur during the transmission of data from information source to destination, so we need a methodto detect there errors and then correct them.

Error-Detecting and -Correcting Codes

• Add extra bits to the data bits for the sole purpose of detecting when errors have been introduced and correcting the errors if possible

Repetition Code• Repeat every bit 3 times• 0110 ==> 000,111,111,000• Error detected if all 3 bits are not the same• 000,110,111,000

Analysis of Repetition Code

• 200% overhead (code is 3x size of data)• Error can go undetected only if 3 consecutive bits

are in error• 0110 => 000,111,111,000 => 000,000,111,000• If probability of one-bit error is p, then probability

of undetected error is p3

• E.g. one-bit error = 10-5 => undetected error = 10-15

• (Assumes independence)

What if an Error is Detected?

• Strategy #1: Throw the data out and ask for it to be sent again

• Slow but very low odds of erroneous data• Strategy #2: Majority rules• 0110 => 000,111,111,000 => 000,110,111,000

=> 000,111,111,000• Quicker, but higher odds of error • Was it actually 0010 => 000,000,111,000 =>

000,110,111,000?

Parity Codes

• Add one bit to every block of, say, 4 bits

• Parity bit = 0 or 1 so that total number of 1 bits is even

• Detects all 1-bit errors, no 2-bit errors

11110111011101111000101111010010010100010111101100010100100100110001010001100000

Channel Coding/DecodingHamming Method

- It was the first complete error-detecting and error-correcting procedure.

- It represents one of the simplest and most common method for the

transmission of information ( in the presence of noise ).

- It assumes that the source transmits binary messages ( i.e. Theinformation source alphabet is { 0, 1 })

- It assumes that the channel is a binary symmetric channel (BSC)

- It uses the parity checker method to detect an error

Hamming Codes

• (4,7) Hamming Code detects all one- and two-bit errors

• Corrects all 1-bit errors• Magic: Any two different codewords differ in at

least 3 places!

1111111111010011010001100011

1011001101001010011101000101

0111010011000101011010100110

0011100001011100010110000000

Hamming Distance• Number of places in which two bit strings

differ• = Hamming distance 3

• Acts like a distance:

01110011010001

a b

= a+b

Error Correcting Codes• ECC design is a kind of geometry problem:

Find 16 bit strings of length 7, no two of which are separated by distance less than 3

Hamming Distance as Geometry

1000101

1000100

11001011001101

10011111100111

10101001100100

1001110

1101110

Definitions

• Hamming distance between x and y is

dH := d(x, y) is the # of positions where xi ≠yi

• The minimum distance of a code C is • dmin = min { d(x, y) | x ∈ C, y ∈ C, x y}

• Hamming weight of a vector x is

- w(x) := d(x, 0 ) is the # of positions where xi ≠0

Example

• Hamming distance d( 1001, 0111) = 3

• Minimum distance (101, 011, 110) = 2

• Hamming weight w(0110101) = 4

PerformanceA code with minimum distance dmin is capable of

correcting t errors if

dmin ≥ 2 t + 1.

Proof: If ≤ t errors occur, then since dmin ≥ 2 t + 1an incorrect code word has at least t+1 differences with the received word.

Hamming codes

• Minimum distance 3• Construction

• G = Im All k-tuples of Hamming weight > 1

• where m = 1k2 k −−

Example k = 4, n = 7

1000 110

0100 101G =

0010 011

0001 111

4132m 3 =−−=

Syndrome decoding

Let G = [ Ik P ] then construct HT = P

In-k

For all code words c = xG, cHT = xGHT = 0

Hence, for a received noisy vector

( c ⊕ n ) HT = c HT ⊕ n HT

= n HT

= : S

Example

100 110

G = 010 101

001 011

110

101

011

100

010

001

HT =

x = 1 0 1

c = 1 0 1 1 0 1

c HT = 0 0 0

n = 0 1 0 0 0 0

c ⊕ n = 1 1 1 1 0 1

[c ⊕ n] HT = S = 1 0 1

Obvious fast decoder:

precalculate all syndromes at receiver

for correctable errors

In system form

c ⊕ nCalculate syndrome

[c ⊕ n] HT = S

Precalculated syndromes

n*

c ⊕ n

c ⊕ n ⊕ n*

when n = n*

then n ⊕ n* = 0

Reed Solomon Codes (CD, DVD)

• Structure:

m

k information symbols n-k check symbols

Properties: minimum distance = n-k+1 (symbols)

length 2m -1

General remarks

• The general problem is the decoding

• RS codes can be decoded using • Euclids algorithm• Berlekamp Massey algorithm

Why error correction?

• Systems with errors can be made almost error free

• CD, DvD would not work without RS codes