struggle ! two practical classes of channel (error- correcting) codes cyclic codes...
TRANSCRIPT
struggle !
two practical classes of channel (error- correcting) codescyclic codes (巡回符号)
a subclass of linear codeslinear-time encoding and error detection (not correction)
convolutional codes (畳み込み符号)different principle from linear codessoft-decision optimal decoding is available
both codes widely used todayexpected to be replaced by “next-generation” codes
1
what’s wrong with linear codes?
Textbooks say thatlinear codes are a practical class of channel codes,but they are not fully practical ...
not scalable complexity for encoding/decoding,
due to matrix computations
not easy to constructgenius idea or mathematical talent is needed
2
channel codes evolution
3
general channel (error-correcting) codes
linear codes(linear block codes) convolutional codes
cyclic codes
LDPC codes turbo codes
BCHReed-Solomon
HammingGolay
more structure ...
more efficiency...
more power...
more power... more power...
next class next class
this class(& next class)
cyclic codes
Linear codes are characterized by matrix operations.Encoders/decoders are realized by combinatorial circuits.
simple but not scalable ... for the code length
cyclic codesa subclass of linear codesEncoders/decoders are realized by shift registers.
slightly complicated but scalable ... complexity
4
codes linearcodes
cyclic codes
preliminary (1)
Binary vectors can be written as binary polynomials (多項式) .
5
11101 x4 + x3 + x2 + x + 1
addition (=subtraction) ⇒ XOR
x4 + x3 + x2 + 1 x3 + x + 1+)
x4 + x2 + x
111010101110110
+)
multiplication ⇒ (unnamed operation)
x4 + x3 + x2 + 1 x3 + x + 1×)x4 + x3 + x2 + 1
x5 + x4 + x3 + xx7 + x6 + x5 + x3
x7 + x6 + x3 + x2+ x + 1 11001111
1110101011×)11101
1110111101
multiplication with xm
= left-shift of m bits
preliminary (2)
6
division (unnamed operation)⇒
x6 + x4x4 + x3 + x2 + 1 )x2 + x + 1
x6 + x5 + x4 + x2
x5 + x2
x5 + x4 + x3 + xx4 + x3 + x2 + xx4 + x3 + x2 + 1
x + 1
11101 ) 101000011101
1001011101
1111011101
11
111
the division circuit is easily implemented by a shift register
1. store p(x) to the registers2. if MSB = 1, then AND gates are activated,
and registers are XOR’ed with q(x)3. left-shift, and go to step 1
division circuit (1)
divide p(x) = x6 + x4 by q(x) = x4 + x3 + x2 + 1:
7
1 1 1 0 1
1 0 1 0 0 0 0
q(x)
p(x)
quotient( 商 ) remainder ( 剰余 )
𝑝 (𝑥 ) ÷𝑞 (𝑥)M
SBdivisor
dividend
division circuit (2)
8
0 1 0 0 1
1 1 1 0 1
0 0
11101 ) 101000011101010010
111011111011101
11
111
1 0 1 0 0
1 1 1 0 1
0 0
one-cycle of operation= one-step of the calculuscontents of the register= remainder of the division complexity
definition of cyclic codes
How to construct an cyclic code: bit codeword, information bits, parity bits
Step 1: choose an degree-m polynomial G(x) that divides xn + 1.Step 2: code C = {multiples of G(x) with degree < n}
Example: n = 7, m = 4:choose G(x) = x4 + x3 + x2 + 1,because x7+1 = (x3 + x2 + 1) (x4 + x3 + x2 + 1).
G(x)...generator polynomial
9
1000000111101 )
111011101001
11101
1110111101
0
1101
G(x)x7+1
construction of a cyclic code
Step 1: choose an degree-m polynomial G(x) that divides xn + 1.Step 2: C = {multiples of G(x) with degree < n}
n = 7, m = 4: G(x) = x4 + x3 + x2 + 1
10
G(x)(x2+x+0)×
C=
00000000011101011101001001111110100110100110011101010011
G(x)(x2+x+1)×G(x)(x2+x+1)×G(x)(x2+x+1)×G(x)(x2+x+1)×G(x)(x2+x+1)×G(x)(x2+x+1)×G(x)(x2+x+1)×
=
properties of cyclic codes (1)
Anyway, we have defined a set of vectors, but is it a linear code?
Lemma: a cyclic code is a linear code
proof: Show that c1+c2C for any c1, c2C;
c1 C c1 = f1(x)G(x)
c2 C c2 = f2(x)G(x)
c1+c2 = (f1(x)+f2(x)) G(x) C
11
properties of cyclic codes (2)
Lemma: if (an-1, an-2, ..., a0)C, then (an-2, ..., a0, an-1)C
proof:Let W(x) = an-1xn-1 + ... + a0 and W’(x) = an-2xn-1 + ... + a0x + an-1.
W(x) is a multiple of G(x), because (an-1, an-2, ..., a0)C
W’(x) = an-2xn-1 + ... + a0x + an-1
= an-1xn + an-2xn-1 + ... + a0x + an-1+ an-1xn
= xW(x) + an-1(xn + 1)
12
multiple of G(x) multiple of G(x) ... construction step 1
W’(x) is a multiple of G(x), and (an-2, ..., a0, an-1)C.
A cyclic code C is closed for a cyclic shift.
three approaches for encoding
three approaches for an encoding procedure:matrix approach
use a generator matrix ... no advantage of cyclic codesmultiplication approach
codeword = (information bits) × G(x)the code not systematic (cf. p.10)
division approachslightly complicated (for me)make the code systematiceasily implemented by shift registers
13
3-step encoding by division
1. A(x) = polynomial of information bits2. B(x) = remainder of A(x)xm divided by G(x)3. codeword = A(x)xm + B(x)
encode 011, with n = 7, k = 3, m = 4, G(x) = x4 + x3 + x2 + 1 1. A(x) = x + 12. A(x)x4 = x5 + x4 = x(x4 + x3 + x2 + 1) + (x3 + x), and B(x) = x3 + x3. A(x)x4 + B(x)= x5 + x4 + x3 + x, the codeword = 0111010
14
A(x)
(not necessary)G(x)
B(x)
x4
÷divisor
dividend
quotient
remainder
W(x) = A(x)x4 + B(x)
multiply,divide,
and add
did we really make “encoding”?
simple question: Is A(x)xm + B(x) really a codeword? Is A(x)xm + B(x) divided by
G(x)?
Yes, note that...B(x) is a remainder of A(x)xm
in the binary world, A(x)xm + B(x) = A(x)xm – B(x)“A(x)xm – B(x)” = “removing the remainder”
if A(x)xm – B(x) is divided by G(x), then there is no remainder...
15
example
n = 7, k = 3, m = 4, G(x) = x4 + x3 + x2 + 1
16
data000001010011100101110111
A(x)x2+x+0x2+x+1x2+x+1x2+x+1x2+x+1x2+x+1x2+x+1x2+x+1
A(x)x4
00000000010000010000001100001000000101000011000001110000
B(x)x3+x2+x+0x3+x2+x+1x3+x2+x+1x3+x2+x+1x3+x2+x+1x3+x2+x+1x3+x2+x+1x3+x2+x+1
A(x)xm + B(x) 00000000011101010011101110101001110101001111010011110100
systematic codeencoder ≈ division circuit O(n) < O(n2) of matrix operation
error “detection” of cyclic codes
error “detection” is easy for cyclic codesu C u (in a polynomial representation) is divided by G(x)
17
G(x)
= 0 ... no error 0 ... error÷divisor
dividend
quotient
remainderreceived u
no need of parity check matrix, division circuit sufficesencoding and error detection can share one division circuit
reduces the cost of realizationCyclic Redundancy Check (CRC)
... used in many communication systems
error “correction” of cyclic codes
general algorithm for all cyclic codeserror-trapping decoder [Kasami 61]
18
Tadao Kasami1930-2007
E. Berlekamp,1940-
J. L. Massey,1934-2013
I. Reed, 1923-2012 (left)andG. Solomon,1930-1996
special algorithms for special cyclic codesBerlekamp-Massey algorithm for...
BCH (Bose-Chaudhuri-Hocquenghem) codesReed-Solomon Codes
intermission
and
convolutional code
19
the channel and modulation
We have considered “digital channels”.At the physical-layer,
almost all channels are continuous.digital channel=modulator + continuous channel + demodulator
20
modulator( 変調器 )
demodulator( 復調器 )
a naive demodulator translates the waveform to 0 or 1 we are losing something...
0010 0010
0010 0010
continuous(analogue)
“more informative” demodulator
From the viewpoint of error correction, the waveform contains more information than the binary output of the demodulator.
demodulators with multi-level output can help error correction.
21
0
1
definitely 0maybe 0
definitely 1maybe 1
to make use of this multi-level demodulator,the decoding algorithm must be able to handle multi-level inputs
hard-decision vs. soft-decision
hard-decision decodingthe input to the decoder is binary (0 or 1)decoding algorithms discussed so far are hard-decision type
soft-decision decodingthe input to the decoder can have three or more levelsthe “check matrix and syndrome” approach does not workthe use of polynomials is not obvious
complicated, but should have more power
22
soft-decision decoding as optimization problem
outputs of the demodulator0+ (definitely 0), 0- (maybe 0), 1- (maybe 1), 1+ (definitely 1)
code C = {00000, 01011, 10101, 11110}for a received vector 0- 0+ 1+ 0- 1-,
find a codeword which minimizes the penalty .
23
received0+
0-
1-
1+
penalty of “0”
0123
penalty of “1”
3210
(hard-decision ... penalty = Hamming distance)
0- 0+ 1+ 0- 1-
0 0 0 0 01 0 3 1 2 7
r
c0+ + + + =
1 0 1 0 12 0 0 1 1 4+ + + + =
c2
=
=
smaller penalty,more likely
algorithms for the soft-decision decoding
We just formalized the problem... how can we solve it?by exhaustive search?
... not practical for codes with many codewordsby matrix operation?
... need to solve an integer programming, NP-hardby approximation?
... yes, this is one practical approach anyway...
design a special codefor which soft-decision decoding is not “too difficult”
convolutional code ( 畳み込み符号 )
24
convolutional codes
the codes we studied so far... block codesa block of k-bit data is encoded to a codeword of length nthe encoding is done independently for block to block
convolutional codesencoding is done in a bit-by-bit mannerprevious inputs are stored in shift-registers in the encoder, and affects future encoding
25
input data
combinatorial logicencoder outputs
encoding of a convolutional code
at the beginning, the contents of registers are all 0when a data bit is given, the encoder outputs several bits,
and the contents of registers are shifted by one-bitafter encoding, give 0’s until all registers are filled by 0
26
r3 r2 r1
encoder exampleconstraint length = 3 ( = # of registers)the output is constrained by three previous input bits
encoding example (1)
to encode 1101...
27
0 0 0 1
11 0 0 1 1
10
0 1 1 0
01 1 1 0 1
10
give additional 0’s to push-out 1’s in the register...
encoding example (2)
28
1 0 1 0
00 0 1 0 0
00
1 1 0 1
10
1 0 0 0
01
the output is 11 10 01 10 00 00 01
encoder as a finite-state machine
An encoder with k registers has 2k internal states.An input causes a state transition, accompanied with outputs
29
r2 r1
inpu
t
outp
ut
internal state = (r2, r1)
0123
(0,0)(0,1)(1,0)(1,1)
(r2, r1)0
1 2
3
0 / 00
1 / 11
1 / 11
0 / 00
1 / 100 / 01
0 / 01
1 / 10
input / output
constraint: initial state = final state = 0
encoding = state transition
30
0 0 111
0 1 111
1 1 001
1 0 001
input 0input 1
2
3
00
11
11
00
10 01
01
10
0
1
00
11
11
00
10 01
01
10
1
3
00
11
11
00
10 01
01
10
0
2
00
11
11
00
10 01
01
10
0 0
at the receiver’s end
the receiver knows...the state diagramthe initial state and the final statethe transmitted sequence but corrupted by errors
31
encoder receiver
01001... 01100...
errors
to correct errors = to estimate the most-likely transition
trellis diagram
expand the state diagram to the time axis...
32
0
1
2
3
0
1
2
3
0011
10
01
11
00
1001
0
1
2
3
0
1
2
3
0
1
2
3
trellis diagram time 0 1 2 3 4
initial state final state
input 0input 1
0
1
00
11
11
00
10 01
01
10
2
3
trellis diagram and code
33
possible encoding sequences= paths connecting the initial state and the final state
the transmitted sequence = the path with the minimum penalty
10
0
1
2
3
0
1
2
3
0011
10
01
11
00
01
0
1
2
3
0
1
2
3
0
1
2
3
error correction = shortest path problem
Viterbi algorithm
given a received sequence...the demodulator defines penalties for symbols at each positionthe penalties are assigned to edges of the trellis diagramfind the path with the minimum penalty using a good algorithm
Viterbi algorithmDijkstra algorithm over a trellis diagramrecursive width-first search
34
Andrew Viterbi1935-
s0,0
pA
pB
qA
qB
the minimum penaltyof this state ismin (pA+qA, pB+qB)
soft-decision decoding for convolutional codes
the complexity of Viterbi algorithm ≈ the size of the trellis diagram
for convolutional codes (with constraint length k):the size of trellis ≈ 2k×data length ... manageablewe can extract 100% performance of the code
for block codes:the size of trellis ≈ 2data length ... too largeit is difficult to extract the full performance
35
block code high potential,but difficult to use
performance
convolutionalcode
moderated potential,full power available
summary
cyclic codesscalable, and have good mathematical structure
some good codes have been discoverederror-correction is not straight-forward
convolutional codessoft-decision decoding is practically realizableno good algorithm for constructing good codes
design in the “trial-and-error” manner with computer
still used widely, but “better” codes are studied recently...
36