lecture4 capacity’of’wirelesschannels -...
TRANSCRIPT
What we have learned• So far: looked at specific schemes and techniques
• Lecture 2: point-to-point wireless channel- Diversity: combat fading by exploiting inherent diversity- Coding: combat noise, and further exploits degrees of freedom
• Lecture 3: cellular system- Multiple access: TDMA, CDMA, OFDMA- Interference management: orthogonalization (partial frequency
reuse), treat-interference-as-noise (interference averaging)
2
Information Theory• Is there a framework to …- Compare all schemes and techniques fairly? - Assert what is the fundamental limit on how much rate can be
reliably delivered over a wireless channel?
• Information theory!- Provides a fundamental limit to (coded) performance- Identifies the impact of channel resources on performance- Suggest novel techniques to communicate over wireless channels
• Information theory provides the basis for the modern development of wireless communication
3
Historical Perspective
4
• First radio built 100+ years ago
• Great stride in technology
• But design was somewhat ad-hoc
1901 1948
G. Marconi C. Shannon
Engineering meets scienceNew points of view arise
• Information theory: every channel has a capacity
• Provides a systematic view of all communication problems
Modern View on Multipath Fading
• Classical view: fading channels are unreliable- Diversity techniques: average out the variation
• Modern view: exploit fading to gain spectral efficiency- Thanks to the study on fading channel through the lens of
information theory!
5
11 2.1 Physical modeling for wireless channels
Figure 2.1 Channel qualityvaries over multipletime-scales. At a slow scale,channel varies due tolarge-scale fading effects. At afast scale, channel varies dueto multipath effects.
Time
Channel quality
electromagnetic field impinging on the receiver antenna. This would have tobe done taking into account the obstructions caused by ground, buildings,vehicles, etc. in the vicinity of this electromagnetic wave.1
Cellular communication in the USA is limited by the Federal Commu-nication Commission (FCC), and by similar authorities in other countries,to one of three frequency bands, one around 0.9GHz, one around 1.9GHz,and one around 5.8GHz. The wavelength ! of electromagnetic radiation atany given frequency f is given by ! = c/f , where c = 3× 108 m/s is thespeed of light. The wavelength in these cellular bands is thus a fraction of ameter, so to calculate the electromagnetic field at a receiver, the locations ofthe receiver and the obstructions would have to be known within sub-meteraccuracies. The electromagnetic field equations are therefore too complex tosolve, especially on the fly for mobile users. Thus, we have to ask what wereally need to know about these channels, and what approximations might bereasonable.One of the important questions is where to choose to place the base-stations,
and what range of power levels are then necessary on the downlink and uplinkchannels. To some extent this question must be answered experimentally, butit certainly helps to have a sense of what types of phenomena to expect.Another major question is what types of modulation and detection techniqueslook promising. Here again, we need a sense of what types of phenomena toexpect. To address this, we will construct stochastic models of the channel,assuming that different channel behaviors appear with different probabilities,and change over time (with specific stochastic properties). We will return tothe question of why such stochastic models are appropriate, but for now wesimply want to explore the gross characteristics of these channels. Let us startby looking at several over-idealized examples.
1 By obstructions, we mean not only objects in the line-of-sight between transmitter andreceiver, but also objects in locations that cause non-negligible changes in the electro-magnetic field at the receiver; we shall see examples of such obstructions later.
Plot• Use a heuristic argument (geometric) to introduce the
capacity of the AWGN channel
• Discuss the two key resources in the AWGN channel:- Power- Bandwidth
• The AWGN channel capacity serves as a building block towards fading channel capacity:- Slow fading channel: outage capacity- Fast fading channel: ergodic capacity
6
Outline• AWGN Channel Capacity
• Resources of the AWGN Channel
• Capacity of some LTI Gaussian Channels
• Capacity of Fading Channels
7
8
AWGN Channel Capacity
9
Channel Capacity• Capacity := the highest data rate can be delivered
reliably over a channel- Reliably ≡ Vanishing error probability
• Before Shannon, it was widely believed that:- to communicate with error probability → 0 - ⟹ data rate must also → 0
• Repetition coding (with M-level PAM) over N time slots on AWGN channel:- Error probability
- Data rate- As long as M ≤ N⅓ , the error probability → 0 as N → ∞- But, the data rate ! ! ! still → 0 as N → ∞
⇠ 2Q
r6N
M3SNR
!
=
log2 M
N
=
log2 N
3N
Channel Coding Theorem• For every memoryless channel, there is a definite number C
that is computable such that:- If the data rate R < C, then there exists a coding scheme that can
deliver rate R data over the channel with error probability → 0 as the coding block length N → ∞
- Conversely, if the data rate R > C, then no matter what coding scheme is used, the error probability → 1 as N → ∞
• We shall focus on the additive white Gaussian noise (AWGN) channel- Give a heuristic argument to derive the AWGN channel capacity
10
AWGN Channel
• We consider real-valued Gaussian channel• As mentioned earlier, repetition coding yield zero rate if
the error probability is required to vanish as N → ∞
• Because all codewords are spread on a single dimension in an N-dimensional space
• How to do better?
11
y[n] = x[n] + z[n]x[n]
z[n] ⇠ N (0,�2)Power constraint:
NX
n=1
|x[n]|2 NP
Sphere Packing Interpretation
12
y = x+ z
RN • By the law of large numbers, as N → ∞, most y will lie inside the N-dimensional sphere of radius
pN(P + �2)
• Also by the LLN, as N → ∞, y will lie near the surface of the N-dimensional sphere centered at x with radiusp
N�2
How many non-overlapping spheres can be packed into the large sphere?
• Vanishing error probability ⟹ non-overlapping spheres
pN(P + �2)
pN�2
Why Repetition Coding is Bad
13
y = x+ z
RN
It only uses one dimension out of N !p
N(P + �2)
Capacity Upper Bound
14
y = x+ z
RN
pN(P + �2)
pN�2
Maximum # of non-overlapping spheres = Maximum # of codewords that can be reliably delivered
=) R 1
Nlog
pN(P + �2
)
N
pN�2
N
!
=
1
2
log
✓1 +
P
�2
◆
2NR p
N(P + �2)N
pN�2
N
This is hence an upper bound of the capacity C. How to achieve it?
Achieving Capacity (1/3)• (random) Encoding: randomly generate 2NR codewords
{x1, x2, ...} lying inside the “x-sphere” of radius
• Decoding:
• Performance analysis: WLOG let x1 is sent- By the LLN,
- As long as αy lies inside the uncertainty sphere centered at x1
with radius! ! ! , decoding will be correct
- Pairwise error probability (see next slide) =
15
pNP
y �! MMSE �! ↵y �! Nearest Neighbor �! bx
↵ :=P
P + �2
||↵y � x1||2 = ||↵w + (↵� 1)x1||2 ⇡ ↵2N�2 + (↵� 1)2NP = NP�2
P + �2
rN
P�2
P + �2 ✓�2
P + �2
◆N/2
Achieving Capacity (2/3)
16
pNP
x-sphere
rN
P�2
P + �2
x1
When does an error occur?Ans: when another codeword falls inside the uncertainty sphere of x1
What is that probability (pairwise error probability)?Ans: the ratio of the volume of the two spheresPr {x1 ! x2} =
pNP�2/(P + �2)
N
pNP
N
=
✓�2
P + �2
◆N/2
Union bound: Total error probability 2NR
✓�2
P + �2
◆N/2
Achieving Capacity (3/3)• Total error probability (by union bound)
• As long as the following holds,
• Hence, indeed the capacity is
17
Pr {E} ! 0 as N ! 1
R <1
2
log
✓1 +
P
�2
◆
Pr {E} 2NR
✓�2
P + �2
◆N/2
= 2N
R+
12 log
1
1+ P�2
!!
C =
1
2
log
✓1 +
P
�2
◆bits per symbol time
18
Resources of AWGN Channel
Continuous-‐Time AWGN Channel• System parameters:- Power constraint: P watts; Bandwidth: W Hz- Spectral density of the white Gaussian noise: N0/2
• Equivalent discrete-time baseband channel (complex)
- 1 complex symbol = 2 real symbols
• Capacity:
19
Power constraint:
NX
n=1
|x[n]|2 NP
y[n] = x[n] + z[n]x[n]
z[n] ⇠ CN (0, N0W )
CAWGN(P,W ) = 2⇥ 1
2
log
✓1 +
P/2
N0W/2
◆bits per symbol time
= W log
✓1 +
P
N0W
◆bits/s
= log (1 + SNR) bits/s/Hz
SNR := P/N0WSNR per complex symbol
Complex AWGN Channel Capacity
• The capacity formula provides a high-level way of thinking about how the performance fundamentally depends on the basic resources available in the channel
• No need to go into details of specific coding and modulation schemes
• Basic resources: power P and bandwidth W
20
CAWGN(P,W ) = W log
✓1 +
P
N0W
◆bits/s
= log (1 + SNR) bits/s/Hz Spectral Efficiency
Power
• High SNR:- Logarithmic growth with power
• Low SNR:- Linear growth with power
21
173 5.2 Resources of the AWGN channel
5.2.2 Power and bandwidth
Let us ponder the significance of the capacity formula (5.12) to a communica-tion engineer. One way of using this formula is as a benchmark for evaluatingthe performance of channel codes. For a system engineer, however, the mainsignificance of this formula is that it provides a high-level way of thinkingabout how the performance of a communication system depends on the basicresources available in the channel, without going into the details of specificmodulation and coding schemes used. It will also help identify the bottleneckthat limits performance.
The basic resources of the AWGN channel are the received power P andthe bandwidth W . Let us first see how the capacity depends on the receivedpower. To this end, a key observation is that the function
f!SNR" #= log!1+ SNR" (5.14)
is concave, i.e., f ′′!x"≤ 0 for all x≥ 0 (Figure 5.4). This means that increasingthe power P suffers from a law of diminishing marginal returns: the higherthe SNR, the smaller the effect on capacity. In particular, let us look at thelow and the high SNR regimes. Observe that
log2!1+x" ≈ x log2 e whenx ≈ 0$ (5.15)
log2!1+x" ≈ log2 x whenx≫ 1% (5.16)
Thus, when the SNR is low, the capacity increases linearly with the receivedpower P: every 3 dB increase in (or, doubling) the power doubles the capacity.When the SNR is high, the capacity increases logarithmically with P: every3 dB increase in the power yields only one additional bit per dimension.This phenomenon should not come as a surprise. We have already seen in
Figure 5.4 Spectral efficiencylog!1+ SNR" of the AWGNchannel.
0
3
4
5
6
7
0 20 40 60 80 100
1
2
SNR
log (1 + SNR)
C = log(1 + SNR) ⇡ log SNR
C = log(1 + SNR) ⇡ SNR log2 e
SNR =P
N0W
Fix W:
Bandwidth
22
174 Capacity of wireless channels
Chapter 3 that packing many bits per dimension is very power-inefficient.The capacity result says that this phenomenon not only holds for specificschemes but is in fact fundamental to all communication schemes. In fact,for a fixed error probability, the data rate of uncoded QAM also increaseslogarithmically with the SNR (Exercise 5.7).
The dependency of the capacity on the bandwidth W is somewhat morecomplicated. From the formula, the capacity depends on the bandwidth in twoways. First, it increases the degrees of freedom available for communication.This can be seen in the linear dependency on W for a fixed SNR= P/!N0W ".On the other hand, for a given received power P, the SNR per dimensiondecreases with the bandwidth as the energy is spread more thinly across thedegrees of freedom. In fact, it can be directly calculated that the capacity isan increasing, concave function of the bandwidth W (Figure 5.5). When thebandwidth is small, the SNR per degree of freedom is high, and then thecapacity is insensitive to small changes in SNR. Increasing W yields a rapidincrease in capacity because the increase in degrees of freedom more thancompensates for the decrease in SNR. The system is in the bandwidth-limitedregime. When the bandwidth is large such that the SNR per degree of freedomis small,
W log!1+ P
N0W
"≈W
!P
N0W
"log2 e=
P
N0log2 e# (5.17)
In this regime, the capacity is proportional to the total received power acrossthe entire band. It is insensitive to the bandwidth, and increasing the bandwidthhas a small impact on capacity. On the other hand, the capacity is now linearin the received power and increasing power has a significant effect. This isthe power-limited regime.
Figure 5.5 Capacity as afunction of the bandwidth W .Here P/N0 = 106.
305Bandwidth W (MHz)
Capacity
Limit for W → ∞
Power limited region
0.2
1
Bandwidth limited region
(Mbps)C(W )
0.4
252015100
1.6
1.4
1.2
0.8
0.6
0
PN0
log2 e
C(W ) = W log
✓1 +
P
N0W
◆⇡ W
P
N0Wlog2 e =
P
N0log2 eFix P
Bandwidth-‐limited vs. Power-‐limited
• When SNR ≪ 1: (Power-limited regime)
- Linear in power; Insensitive to bandwidth
• When SNR ≫ 1: (Bandwidth-limited regime)
- Logarithmic in power; Approximately linear in bandwidth
23
CAWGN(P,W ) = W log
✓1 +
P
N0W
◆bits/s
SNR =P
N0W
CAWGN(P,W ) ⇡ W
✓P
N0W
◆log2 e =
P
N0log2 e
CAWGN(P,W ) ⇡ W log
✓P
N0W
◆
24
Capacity of SomeLTI Gaussian Channels
25
SIMO Channel
• MRC is a lossless operation: we can generate y from :
• Hence the SIMO channel capacity is equal to the capacity of the equivalent AWGN channel, which is
hx
y
ey = ||h||x+ ew, ew ⇠ CN�0,�2
�
#
MRC,h⇤
||h||#
y = hx+w 2 CLPower constraint: P
w ⇠ CN�0,�2IL
�
eyy = ey (h/||h||)
CSIMO = log
✓1 +
||h||2P�2
◆Power gain due to Rx beamforming
MISO Channel
• Goal: maximize the received power ||h*x||2
- The answer is ||h||2P ! (check. Hint: Cauchy–Schwarz inequality)
• Achieved by Tx beamforming- Send a scalar symbol x on the direction of h- Power constraint on x : still P
• Capacity:
26
hx
y
y = h
⇤x+ w 2 C, x,h 2 CL
h⇤ =⇥h1 h2
⇤y = x||h||+ w
#Tx Beamforming
x = xh/||h||#
Power constraint:
NX
n=1
||x||2 NP
CMISO = log
✓1 +
||h||2P�2
◆
Frequency-‐Selective Channel
• Key idea 1: use OFDM to convert the channel with ISI into a bunch of parallel AWGN channels- But there is loss/overhead due to cyclic prefix
• Key idea 2: CP overhead → 0 as Nc → ∞
• First focus on finding the capacity of parallel AWGN channels of any finite Nc
• Then take Nc → ∞ to find the capacity of the frequency-selective channel
27
y[m] =L�1X
l=0
hlx[m� l] + w[m]
Recap: OFDM
28
99 3.4 Frequency diversity
d[N–1]
y0
x [N + L – 1] = d[N – 1]
Cyclic prefix
y [N + L – 1]dN–1
IDFT DFT
Remove prefix
yN–1
y[L]
y[N + L – 1]
y[1]
y[L – 1]y[L]
x [L – 1] = d[N – 1]x [L] = d[0]
x [1] = d[N – L + 1]
d0 d[0]Channel
This representation suggests a natural rotation at the input and at the outputFigure 3.22 The OFDMtransmission and receptionschemes.
to convert the channel to a set of non-interfering channels with no ISI.In particular, the actual data symbols (denoted by the length Nc vector d)in the frequency domain are rotated through the IDFT (inverse DFT) matrixU−1 to arrive at the vector d. At the receiver, the output vector of lengthNc (obtained by ignoring the first L symbols) is rotated through the DFTmatrix U to obtain the vector y. The final output vector y and the actual datavector d are related through
yn = hndn+ wn! n= 0! " " " !Nc−1# (3.145)
We have denoted w $= Uw as the DFT of the random vector w and we seethat since w is isotropic, w has the same distribution as w, i.e., a vector ofi.i.d. !" %0!N0& random variables (cf. (A.26) in Appendix A).These operations are illustrated in Figure 3.22, which affords the following
interpretation. The data symbols modulate Nc tones or sub-carriers, whichoccupy the bandwidth W and are uniformly separated by W/Nc. The datasymbols on the sub-carriers are then converted (through the IDFT) to timedomain. The procedure of introducing the cyclic prefix before transmissionallows for the removal of ISI. The receiver ignores the part of the output signalcontaining the cyclic prefix (along with the ISI terms) and converts the lengthNc symbols back to the frequency domain through a DFT. The data symbolson the sub-carriers are maintained to be orthogonal as they propagate throughthe channel and hence go through narrowband parallel sub-channels. Thisinterpretation justifies the name of OFDM for this communication scheme.Finally, we remark that DFT and IDFT can be very efficiently implemented(using Fast Fourier Transform) whenever Nc is a power of 2.
OFDM block lengthThe OFDM scheme converts communication over a multipath channel intocommunication over simpler parallel narrowband sub-channels. However, thissimplicity is achieved at a cost of underutilizing two resources, resulting ina loss of performance. First, the cyclic prefix occupies an amount of timewhich cannot be used to communicate data. This loss amounts to a fraction
y := y[L : Nc + L� 1], w := w[L : Nc + L� 1],
h :=⇥h0 h1 · · · hL�1 0 · · · 0
⇤T
eyn = ehnedn + ewn, n = 0, 1, . . . , Nc � 1
eyn := DFT (y)n ,edn := DFT (d)n , ewn := DFT (w)n ,
ehn :=pNcDFT (h)n
Nc parallel AWGN channels
Parallel AWGN Channels
29
ed0[m]
eh0
ey0[m]
ew0[m]
...
ed1[m]
eh1
ey1[m]
ew1[m]
edNc�1[m]
ehNc�1 ewNc�1[m]
eyNc�1[m]
Parallel Channels
Equivalent Vector Channeley = eHed+ eweH = diag
⇣eh0, . . . ,ehNc�1
⌘
eyn = ehnedn + ewn, n 2 [0 : 1 : Nc � 1]
Power Constraint
m = 1, 2, . . . ,M
(M channel uses)
MX
n=1
||ed[n]||2 MNcP
Due to Parseval theorem of DFT
ew ⇠ CN�0,�2I
�
Independent Uses of Parallel Channels• One way to code over such parallel channels (a special
case of a vector channel): treat each channel separately- It turns out that coding across parallel channels does not help!
• Power allocation: - Each of the Nc channels get a portion of the total power
- Channel n gets power Pn , which must satisfy
• For a given power allocation {Pn}, the following rate can be achieved:
30
Nc�1X
n=0
Pn NcP
R =
Nc�1X
n=0
log
1 +
|ehn|2Pn
�2
!
Optimal Power Allocation• Power allocation problem:
• It can be solved explicitly by Lagrangian methods• Final solution: let (x)+ := max(x , 0)
31
max
P0,...,PNc�1
Nc�1X
n=0
log
1 +
|ehn|2Pn
�2
!,
subject to
Nc�1X
n=0
Pn = NcP, Pn � 0, n = 0, . . . , Nc � 1
P ⇤n =
⌫ � �2
|ehn|2
!+
, ⌫ satisfiesNc�1X
n=0
⌫ � �2
|ehn|2
!+
= NcP
Water]illing
32
�2
|ehn|2
⌫
Note:
ehn = Hb
✓nW
Nc
◆
Baseband frequency response at f = nW/Nc
184 Capacity of wireless channels
Figure 5.11 Waterfilling powerallocation over the Nc sub-carriers.
P1 = 0
N0
|H( f )|2
Subcarrier n
P2
P3
*
*
*
1λ
Observe that
hn =L−1!
ℓ=0
hℓ exp"− j2"ℓn
Nc
## (5.45)
is the discrete-time Fourier transform H$f % evaluated at f = nW/Nc, where(cf. (2.20))
H$f % &=L−1!
ℓ=0
hℓ exp"− j2"ℓf
W
## f ∈ '0#W() (5.46)
As the number of sub-carriers Nc grows, the frequency width W/Nc of thesub-carriers goes to zero and they represent a finer and finer sampling of thecontinuous spectrum. So, the optimal power allocation converges to
P∗$f %="1*− N0
$H$f %$2#+
# (5.47)
where the constant * satisfies (cf. (5.44))
$ W
0P∗$f %df = P) (5.48)
The power allocation can be interpreted as waterfilling over frequency (seeFigure 5.12). With Nc sub-carriers, the largest reliable communication rate
Frequency-‐Selective Channel Capacity• Final step: making Nc → ∞- Replace all ! ! ! ! ! by Hb(f), summation over [0 : Nc – 1]
becomes integration from 0 to W
• Power allocation problem becomes
• Optimal solution becomes
33
ehn = Hb
✓nW
Nc
◆
max
P (f)
Z W
0log
✓1 +
|H(f)|2P (f)
�2
◆df,
subject to
Z W
0P (f) = P, P (f) � 0, f 2 [0,W ]
P (f)⇤ =
✓⌫ � �2
|H(f)|2
◆+
, ⌫ satisfies
Z W
0
✓⌫ � �2
|H(f)|2
◆+
df = P
Water]illing over the Frequecy Spectrum
34
185 5.3 Linear time-invariant Gaussian channels
Figure 5.12 Waterfilling powerallocation over the frequencyspectrum of the two-tapchannel (high-pass filter):h!0"= 1 and h!1"= 0#5.
P ( f )
Frequency ( f )
0.4W0.2W0– 0.2W– 0.4W
4
0
3.5
3
2.5
2
1.5
1
0.5
N0
|H( f )|2
*
1λ
with independent coding is CNcbits per OFDM symbol or CNc
/Nc bits/s/Hz(CNc
given in (5.39)). So as Nc →", the WCNc/Nc converges to
C =! W
0log"1+ P∗!f "$H!f "$2
N0
#df bits/s# (5.49)
Does coding across sub-carriers help?So far we have considered a very simple scheme: coding independently overeach of the sub-carriers. By coding jointly across the sub-carriers, presumablybetter performance can be achieved. Indeed, over a finite block length, codingjointly over the sub-carriers yields a smaller error probability than can beachieved by coding separately over the sub-carriers at the same rate. However,somewhat surprisingly, the capacity of the parallel channel is equal to thelargest reliable rate of communication with independent coding within eachsub-carrier. In other words, if the block length is very large then coding jointlyover the sub-carriers cannot increase the rate of reliable communication anymore than what can be achieved simply by allocating power and rate overthe sub-carriers but not coding across the sub-carriers. So indeed (5.49) is thecapacity of the time-invariant frequency-selective channel.
To get some insight into why coding across the sub-carriers with largeblock length does not improve capacity, we turn to a geometric view. Considera code, with block length NcN symbols, coding over all Nc of the sub-carrierswith N symbols from each sub-carrier. In high dimensions, i.e., N ≫ 1, theNcN -dimensional received vector after passing through the parallel channel(5.33) lives in an ellipsoid, with different axes stretched and shrunk by thedifferent channel gains hn. The volume of the ellipsoid is proportional to
Nc−1$
n=0
%$hn$2Pn+N0
&N$ (5.50)
⌫
�2
|H(f)|2