1 00:00:01,070 --> 00:00:03,550 PROFESSOR: There is one handout being passed out, it's 2 00:00:03,550 --> 00:00:07,360 chapters four and five, so be sure to pick it up. 3 00:00:07,360 --> 00:00:10,380 And just a reminder that there is a homework due on next 4 00:00:10,380 --> 00:00:13,500 Wednesday, so we have homeworks due every week in 5 00:00:13,500 --> 00:00:16,560 this course as well. 6 00:00:16,560 --> 00:00:19,010 So last class, we covered Chapters One through Three 7 00:00:19,010 --> 00:00:22,240 quite quickly rather, because it was mostly a review of 8 00:00:22,240 --> 00:00:27,350 6.450, and one of the key ideas we covered last class 9 00:00:27,350 --> 00:00:29,810 was the connection between continuous time and discrete 10 00:00:29,810 --> 00:00:31,060 time systems. 11 00:00:33,470 --> 00:00:40,756 So we have continuous time, discrete time. 12 00:00:46,180 --> 00:00:49,560 A specific example that we saw was that of an 13 00:00:49,560 --> 00:00:50,930 orthonormal PAM system. 14 00:01:02,480 --> 00:01:07,660 The architecture for a PAM system is as follows: you have 15 00:01:07,660 --> 00:01:12,000 Xk, a sequence of symbols coming in, they 16 00:01:12,000 --> 00:01:13,640 get into a PAM modulator. 17 00:01:21,530 --> 00:01:25,870 What you get out is a wave form, X of t, is what the PAM 18 00:01:25,870 --> 00:01:28,960 modulator produces. 19 00:01:28,960 --> 00:01:34,400 You have a little noise on the channel, and what you receive 20 00:01:34,400 --> 00:01:36,530 out is Y of t. 21 00:01:36,530 --> 00:01:40,430 So Y of t is the wave form that the receiver receives, 22 00:01:40,430 --> 00:01:45,310 and what a canonical receiver's structure to a 23 00:01:45,310 --> 00:01:49,245 matched filter followed by sampling. 24 00:01:52,330 --> 00:01:57,710 So what you get out is a sequence of symbols Y of t. 25 00:01:57,710 --> 00:02:02,610 So this is a structure for an orthonormal PAM modulator, and 26 00:02:02,610 --> 00:02:06,640 the continuous time version of the channel is that you have y 27 00:02:06,640 --> 00:02:09,410 of p, which is received at the receiver, plus X of 28 00:02:09,410 --> 00:02:10,764 t, plus N of t. 29 00:02:19,790 --> 00:02:26,820 The discrete time model is you have the symbol Y of k as the 30 00:02:26,820 --> 00:02:29,180 output of the sampler. 31 00:02:29,180 --> 00:02:33,310 It equals X of k plus N of k. 32 00:02:36,130 --> 00:02:39,800 Now basically, we say that the two systems are equivalent in 33 00:02:39,800 --> 00:02:43,415 the following sense: if you want to make a detection of X 34 00:02:43,415 --> 00:02:47,890 of k here, at the receiver, Y of k is a sufficient 35 00:02:47,890 --> 00:02:52,100 statistic, given that Y of t is received at the front end 36 00:02:52,100 --> 00:02:53,670 of the receiver. 37 00:02:53,670 --> 00:02:55,750 So that's the equivalence between discrete time and 38 00:02:55,750 --> 00:02:57,000 continuous time. 39 00:03:20,390 --> 00:03:23,140 And the way we established this fact was by using the 40 00:03:23,140 --> 00:03:25,530 theorem of irrelevance. 41 00:03:25,530 --> 00:03:28,860 The noise here is white Gaussian noise, so if we 42 00:03:28,860 --> 00:03:32,180 project it onto orthonormal wave forms, the corresponding 43 00:03:32,180 --> 00:03:34,185 noise samples would be IID. 44 00:03:34,185 --> 00:03:36,690 The noise will be independent of everything which is out of 45 00:03:36,690 --> 00:03:39,980 band, and so there is no correlation among the noise 46 00:03:39,980 --> 00:03:43,100 samples, and Y of k is a sufficient statistic 47 00:03:43,100 --> 00:03:44,490 to detect X of k. 48 00:03:52,620 --> 00:03:55,060 So that is the basic idea. 49 00:03:55,060 --> 00:03:57,520 Now, a similar architecture also holds when your 50 00:03:57,520 --> 00:04:00,140 continuous time system operates in fast band rather 51 00:04:00,140 --> 00:04:03,560 than base band, except the main difference now is instead 52 00:04:03,560 --> 00:04:07,390 of a PAM modulation, you have a QAM modulator, and your 53 00:04:07,390 --> 00:04:10,960 symbols, X of k and N of k, will be complex numbers rather 54 00:04:10,960 --> 00:04:13,160 than the real numbers. 55 00:04:13,160 --> 00:04:15,870 Now the connection between continuous time and discrete 56 00:04:15,870 --> 00:04:19,519 time systems can be made more precise by relating some 57 00:04:19,519 --> 00:04:23,930 parameters in continuous time with those in discrete time. 58 00:04:23,930 --> 00:04:24,530 OK? 59 00:04:24,530 --> 00:04:33,590 So we have continuous time parameters, and discrete time 60 00:04:33,590 --> 00:04:34,840 parameters. 61 00:04:41,020 --> 00:04:50,950 So a continuous time parameter is a bandwidth, W. A discrete 62 00:04:50,950 --> 00:05:00,505 time parameter is given by symbol interval T, and the 63 00:05:00,505 --> 00:05:03,170 relation is T equals 1 over 2W. 64 00:05:03,170 --> 00:05:04,420 This is for a PAM system. 65 00:05:08,070 --> 00:05:08,530 OK? 66 00:05:08,530 --> 00:05:12,600 So something is shown in 6.450, and it's shown by using 67 00:05:12,600 --> 00:05:14,630 Nyquist's ISI criteria. 68 00:05:14,630 --> 00:05:18,040 Do not want to have ISI in the system, the maximum symbol 69 00:05:18,040 --> 00:05:20,410 rate, or rather the minimum symbol rate, 70 00:05:20,410 --> 00:05:21,910 would be 1 over 2W. 71 00:05:21,910 --> 00:05:25,920 You cannot send symbols faster than this rate. 72 00:05:25,920 --> 00:05:30,630 A second criteria is power, which is P here, 73 00:05:30,630 --> 00:05:32,600 in continuous time. 74 00:05:32,600 --> 00:05:37,930 In discrete time the equivalent parameter is energy 75 00:05:37,930 --> 00:05:39,580 per two dimensions. 76 00:05:43,800 --> 00:05:49,420 It's denoted by Es, and Es is related to P by using Es 77 00:05:49,420 --> 00:05:51,130 equals 2 times PT. 78 00:05:51,130 --> 00:05:55,340 Note that this is for two dimensions, meaning in PAM we 79 00:05:55,340 --> 00:06:00,040 have one symbol per dimension, so this is for two symbols. 80 00:06:00,040 --> 00:06:03,690 This is energy for two symbols for a PAM modulator, and the 81 00:06:03,690 --> 00:06:06,090 relation is Es equals 2 times PT. 82 00:06:06,090 --> 00:06:08,915 T is the time for the one symbol. 83 00:06:08,915 --> 00:06:11,950 We also -- we are looking at energy for two symbols, so we 84 00:06:11,950 --> 00:06:13,200 multiply P by 2T. 85 00:06:16,470 --> 00:06:18,900 OK? 86 00:06:18,900 --> 00:06:25,560 Noise in continuous time is AWGN, for Additive White 87 00:06:25,560 --> 00:06:30,800 Gaussian Noise process, and the power spectral density is 88 00:06:30,800 --> 00:06:35,110 flat on positive support of bandwidth, and 89 00:06:35,110 --> 00:06:37,890 the height is N_0. 90 00:06:37,890 --> 00:06:40,070 OK, so this is how the spectral density is. 91 00:06:40,070 --> 00:06:45,740 In 6.450, we looked at double-sided power spectral 92 00:06:45,740 --> 00:06:48,980 density when the height was N_0 over two, but it was going 93 00:06:48,980 --> 00:06:50,660 over both the positive and 94 00:06:50,660 --> 00:06:53,200 negative part of the bandwidth. 95 00:06:53,200 --> 00:06:55,180 Here, we are only looking at the positive part of the 96 00:06:55,180 --> 00:06:57,730 bandwidth, and we are going to scale noise by N_0. 97 00:06:57,730 --> 00:06:58,980 This is just a convention. 98 00:07:02,760 --> 00:07:09,780 In discrete time, your noise sequence is Nk, and is an IID 99 00:07:09,780 --> 00:07:14,090 and you take them as Gaussians with zero mean, and variance 100 00:07:14,090 --> 00:07:15,930 of N_0 over 2. 101 00:07:15,930 --> 00:07:17,530 So we have noise which has a variance of 102 00:07:17,530 --> 00:07:20,220 N_0 over 2 per dimension. 103 00:07:20,220 --> 00:07:23,360 And this was, again, shown by this theorem that when you 104 00:07:23,360 --> 00:07:28,040 project noise onto each of the orthonormal wave forms, you 105 00:07:28,040 --> 00:07:32,155 get the variance is N_0 over 2. 106 00:07:32,155 --> 00:07:33,990 OK? 107 00:07:33,990 --> 00:07:38,990 Instead of a base band system, if you had a fast band system, 108 00:07:38,990 --> 00:07:42,000 then instead of having a PAM modulator, we 109 00:07:42,000 --> 00:07:44,280 would have a QAM modulator. 110 00:07:44,280 --> 00:07:48,330 So if instead of PAM we had a QAM then the main difference 111 00:07:48,330 --> 00:07:50,350 now would be that these symbols are going to be 112 00:07:50,350 --> 00:07:53,310 complex numbers instead of real. 113 00:07:53,310 --> 00:07:57,540 So what's the symbol interval now going to be? 114 00:07:57,540 --> 00:07:58,500 AUDIENCE: 1 over W. 115 00:07:58,500 --> 00:08:02,070 PROFESSOR: It's going to be 1 over W. 116 00:08:02,070 --> 00:08:04,630 Instead of sending a real symbol, we are sending one 117 00:08:04,630 --> 00:08:08,370 complex symbol, which is occupying two dimensions, and 118 00:08:08,370 --> 00:08:12,770 our symbol rate is going to be 1 over W. Well, the energy for 119 00:08:12,770 --> 00:08:18,410 two dimensions, is still given by Es equals 2PT, or 120 00:08:18,410 --> 00:08:23,570 equivalently, P over W. OK? 121 00:08:23,570 --> 00:08:24,820 Or that's -- 122 00:08:32,200 --> 00:08:38,080 The noise samples Nk are still IID, but now they are complex 123 00:08:38,080 --> 00:08:44,240 Gaussians, with zero mean, and variance N_0. 124 00:08:44,240 --> 00:08:45,790 Or equivalently, they have a variance of 125 00:08:45,790 --> 00:08:48,590 N_0 over 2 per dimension. 126 00:08:48,590 --> 00:08:52,060 So what we see here is that we can have analogous definitions 127 00:08:52,060 --> 00:08:55,590 for discrete time and continuous time, and one of 128 00:08:55,590 --> 00:08:59,750 the key parameters that comes up over and over again in the 129 00:08:59,750 --> 00:09:04,220 analysis is this notion of signal to noise ratio. 130 00:09:04,220 --> 00:09:09,550 The signal to noise ratio is defined as the energy per two 131 00:09:09,550 --> 00:09:13,335 dimensions, over the noise variance per two dimensions. 132 00:09:17,350 --> 00:09:19,570 So that's the definition of signal to noise ratio. 133 00:09:22,310 --> 00:09:28,630 Es, well, it's 2PT, or equivalently, P over W. The 134 00:09:28,630 --> 00:09:33,040 noise variance for two dimensions is N_0, so the 135 00:09:33,040 --> 00:09:35,190 definition is the same as -- 136 00:09:35,190 --> 00:09:40,510 SNR equals P over N_0 W. OK, are there any questions? 137 00:09:44,600 --> 00:09:49,150 The other notion we talked about last time is this idea 138 00:09:49,150 --> 00:09:50,400 of the spectral efficiency. 139 00:09:58,340 --> 00:10:02,310 In continuous time, the definition is quite natural. 140 00:10:02,310 --> 00:10:05,380 It's denoted by symbol rho. 141 00:10:05,380 --> 00:10:13,060 The units are bits per second per Hertz, and it's basically 142 00:10:13,060 --> 00:10:20,916 R over W. You have R bits per second over W hertz. 143 00:10:20,916 --> 00:10:22,600 So it's the amount of information bits that I'm able 144 00:10:22,600 --> 00:10:25,050 to send over the amount of bandwidth that 145 00:10:25,050 --> 00:10:26,300 I have in my system. 146 00:10:29,050 --> 00:10:40,300 In discrete time, we can also define the same idea of 147 00:10:40,300 --> 00:10:48,220 spectral efficiency, but it's usually -- and a good way to 148 00:10:48,220 --> 00:10:51,600 think about spectral efficiency from a point of 149 00:10:51,600 --> 00:10:54,100 view of a design of an encoder. 150 00:10:54,100 --> 00:11:02,680 What the encoder does, is you have an encoder here, it takes 151 00:11:02,680 --> 00:11:04,620 a sequence of bits in-- 152 00:11:04,620 --> 00:11:08,220 so say you have b bits coming in-- 153 00:11:08,220 --> 00:11:09,470 and it produces N symbols. 154 00:11:14,050 --> 00:11:19,100 So this could be X1, X2, Xn, and you have a 155 00:11:19,100 --> 00:11:21,160 sequence of B bits -- 156 00:11:21,160 --> 00:11:28,100 b1, b2, b subcapital B. This is how an encoder operates. 157 00:11:28,100 --> 00:11:31,950 It maps a sequence of bits to a sequence of symbols. 158 00:11:31,950 --> 00:11:34,045 Now where does the encoder fit into this architecture? 159 00:11:38,200 --> 00:11:40,640 It fits right here at the front, right? 160 00:11:40,640 --> 00:11:43,220 You have bits coming in, you encode them, you produce a 161 00:11:43,220 --> 00:11:47,440 sequence of symbols, and you send them over the channel. 162 00:11:47,440 --> 00:11:51,040 So if I have this encoder, what's my the spectral 163 00:11:51,040 --> 00:11:52,300 efficiency going to be? 164 00:12:03,790 --> 00:12:06,100 Well, you have to ask what the encoder does, right? 165 00:12:06,100 --> 00:12:08,610 So from here, we have a PAM modulator. 166 00:12:15,210 --> 00:12:17,315 So it's basically this from here on, we 167 00:12:17,315 --> 00:12:19,910 are back to the system. 168 00:12:19,910 --> 00:12:21,560 So what's the spectral efficiency 169 00:12:21,560 --> 00:12:22,810 now for this system? 170 00:12:29,580 --> 00:12:31,035 How many bits do you have per symbol? 171 00:12:34,823 --> 00:12:36,820 AUDIENCE: [UNINTELLIGIBLE] 172 00:12:36,820 --> 00:12:39,730 PROFESSOR: You have B over N bits per symbol. 173 00:12:39,730 --> 00:12:43,620 Now how many symbols do you have per dimension, if this is 174 00:12:43,620 --> 00:12:44,960 an orthonormal PAM system? 175 00:12:51,890 --> 00:12:52,890 AUDIENCE: One symbol per dimension. 176 00:12:52,890 --> 00:12:54,900 PROFESSOR: You have one symbol per dimension, right? 177 00:12:54,900 --> 00:12:59,060 So you have B bits per N dimensions, in other words. 178 00:12:59,060 --> 00:13:00,310 AUDIENCE: [UNINTELLIGIBLE] 179 00:13:04,490 --> 00:13:07,970 PROFESSOR: So in QAM how many -- you have usually how many 180 00:13:07,970 --> 00:13:09,300 symbols per dimension? 181 00:13:09,300 --> 00:13:09,740 AUDIENCE: [UNINTELLIGIBLE] 182 00:13:09,740 --> 00:13:12,880 PROFESSOR: Half symbols per dimension, right. 183 00:13:12,880 --> 00:13:15,650 So in that case, the spectral efficiency -- 184 00:13:15,650 --> 00:13:23,160 the units are bits per two dimensions, is 2B over N. B 185 00:13:23,160 --> 00:13:26,805 over N bits per dimension, because the units are bits per 186 00:13:26,805 --> 00:13:29,890 two dimensions, you get 2B over N bits per two 187 00:13:29,890 --> 00:13:31,640 dimensions. 188 00:13:31,640 --> 00:13:32,600 OK? 189 00:13:32,600 --> 00:13:36,480 Now a natural question to ask is, how is this definition 190 00:13:36,480 --> 00:13:39,060 related to this definition here? 191 00:13:39,060 --> 00:13:42,130 This is quite the natural definition, and here we have 192 00:13:42,130 --> 00:13:44,300 imposed this encoder structure. 193 00:13:44,300 --> 00:13:47,490 Are the two definitions equivalent in any way? 194 00:13:47,490 --> 00:13:57,620 And in order to understand this, let us take 195 00:13:57,620 --> 00:13:58,870 a one-second snapshot. 196 00:14:11,370 --> 00:14:19,250 So now in one second, I can send N equals 2W symbols. 197 00:14:19,250 --> 00:14:23,010 Because this is an orthonormal PAM system, I can send 2W 198 00:14:23,010 --> 00:14:25,542 symbols per second, but in one second, I can 199 00:14:25,542 --> 00:14:28,220 send N equals 2W symbols. 200 00:14:28,220 --> 00:14:33,680 Because my rate is R bits per second, B equals R, in one 201 00:14:33,680 --> 00:14:35,850 second I can send R bits. 202 00:14:35,850 --> 00:14:39,980 So now my definition of rho, which I defined to be 2B over 203 00:14:39,980 --> 00:14:47,000 N, is same as 2R over 2W, which is R over W. So the 204 00:14:47,000 --> 00:14:49,380 definitions in continuous time and discrete time are 205 00:14:49,380 --> 00:14:50,630 equivalent. 206 00:14:53,130 --> 00:14:54,380 OK? 207 00:15:25,540 --> 00:15:28,950 Now, why is spectral efficiency very important? 208 00:15:28,950 --> 00:15:32,200 Well, there is a very famous theorem by Shannon which gives 209 00:15:32,200 --> 00:15:35,100 us a nice upper bound on the spectral efficiency. 210 00:15:35,100 --> 00:15:38,600 Perhaps the most important theorem in communications is 211 00:15:38,600 --> 00:15:44,610 Shannon's Theorem, it says that if you have an AWGN 212 00:15:44,610 --> 00:15:48,830 system with a certain SNR, then you can immediately bound 213 00:15:48,830 --> 00:15:56,720 the spectral efficiency by log2 of 1 plus SNR bits per 214 00:15:56,720 --> 00:15:59,070 two dimensions. 215 00:15:59,070 --> 00:16:01,410 This is a very powerful statement. 216 00:16:01,410 --> 00:16:10,460 Equivalently, the capacity of an AWGN channel is log2 1 plus 217 00:16:10,460 --> 00:16:15,090 SNR bits per second. 218 00:16:15,090 --> 00:16:17,750 So one important observation here is if I have a 219 00:16:17,750 --> 00:16:20,850 communication system, and if what I care about is the 220 00:16:20,850 --> 00:16:24,370 spectral efficiency of the capacity, there are only two 221 00:16:24,370 --> 00:16:26,360 terms that are important. 222 00:16:26,360 --> 00:16:30,750 One is the signal to noise ratio, which is P over N_0 W, 223 00:16:30,750 --> 00:16:33,020 which is defined here. 224 00:16:33,020 --> 00:16:36,230 So the individual units of P and N_0 doesn't matter, it's 225 00:16:36,230 --> 00:16:38,710 only the ratio of P over N_0 that matters. 226 00:16:38,710 --> 00:16:41,240 And the second parameter is the bandwidth W that we have 227 00:16:41,240 --> 00:16:42,440 in the system. 228 00:16:42,440 --> 00:16:45,930 So signal to noise ratio and bandwidth are in some sense 229 00:16:45,930 --> 00:16:49,000 fundamental to the system. 230 00:16:49,000 --> 00:16:53,050 An operational meaning of this theorem is that if I look at 231 00:16:53,050 --> 00:16:56,260 this encoder, then it gives me an upper bound of how many 232 00:16:56,260 --> 00:16:59,090 bits I can put for each symbol. 233 00:16:59,090 --> 00:17:02,250 The number of bits that I can put on each symbol is upper 234 00:17:02,250 --> 00:17:06,060 bounded by this term here, log2 1 plus SNR. 235 00:17:06,060 --> 00:17:10,500 I cannot put arbitrarily many bits per each symbol here. 236 00:17:10,500 --> 00:17:12,819 Now in order to make such a statement, there has to be 237 00:17:12,819 --> 00:17:15,490 some criteria that we need to satisfy. 238 00:17:15,490 --> 00:17:17,119 And what is the criteria for Shannon's Theorem? 239 00:17:23,609 --> 00:17:26,220 In order to make such a statement, I need to say that 240 00:17:26,220 --> 00:17:28,630 some objective function has to be satisfied. 241 00:17:28,630 --> 00:17:30,800 Because in some sense, I could just put any number of bits I 242 00:17:30,800 --> 00:17:33,270 could have on upper symbol, right? 243 00:17:33,270 --> 00:17:36,130 The encoder could just put 100 or 200 bits. 244 00:17:36,130 --> 00:17:38,390 What's going to limit me? 245 00:17:38,390 --> 00:17:39,580 AUDIENCE: Probability of error? 246 00:17:39,580 --> 00:17:40,920 PROFESSOR: The probability of error. 247 00:17:40,920 --> 00:17:42,280 So this assumes that -- 248 00:17:53,220 --> 00:17:54,460 OK? 249 00:17:54,460 --> 00:17:58,256 Now, you're not responsible for the proof of this theorem, 250 00:17:58,256 --> 00:18:00,435 it is in Chapter Three of the notes. 251 00:18:09,420 --> 00:18:11,890 Basically, it's just a random coding argument, which is 252 00:18:11,890 --> 00:18:15,720 quite standard in information theory. 253 00:18:15,720 --> 00:18:19,330 So if you already taken information theory or 254 00:18:19,330 --> 00:18:22,370 otherwise, you would probably have seen that argument 255 00:18:22,370 --> 00:18:25,440 involves bounding atypical events that happen. 256 00:18:25,440 --> 00:18:28,190 So the probability of error is an atypical event, and we use 257 00:18:28,190 --> 00:18:30,230 asymptotic equipartition property to 258 00:18:30,230 --> 00:18:32,010 bound the error event. 259 00:18:32,010 --> 00:18:35,210 There's a standard proof in Cover and Thomas, for example. 260 00:18:35,210 --> 00:18:39,310 Now the main difference in Professor Forney's approach is 261 00:18:39,310 --> 00:18:42,320 that he uses this Theory of Large Deviations. 262 00:18:42,320 --> 00:18:46,640 Theory of Large Deviations basically gives you a bound on 263 00:18:46,640 --> 00:18:49,370 the occurrence of rare events, and it is well-known in 264 00:18:49,370 --> 00:18:51,070 statistical mechanics. 265 00:18:51,070 --> 00:18:53,620 So it's kind of a different approach to the same problem. 266 00:18:53,620 --> 00:18:55,830 The basic idea is same as you would find in a standard 267 00:18:55,830 --> 00:18:57,720 proof, but it uses -- 268 00:18:57,720 --> 00:19:00,010 it comes from this idea of large deviations theory. 269 00:19:07,830 --> 00:19:10,650 So for those of you who are taking information theory or 270 00:19:10,650 --> 00:19:13,570 already have seen it, I urge you, go at some point, take a 271 00:19:13,570 --> 00:19:14,950 look at this proof. 272 00:19:14,950 --> 00:19:16,670 It's quite cool. 273 00:19:16,670 --> 00:19:18,950 I already saw somebody reading this last Friday, and it was 274 00:19:18,950 --> 00:19:19,950 quite impressive. 275 00:19:19,950 --> 00:19:21,800 So I urge more of you to do that. 276 00:19:28,220 --> 00:19:32,810 So now that we have the spectral efficiency, a natural 277 00:19:32,810 --> 00:19:34,200 thing is to plot how the spectral 278 00:19:34,200 --> 00:19:36,220 efficiency looks like. 279 00:19:36,220 --> 00:19:41,080 So what I'm going to plot is as a function of SNR the 280 00:19:41,080 --> 00:19:43,990 spectral efficiency. 281 00:19:43,990 --> 00:19:46,700 Typically, when you plot SNR on the x-axis, you almost 282 00:19:46,700 --> 00:19:49,140 always plot it on a dB scale. 283 00:19:49,140 --> 00:19:51,280 But I'm going to make one exception this time, and I'm 284 00:19:51,280 --> 00:19:55,900 going to plot this on a linear scale. 285 00:19:55,900 --> 00:20:02,400 So this point is zero, or minus infinity dB here, and 286 00:20:02,400 --> 00:20:03,590 I'm going to plot -- 287 00:20:03,590 --> 00:20:06,580 well I should call this, actually, rho Shannon, so I 288 00:20:06,580 --> 00:20:08,910 don't confuse the notation. 289 00:20:08,910 --> 00:20:11,650 So we'll define rho Shannon as log2 1 plus SNR. 290 00:20:16,030 --> 00:20:19,840 So this is rho Shannon here, and we want to plot rho 291 00:20:19,840 --> 00:20:22,630 Shannon as a function of SNR. 292 00:20:22,630 --> 00:20:26,160 Now if my SNR is really small, log 1 plus SNR is 293 00:20:26,160 --> 00:20:31,000 approximately linear, so I get a linear increase here. 294 00:20:31,000 --> 00:20:38,440 If my SNR is large, then the logarithmic behavior kicks 295 00:20:38,440 --> 00:20:41,550 into this expression here, so now the spectral efficiency 296 00:20:41,550 --> 00:20:44,900 grows slower and slower with SNR. 297 00:20:44,900 --> 00:20:49,630 So this is a basic shape for my spectral efficiency. 298 00:20:49,630 --> 00:20:52,060 And this immediately suggests that there are two different 299 00:20:52,060 --> 00:20:54,110 operating regimes we have. 300 00:20:54,110 --> 00:20:57,030 One regime where the spectral efficiency increases linearly 301 00:20:57,030 --> 00:21:00,320 with SNR, another regime where the spectral efficiency 302 00:21:00,320 --> 00:21:03,220 increases logarithmically with SNR. 303 00:21:03,220 --> 00:21:11,250 So if SNR is very small, then we call this regime as 304 00:21:11,250 --> 00:21:12,500 power-limited regime. 305 00:21:21,200 --> 00:21:26,920 And if SNR is large, we call this the 306 00:21:26,920 --> 00:21:29,320 bandwidth-limited regime. 307 00:21:34,820 --> 00:21:36,790 These are our definitions. 308 00:21:36,790 --> 00:21:39,230 And let's see what motivates their names. 309 00:21:39,230 --> 00:21:43,200 Suppose I have a 3 dB increase in my SNR, and I am in power 310 00:21:43,200 --> 00:21:44,760 limited regime. 311 00:21:44,760 --> 00:21:46,025 How does rho Shannon increase? 312 00:21:51,588 --> 00:21:52,838 AUDIENCE: [UNINTELLIGIBLE] 313 00:21:56,370 --> 00:21:58,440 PROFESSOR: A factor of 2, right? 314 00:21:58,440 --> 00:22:02,220 Basically, if I have a 3 dB increase in SNR, my SNR 315 00:22:02,220 --> 00:22:04,330 increases by a factor of 2. 316 00:22:04,330 --> 00:22:08,230 In this regime, rho Shannon increases linearly with SNR, 317 00:22:08,230 --> 00:22:10,590 so I have a factor of 2 increase in my spectral 318 00:22:10,590 --> 00:22:11,680 efficiency. 319 00:22:11,680 --> 00:22:13,840 What about this regime here? 320 00:22:13,840 --> 00:22:17,760 If SNR increases by 3 dB, how does rho Shannon increase? 321 00:22:31,080 --> 00:22:34,690 It increases by one bit per two dimensions. 322 00:22:34,690 --> 00:22:37,200 Units of rho are bits per two dimensions. 323 00:22:37,200 --> 00:22:40,130 I have a logarithmic behavior kicking in, so rho is 324 00:22:40,130 --> 00:22:42,060 approximately log SNR. 325 00:22:42,060 --> 00:22:45,120 If I increase SNR by a factor of 2, I get an additional 1 326 00:22:45,120 --> 00:22:48,030 bit per two dimension scale. 327 00:22:48,030 --> 00:22:49,590 OK? 328 00:22:49,590 --> 00:22:53,140 If I increase my bandwidth in the power-limited regime by a 329 00:22:53,140 --> 00:22:54,650 factor of 2. 330 00:22:54,650 --> 00:22:57,870 So this bandwidth here increases by a factor of 2, 331 00:22:57,870 --> 00:23:02,750 how does my capacity change, if I'm in the 332 00:23:02,750 --> 00:23:04,000 power-limited regime? 333 00:23:09,820 --> 00:23:12,980 There's no change, right? 334 00:23:12,980 --> 00:23:16,270 Or is there a change? 335 00:23:16,270 --> 00:23:19,570 I'm in the power-limited regime here, and I increase my 336 00:23:19,570 --> 00:23:21,185 bandwidth by a factor of 2. 337 00:23:21,185 --> 00:23:22,435 AUDIENCE: [UNINTELLIGIBLE] 338 00:23:26,520 --> 00:23:27,500 PROFESSOR: Right. 339 00:23:27,500 --> 00:23:29,590 What is my SNR? 340 00:23:29,590 --> 00:23:32,960 It's P over W N_0. 341 00:23:32,960 --> 00:23:36,400 So what happens if I fix P over N_0? 342 00:23:36,400 --> 00:23:38,190 OK, yeah, so you had it. 343 00:23:38,190 --> 00:23:41,370 Basically, if I double my bandwidth, my SNR decreases by 344 00:23:41,370 --> 00:23:47,350 a factor of 2, so this term here is like SNR over 2, W 345 00:23:47,350 --> 00:23:49,440 increases by a factor of 2, and there is 346 00:23:49,440 --> 00:23:51,980 no change in bandwidth. 347 00:23:51,980 --> 00:23:53,230 So let's do it more slowly. 348 00:24:00,590 --> 00:24:01,840 So we have, say, in the -- 349 00:24:12,680 --> 00:24:19,390 and say, my C is now W log2 1 plus SNR. 350 00:24:19,390 --> 00:24:24,970 Instead of SNR, I will write P over N_0 W. So this 351 00:24:24,970 --> 00:24:31,970 approximately is W times P over N_0 W times log 352 00:24:31,970 --> 00:24:35,090 E to the base 2. 353 00:24:35,090 --> 00:24:37,330 And this is approximately -- 354 00:24:37,330 --> 00:24:44,300 this is basically P over N_0 times log E to the base 2. 355 00:24:44,300 --> 00:24:47,330 So in other words, in the power-limited regime, changing 356 00:24:47,330 --> 00:24:50,970 bandwidth has no effect on the capacity. 357 00:24:50,970 --> 00:24:53,060 What happens in the bandwidth-limited regime? 358 00:25:05,860 --> 00:25:14,130 Well, in this case C equals W log2. 359 00:25:14,130 --> 00:25:16,680 Well, I can say approximately, and I can ignore this factor 360 00:25:16,680 --> 00:25:20,750 of 1, because P over N_0 W is large, because my SNR is large 361 00:25:20,750 --> 00:25:22,730 in the bandwidth-limited regime. 362 00:25:22,730 --> 00:25:27,580 I'm operating in this part of the graph. 363 00:25:27,580 --> 00:25:33,720 So this is P over N_0 W. Now suppose I increase my 364 00:25:33,720 --> 00:25:35,890 bandwidth by a factor of 2. 365 00:25:35,890 --> 00:25:44,110 C prime will be 2W log2 over P over 2 W N_0. 366 00:25:48,040 --> 00:25:49,340 All right? 367 00:25:49,340 --> 00:26:00,220 This, I can write it as 2W log2 P over N_0 W minus -- 368 00:26:00,220 --> 00:26:04,050 this is to the base 2, so I can write a 1 here. 369 00:26:04,050 --> 00:26:06,900 And this term is approximately same -- 370 00:26:06,900 --> 00:26:10,750 if I ignore the 1, if I assume P over N_0 W is quite large, 371 00:26:10,750 --> 00:26:13,330 then I can just ignore the subtraction of 1. 372 00:26:13,330 --> 00:26:19,610 So I get 2W log2 P over N_0 W, or it's equivalently 2C. 373 00:26:19,610 --> 00:26:23,040 OK, so in other words, in the bandwidth-limited regime, if I 374 00:26:23,040 --> 00:26:28,480 increase my W by a factor of 2, the capacity approximately 375 00:26:28,480 --> 00:26:30,330 increases by a factor of 2. 376 00:26:30,330 --> 00:26:31,960 So that's what motivates the name 377 00:26:31,960 --> 00:26:35,040 bandwidth-limited regime here. 378 00:26:35,040 --> 00:26:36,290 Are there any questions? 379 00:26:40,180 --> 00:26:41,430 OK. 380 00:27:49,370 --> 00:27:50,010 All right. 381 00:27:50,010 --> 00:27:54,390 So it turns out that there are two points I wanted to say. 382 00:27:54,390 --> 00:27:57,920 First of all, one might ask, that fine, these seem to be 383 00:27:57,920 --> 00:28:01,580 interesting definitions, that SNR is much smaller than one, 384 00:28:01,580 --> 00:28:04,860 and SNR is much larger than one, but is there any kind of 385 00:28:04,860 --> 00:28:07,830 hard division between the bandwidth-limited regime and 386 00:28:07,830 --> 00:28:09,450 power-limited regime? 387 00:28:09,450 --> 00:28:10,900 I mean, the general answer is no. 388 00:28:10,900 --> 00:28:11,930 It's basically -- 389 00:28:11,930 --> 00:28:15,680 because there is some point when the capacity appears not 390 00:28:15,680 --> 00:28:18,000 strictly logarithmically, or strictly 391 00:28:18,000 --> 00:28:19,910 linearly in terms of SNR. 392 00:28:19,910 --> 00:28:23,980 But from an engineering point of view, we take rho equals 393 00:28:23,980 --> 00:28:34,510 two bits per two dimension as a dividing point between the 394 00:28:34,510 --> 00:28:35,760 two regimes. 395 00:28:44,780 --> 00:28:48,100 And one motivation to see why rho equals two bits per two 396 00:28:48,100 --> 00:28:51,180 dimension is a good choice is because this is the maximum 397 00:28:51,180 --> 00:28:55,680 spectral efficiency we can get from binary modulation. 398 00:28:55,680 --> 00:28:58,340 OK, if you want to have spectral efficiency more than 399 00:28:58,340 --> 00:29:01,160 two bits per two dimensions, you have to go to multi-level 400 00:29:01,160 --> 00:29:05,830 modulation, and that's one of the reasons why this choice is 401 00:29:05,830 --> 00:29:09,090 often used in practice. 402 00:29:09,090 --> 00:29:12,180 Another point is that the bandwidth-limited regime and 403 00:29:12,180 --> 00:29:15,340 power-limited regime, they behave quite differently in 404 00:29:15,340 --> 00:29:18,390 almost all criterias that you can think about. 405 00:29:18,390 --> 00:29:20,040 So usually, with the analysis -- 406 00:29:20,040 --> 00:29:22,490 we keep them separately and do the analysis differently in 407 00:29:22,490 --> 00:29:24,780 the bandwidth and power-limited regime, and 408 00:29:24,780 --> 00:29:27,130 that's what we will be doing in the subsequent part of this 409 00:29:27,130 --> 00:29:28,450 lecture and next lecture. 410 00:29:34,110 --> 00:29:36,020 So we start with the power-limited regime. 411 00:29:46,520 --> 00:30:03,230 Now, we already saw that in doubling the power doubles rho 412 00:30:03,230 --> 00:30:19,600 S, and doubling bandwidth does not change C. 413 00:30:19,600 --> 00:30:27,890 OK, the other point I mentioned was binary 414 00:30:27,890 --> 00:30:37,510 modulation is sufficient in this regime, because our 415 00:30:37,510 --> 00:30:39,460 spectral efficiency is less than two bits per two 416 00:30:39,460 --> 00:30:42,950 dimensions, so the idea is to have a strong code followed by 417 00:30:42,950 --> 00:30:44,200 binary modulation. 418 00:30:46,490 --> 00:30:47,740 What else? 419 00:30:51,780 --> 00:30:52,030 Right. 420 00:30:52,030 --> 00:31:01,100 Typically, the normalization is done in 421 00:31:01,100 --> 00:31:02,350 per information bit. 422 00:31:09,840 --> 00:31:11,300 What does this mean? 423 00:31:11,300 --> 00:31:14,100 When we are wanting to compare different systems, we will 424 00:31:14,100 --> 00:31:16,220 look at all the parameters normalized 425 00:31:16,220 --> 00:31:18,150 per information bit. 426 00:31:18,150 --> 00:31:20,570 For example, if we want to look at the probability of 427 00:31:20,570 --> 00:31:25,450 error, we look at this quantity here, Pb of E, which 428 00:31:25,450 --> 00:31:29,060 is the probability of error per information bit, as a 429 00:31:29,060 --> 00:31:30,310 function of Eb/N_0. 430 00:31:34,050 --> 00:31:37,270 This is an important trade-off that we wish to study in 431 00:31:37,270 --> 00:31:39,020 power-limited regime. 432 00:31:39,020 --> 00:31:43,310 Eb is the energy per bit, P sub b of E is the probability 433 00:31:43,310 --> 00:31:45,372 of error per information bit. 434 00:32:08,510 --> 00:32:11,360 So I want to spend some time on this Eb/N_0, because it's 435 00:32:11,360 --> 00:32:13,370 an important concept that we'll be using 436 00:32:13,370 --> 00:32:15,080 often in the course. 437 00:32:15,080 --> 00:32:16,870 So what is Eb/N_0? 438 00:32:16,870 --> 00:32:21,190 It's sometimes also known as Eb over N_0, so it depends how 439 00:32:21,190 --> 00:32:23,340 you wish to call it. 440 00:32:23,340 --> 00:32:25,995 What is energy per bit in terms of Es? 441 00:32:36,490 --> 00:32:37,010 AUDIENCE: [UNINTELLIGIBLE] 442 00:32:37,010 --> 00:32:38,280 PROFESSOR: Well, Es over rho, right? 443 00:32:38,280 --> 00:32:39,890 Energy per bit. 444 00:32:39,890 --> 00:32:42,240 Well, Es is energy per two dimensions. 445 00:32:42,240 --> 00:32:45,330 Rho is bits per two dimensions, so Eb is Es over 446 00:32:45,330 --> 00:32:52,150 rho, and you have N_0 here. 447 00:32:52,150 --> 00:32:57,403 And Es over N_0 is also our SNR, so this is SNR over rho. 448 00:33:00,160 --> 00:33:04,410 OK, so that's how Eb/N_0 is defined. 449 00:33:04,410 --> 00:33:11,270 Now we know from Shannon that rho is always less than log2 1 450 00:33:11,270 --> 00:33:16,810 plus SNR, or equivalently, 2 to the rho minus 1 451 00:33:16,810 --> 00:33:19,990 is less than SNR. 452 00:33:19,990 --> 00:33:24,216 If I sub in here, this means that SNR is greater than 2 to 453 00:33:24,216 --> 00:33:27,980 the rho minus 1 over rho, and then we're 454 00:33:27,980 --> 00:33:29,370 just dividing by rho. 455 00:33:29,370 --> 00:33:31,590 So Eb/N_0 is always greater than 2 to the rho 456 00:33:31,590 --> 00:33:33,760 minus 1 over rho. 457 00:33:33,760 --> 00:33:36,420 And this is quite an interesting observation. 458 00:33:36,420 --> 00:33:39,460 For example, if you are analyzing the feasibility of a 459 00:33:39,460 --> 00:33:42,710 communication system, which has a certain spectral 460 00:33:42,710 --> 00:33:46,240 efficiency, and a certain Eb/N_0, then you can 461 00:33:46,240 --> 00:33:48,990 immediately check this relation, to see if it is a 462 00:33:48,990 --> 00:33:51,840 system in the first place. 463 00:33:51,840 --> 00:33:54,240 What Shannon says is that Eb/N_0 is always going to be 464 00:33:54,240 --> 00:33:57,060 greater than 2 to the rho minus 1 over rho. 465 00:33:57,060 --> 00:34:00,030 So if you see that this relation is not satisfied, 466 00:34:00,030 --> 00:34:01,610 immediately know something is wrong. 467 00:34:04,160 --> 00:34:06,920 This actually reminds me of an interesting anecdote that 468 00:34:06,920 --> 00:34:08,480 Professor Forney once mentioned when I 469 00:34:08,480 --> 00:34:10,090 was taking the class. 470 00:34:10,090 --> 00:34:13,170 Well he has been in this field since the '60's, and so he has 471 00:34:13,170 --> 00:34:14,989 seen a lot of this stuff. 472 00:34:14,989 --> 00:34:16,610 He was saying when turbo codes -- 473 00:34:16,610 --> 00:34:19,449 which is one of the first capacity approaching codes in 474 00:34:19,449 --> 00:34:21,730 recent years when they were proposed -- 475 00:34:21,730 --> 00:34:24,420 they presented the results at ICC, the International 476 00:34:24,420 --> 00:34:27,889 Conference in Communications, and what they saw was that the 477 00:34:27,889 --> 00:34:33,370 performance was very close to the limit that we can predict. 478 00:34:33,370 --> 00:34:36,770 Eb/N_0 was very close to the ultimate limit -- 479 00:34:36,770 --> 00:34:39,250 the Eb/N_0 achieved by turbo codes was very close to the 480 00:34:39,250 --> 00:34:43,500 ultimate limit that one can predict, at least 3 dB better 481 00:34:43,500 --> 00:34:45,679 than the best codes that were available there. 482 00:34:45,679 --> 00:34:47,789 So most people just thought that there was something wrong 483 00:34:47,789 --> 00:34:50,360 in the simulations, and they told them that there was a 484 00:34:50,360 --> 00:34:52,560 factor of 2 missing somewhere, so they 485 00:34:52,560 --> 00:34:53,929 should just double check. 486 00:34:53,929 --> 00:34:55,980 But it turned out when they went back and people actually 487 00:34:55,980 --> 00:34:58,470 implemented these codes, they were actually very close to 488 00:34:58,470 --> 00:35:00,910 the capacity. 489 00:35:00,910 --> 00:35:04,730 So sometimes, you have to be careful. 490 00:35:04,730 --> 00:35:06,770 If you are not going below this limit, it could be that 491 00:35:06,770 --> 00:35:08,100 the system is good. 492 00:35:08,100 --> 00:35:10,217 And when it is, it's a really important breakthrough. 493 00:35:10,217 --> 00:35:14,610 Ok 494 00:35:14,610 --> 00:35:19,830 So one particular concept that comes here is the idea of 495 00:35:19,830 --> 00:35:21,375 ultimate Shannon limit. 496 00:35:30,140 --> 00:35:32,880 And basically, if we are talking about power limited 497 00:35:32,880 --> 00:35:35,640 regime, our SNR is very small. 498 00:35:35,640 --> 00:35:38,920 So our spectral efficiency is going to be quite small. 499 00:35:38,920 --> 00:35:40,970 Now notice that this function here it monotonically 500 00:35:40,970 --> 00:35:43,450 increasing in rho. 501 00:35:43,450 --> 00:35:46,130 It's easy to show that this function here increases 502 00:35:46,130 --> 00:35:47,430 monotonically in rho. 503 00:35:47,430 --> 00:35:50,380 You could just differentiate this, or there's easier ways 504 00:35:50,380 --> 00:35:53,630 to do a Taylor expansion of 2 to the rho minus 1, the series 505 00:35:53,630 --> 00:35:56,090 expansion and show each term is positive. 506 00:35:56,090 --> 00:35:58,080 So if this term is always going to be greater 507 00:35:58,080 --> 00:36:01,360 than this term here. 508 00:36:01,360 --> 00:36:04,420 So I'm going from here to here. 509 00:36:04,420 --> 00:36:10,530 Limit rho tends to zero of 2 to the rho minus 1 over rho. 510 00:36:10,530 --> 00:36:13,740 This is going to be -- it's a simple calculus exercise to 511 00:36:13,740 --> 00:36:16,770 show this is the natural log of 2, or in dB, 512 00:36:16,770 --> 00:36:21,780 it's minus 1.59 dB. 513 00:36:21,780 --> 00:36:25,720 So no matter what system you design, your Eb/N_0 is always 514 00:36:25,720 --> 00:36:29,230 going to be greater than minus 1.59 dB. 515 00:36:29,230 --> 00:36:32,050 That's basically what this calculation shows. 516 00:36:32,050 --> 00:36:33,270 And when is it achieved? 517 00:36:33,270 --> 00:36:34,770 Well, it's only achieved when the spectral 518 00:36:34,770 --> 00:36:37,250 efficiency goes to zero. 519 00:36:37,250 --> 00:36:40,210 So if you have a deep space communication system, where 520 00:36:40,210 --> 00:36:42,500 you have lots and lots of bandwidth, and you do not care 521 00:36:42,500 --> 00:36:46,490 about spectral efficiency, then if your only criteria is 522 00:36:46,490 --> 00:36:49,270 to minimize Eb/N_0, then you can design your system 523 00:36:49,270 --> 00:36:52,260 accordingly, check how much Eb/N_0 you require for a 524 00:36:52,260 --> 00:36:55,110 certain probability of error, and see how far you are from 525 00:36:55,110 --> 00:36:57,300 the ultimate Shannon limit. 526 00:36:57,300 --> 00:36:59,970 So in this way, in the power-limited regime, you can 527 00:36:59,970 --> 00:37:03,650 quantify your gap to capacity, if you will, through this 528 00:37:03,650 --> 00:37:07,090 ultimate Shannon limit. 529 00:37:07,090 --> 00:37:08,370 OK, are there any questions? 530 00:38:44,890 --> 00:38:45,710 OK. 531 00:38:45,710 --> 00:38:47,735 So now let's look at the bandwidth-limited regime. 532 00:39:02,740 --> 00:39:04,650 So we already saw two things in the 533 00:39:04,650 --> 00:39:06,410 bandwidth-limited regime. 534 00:39:06,410 --> 00:39:15,830 If I double P, my spectral efficiency increases by one 535 00:39:15,830 --> 00:39:17,080 bit per two dimension. 536 00:39:19,900 --> 00:39:30,140 Similarly, if I double bandwidth by C, capacity 537 00:39:30,140 --> 00:39:31,390 approximately doubles. 538 00:39:37,648 --> 00:39:40,260 Now, because you want a spectral efficiency of more 539 00:39:40,260 --> 00:39:43,440 than two bits per two dimension, in this system you 540 00:39:43,440 --> 00:39:45,815 typically do a multi-level modulation. 541 00:39:53,650 --> 00:39:56,830 So for those of you who are familiar, we do things like 542 00:39:56,830 --> 00:39:59,760 trellis-coded modulation, and bit-interleaved coded 543 00:39:59,760 --> 00:40:01,080 modulation, and so on. 544 00:40:01,080 --> 00:40:03,280 If we have time, we'll be seeing those things towards 545 00:40:03,280 --> 00:40:05,370 the very end of the course. 546 00:40:05,370 --> 00:40:07,940 This is not a subject of the course as such. 547 00:40:11,470 --> 00:40:20,420 The normalization in this regime is done for two 548 00:40:20,420 --> 00:40:21,670 dimensions. 549 00:40:25,260 --> 00:40:28,405 So if you want to normalize all the quantities, we 550 00:40:28,405 --> 00:40:30,980 normalize them for two dimensions here. 551 00:40:30,980 --> 00:40:35,740 And particularly, we will be seeing probability of error as 552 00:40:35,740 --> 00:40:38,850 a function of this quantity called SNR norm. 553 00:40:42,315 --> 00:40:45,930 So this is the performance analysis that is done in 554 00:40:45,930 --> 00:40:47,930 bandwidth-limited regimes. 555 00:40:47,930 --> 00:40:59,450 Here Ps of E is the probability of error for two 556 00:40:59,450 --> 00:41:00,700 dimensions. 557 00:41:05,290 --> 00:41:06,540 What is SNR norm? 558 00:41:10,360 --> 00:41:18,830 It's defined to be SNR over 2 to the rho minus 1. 559 00:41:18,830 --> 00:41:21,810 Why do we divide by 2 to the rho minus 1? 560 00:41:21,810 --> 00:41:25,420 Well, that's the minimum SNR that you require for the best 561 00:41:25,420 --> 00:41:26,740 possible system. 562 00:41:26,740 --> 00:41:28,970 That's what Shannon says, right? 563 00:41:28,970 --> 00:41:31,320 So this quantity here is always going to be greater 564 00:41:31,320 --> 00:41:33,510 than 1, or 0 dB. 565 00:41:36,140 --> 00:41:38,640 So OK, well, this is the ultimate Shannon limit in the 566 00:41:38,640 --> 00:41:40,860 bandwidth-limited regime. 567 00:41:40,860 --> 00:41:43,720 If you have a system that is designed, that operates at a 568 00:41:43,720 --> 00:41:47,540 certain SNR and a certain spectral efficiency, you can 569 00:41:47,540 --> 00:41:52,220 calculate SNR norm and see how far you are from 0 dB. 570 00:41:52,220 --> 00:41:55,170 If you're very close to 0 dB, then that's great. 571 00:41:55,170 --> 00:41:57,890 You have a very good system in practice. 572 00:41:57,890 --> 00:42:01,310 If not, then you have room for improvement. 573 00:42:01,310 --> 00:42:07,600 And so, in other words, SNR norm 574 00:42:07,600 --> 00:42:12,950 defines the gap to capacity. 575 00:42:30,685 --> 00:42:32,155 OK, so let's do an example. 576 00:42:38,920 --> 00:42:40,760 Suppose we have an M-PAM system. 577 00:42:43,750 --> 00:42:46,620 So we have an M-PAM constellation. 578 00:42:46,620 --> 00:42:48,000 So how does it look like? 579 00:42:48,000 --> 00:42:52,350 Well, you have points here on a linear line. 580 00:42:52,350 --> 00:42:58,830 This point is say minus alpha, alpha, three alpha minus three 581 00:42:58,830 --> 00:43:05,700 alpha, all the way up to m minus one alpha, and here, 582 00:43:05,700 --> 00:43:08,780 minus of m minus one alpha. 583 00:43:08,780 --> 00:43:10,050 Assume m is an even number. 584 00:43:12,600 --> 00:43:16,750 So the distance between two points here is two alpha, any 585 00:43:16,750 --> 00:43:18,450 two points. 586 00:43:18,450 --> 00:43:25,400 Now we want to find the SNR norm given that we are using 587 00:43:25,400 --> 00:43:26,650 this constellation. 588 00:43:34,220 --> 00:43:37,540 So in other words, if I use this constellation in my 589 00:43:37,540 --> 00:43:40,800 communication system, how far am I operating from the 590 00:43:40,800 --> 00:43:43,420 ultimate Shannon limit? 591 00:43:43,420 --> 00:43:46,280 OK, that's the question. 592 00:43:46,280 --> 00:43:51,760 So in order to find what we need to find, is first energy 593 00:43:51,760 --> 00:43:53,970 for two dimensions. 594 00:43:53,970 --> 00:43:55,790 Does anybody remember the formula for M-PAM? 595 00:44:00,140 --> 00:44:04,000 Well, there was a very nice way of doing it in 450. 596 00:44:04,000 --> 00:44:06,890 One natural way is to simply sum up all of the coordinates 597 00:44:06,890 --> 00:44:11,480 here, square them, and divide by M. And because it's for two 598 00:44:11,480 --> 00:44:14,810 dimensions, we could do it -- 599 00:44:14,810 --> 00:44:16,980 we're to multiply by a factor of two, because it's for two 600 00:44:16,980 --> 00:44:18,360 dimensions-- 601 00:44:18,360 --> 00:44:20,060 and summation of Xk squared. 602 00:44:23,290 --> 00:44:26,400 And if you work out, you will get some answer here. 603 00:44:26,400 --> 00:44:29,250 Another way that was shown in 450 -- 604 00:44:29,250 --> 00:44:31,740 AUDIENCE: [UNINTELLIGIBLE] 605 00:44:31,740 --> 00:44:33,940 PROFESSOR: With uniform quantization, exactly. 606 00:44:33,940 --> 00:44:38,260 So the idea here is, you have a source, say, which is 607 00:44:38,260 --> 00:44:44,460 uniformly distributed between M alpha and minus M alpha. 608 00:44:44,460 --> 00:44:47,470 So you have a source uniformly distributed, and it can be 609 00:44:47,470 --> 00:44:52,010 easily seen by inspection that M-PAM is the best quantizer 610 00:44:52,010 --> 00:44:55,540 for this particular source. 611 00:44:55,540 --> 00:44:55,615 Ok. 612 00:44:55,615 --> 00:45:03,510 So the interval regions are just of equal width. 613 00:45:03,510 --> 00:45:08,940 Each interval region is width two alpha, so your mean square 614 00:45:08,940 --> 00:45:19,250 error, if you will, is 2 alpha squared over 12, so it's alpha 615 00:45:19,250 --> 00:45:21,460 squared over 3. 616 00:45:21,460 --> 00:45:24,800 Well, the variance in your source, which I will denote by 617 00:45:24,800 --> 00:45:32,610 sigma square s, is 2M alpha squared over 12, or it's M 618 00:45:32,610 --> 00:45:35,790 squared alpha squared over 3. 619 00:45:35,790 --> 00:45:40,450 So your energy per symbol -- 620 00:45:40,450 --> 00:45:45,090 it's energy in your quantizer, so I'm denoting it by E of A 621 00:45:45,090 --> 00:45:47,850 in order to differentiate it by Es, because Es is two times 622 00:45:47,850 --> 00:45:50,080 E of A, OK? 623 00:45:50,080 --> 00:45:54,260 It's going to be m squared minus 1 times alpha 624 00:45:54,260 --> 00:45:55,530 squared over 3. 625 00:45:55,530 --> 00:46:00,840 So Es is 2 E of A, and so we get it's 2 times alpha 626 00:46:00,840 --> 00:46:03,600 squared, m squared minus 1 over 3. 627 00:46:08,910 --> 00:46:12,110 OK, so can anybody tell me what the spectral efficiency 628 00:46:12,110 --> 00:46:16,020 will be for the system, if I use an M-PAM constellation? 629 00:46:29,982 --> 00:46:32,180 Well, how many bits per symbol do I have? 630 00:46:34,988 --> 00:46:36,860 AUDIENCE: Log2M. 631 00:46:36,860 --> 00:46:38,190 PROFESSOR: Log2 M bits per symbol. 632 00:46:38,190 --> 00:46:41,475 So since it's bits per two dimensions, the sum would be 2 633 00:46:41,475 --> 00:46:44,600 logM to the base 2. 634 00:46:49,660 --> 00:46:49,865 Ok? 635 00:46:49,865 --> 00:46:52,160 So right now, we have pretty much everything we need to 636 00:46:52,160 --> 00:46:54,360 find SNR norm. 637 00:46:54,360 --> 00:47:05,790 The SNR here is Es over N_0, so it's 2 alpha squared, M 638 00:47:05,790 --> 00:47:09,370 squared minus 1, over 3N_0. 639 00:47:09,370 --> 00:47:13,120 But remember, N_0 by definition is 2 sigma squared, 640 00:47:13,120 --> 00:47:15,206 because sigma squared is N_0 over 2. 641 00:47:17,772 --> 00:47:21,140 That's how -- that's the noise variance per dimension. 642 00:47:21,140 --> 00:47:25,830 So, I had 3 times 2 times sigma squared, or I will just 643 00:47:25,830 --> 00:47:30,390 write this as 6 times sigma squared, and that's going to 644 00:47:30,390 --> 00:47:32,280 be -- this is just multiplication, so I should 645 00:47:32,280 --> 00:47:33,180 not even write it -- 646 00:47:33,180 --> 00:47:34,820 six sigma squared. 647 00:47:34,820 --> 00:47:39,650 So this is alpha squared, M squared minus 1 648 00:47:39,650 --> 00:47:43,370 over 3 sigma squared. 649 00:47:43,370 --> 00:47:56,060 OK, so now SNR norm is SNR over 2 to the rho minus 1. 650 00:47:56,060 --> 00:47:59,150 2 to the rho minus 1 is just M squared minus 1. 651 00:47:59,150 --> 00:48:04,030 It cancels with this M squared minus 1, and so I get alpha 652 00:48:04,030 --> 00:48:08,280 squared over 3 sigma squared. 653 00:48:08,280 --> 00:48:11,820 So that's the SNR norm if I use an M-PAM system. 654 00:48:26,730 --> 00:48:27,980 AUDIENCE: Why is [INAUDIBLE] 655 00:48:33,500 --> 00:48:34,150 PROFESSOR: Well, N_0 -- 656 00:48:34,150 --> 00:48:40,830 I've plugged in for SNR 3 here, so I have 3 N_0, but N_0 657 00:48:40,830 --> 00:48:42,080 is 2 sigma squared. 658 00:48:51,015 --> 00:48:52,265 Did I miss anything? 659 00:49:46,202 --> 00:49:46,401 Ok? 660 00:49:46,401 --> 00:49:47,651 Any questions? 661 00:49:50,480 --> 00:49:57,710 OK, so there are two important remarks from this example. 662 00:49:57,710 --> 00:50:11,900 The first remark is that SNR norm is independent of M. So I 663 00:50:11,900 --> 00:50:15,120 started with an M-PAM constellation, so it's a 664 00:50:15,120 --> 00:50:18,570 different constellation for each value of M, right? 665 00:50:18,570 --> 00:50:21,780 If I look at my spectral efficiency rho, it's different 666 00:50:21,780 --> 00:50:24,240 for each value of M, because I can pack more and more bits 667 00:50:24,240 --> 00:50:28,090 per symbol as I increase M. If I look at my signal to noise 668 00:50:28,090 --> 00:50:33,910 ratio, it's also a function of M. But when I took SNR norm, 669 00:50:33,910 --> 00:50:36,870 remarkably, the M squared minus 1 term cancelled out in 670 00:50:36,870 --> 00:50:40,030 the numerator and denominator, and what I was left with was 671 00:50:40,030 --> 00:50:42,770 something independent of M. 672 00:50:42,770 --> 00:50:45,300 So this is actually quite an interesting result. 673 00:50:45,300 --> 00:50:49,060 What it says is suppose I design a system, an M-PAM 674 00:50:49,060 --> 00:50:51,350 system, that has this particular spectral 675 00:50:51,350 --> 00:50:55,620 efficiency, then my gap to capacity is given by this 676 00:50:55,620 --> 00:50:57,000 expression. 677 00:50:57,000 --> 00:51:01,770 If I use a different value of M, my gap to the ultimate 678 00:51:01,770 --> 00:51:05,340 Shannon limit is still given by this expression here. 679 00:51:05,340 --> 00:51:08,630 So by increasing value of M or decreasing the value of M, my 680 00:51:08,630 --> 00:51:12,330 gap to the Shannon limit is still going to be the same. 681 00:51:12,330 --> 00:51:14,200 For each value of M, I will have a different spectral 682 00:51:14,200 --> 00:51:17,920 efficiency, but I'm not getting any kind of coding 683 00:51:17,920 --> 00:51:20,380 gain, if you will, here. 684 00:51:20,380 --> 00:51:28,900 OK, so this motivates that M-PAM is an uncoded system. 685 00:51:34,240 --> 00:51:37,500 All of them have the same gap to the Shannon limit, 686 00:51:37,500 --> 00:51:40,505 regardless of what the value of M is, and so. 687 00:51:46,340 --> 00:51:51,550 The second point to note is if I look at the value of alpha 688 00:51:51,550 --> 00:51:54,420 squared over 3 sigma square, I can make it 689 00:51:54,420 --> 00:51:56,110 quite small, right? 690 00:51:56,110 --> 00:51:59,770 I can even, if I decrease alpha really by a great 691 00:51:59,770 --> 00:52:03,620 amount, I can make this quantity smaller than 1. 692 00:52:03,620 --> 00:52:04,600 OK? 693 00:52:04,600 --> 00:52:09,890 I told you here that SNR norm is always greater than 1. 694 00:52:09,890 --> 00:52:11,890 So what happened? 695 00:52:11,890 --> 00:52:14,580 Did I lie to you here, or did I do something wrong here? 696 00:52:18,570 --> 00:52:20,910 I mean, I can choose any alpha, right, and make this 697 00:52:20,910 --> 00:52:24,300 quantity as small as I please, and then I'm doing better than 698 00:52:24,300 --> 00:52:26,561 the Shannon limit. 699 00:52:26,561 --> 00:52:27,811 AUDIENCE: [UNINTELLIGIBLE] 700 00:52:29,900 --> 00:52:30,270 PROFESSOR: Right. 701 00:52:30,270 --> 00:52:33,310 So basically, what's missing in this calculation is 702 00:52:33,310 --> 00:52:34,940 probability of error. 703 00:52:34,940 --> 00:52:38,240 If I make alpha really small, what I have is all these 704 00:52:38,240 --> 00:52:41,560 points come closer and closer, and sure, I seem like I'm 705 00:52:41,560 --> 00:52:42,800 doing very well at the encoder. 706 00:52:42,800 --> 00:52:44,880 But what happens at the decoder? 707 00:52:44,880 --> 00:52:48,410 There is noise in the system, and so I get too many errors. 708 00:52:48,410 --> 00:52:51,070 This lower bound clearly assumes that you can make your 709 00:52:51,070 --> 00:52:53,000 probability of error quite small, 710 00:52:53,000 --> 00:52:54,800 arbitrarily small, right? 711 00:52:54,800 --> 00:52:57,410 So in any reasonable system, I should also look at the 712 00:52:57,410 --> 00:52:59,020 probability of error. 713 00:52:59,020 --> 00:53:01,540 If I make alpha really small, I'm going to have too much 714 00:53:01,540 --> 00:53:07,270 error at the decoder, and so I won't be able to have a 715 00:53:07,270 --> 00:53:08,520 practical system. 716 00:53:12,000 --> 00:53:23,870 So the comment is SNR norm cannot be seen in isolation. 717 00:53:28,700 --> 00:53:31,050 We need to couple SNR norm with the corresponding 718 00:53:31,050 --> 00:53:32,330 probability of error. 719 00:53:32,330 --> 00:53:32,703 Yes? 720 00:53:32,703 --> 00:53:35,240 AUDIENCE: [UNINTELLIGIBLE] 721 00:53:35,240 --> 00:53:37,450 PROFESSOR: What I mean by uncoded syst -- 722 00:53:37,450 --> 00:53:38,930 that's a good question. 723 00:53:38,930 --> 00:53:41,350 What do I mean by uncoded system? 724 00:53:41,350 --> 00:53:45,390 All I'm really saying here is M-PAM system is a fairly 725 00:53:45,390 --> 00:53:47,730 simple system to implement, right? 726 00:53:47,730 --> 00:53:51,420 Irregardless of what value of M I get, I have a certain gap 727 00:53:51,420 --> 00:53:54,940 to the Shannon limit, which is independent of M. So if I 728 00:53:54,940 --> 00:53:58,230 start with a simple system, where I have bits coming in, 729 00:53:58,230 --> 00:54:00,840 each bit gets mapped to one symbol, and I send it over the 730 00:54:00,840 --> 00:54:03,980 channel, I have a fixed gap to the Shannon capacity. 731 00:54:03,980 --> 00:54:06,360 And this, I will call an uncoded system. 732 00:54:06,360 --> 00:54:09,930 My only hope will be to improve upon this system. 733 00:54:09,930 --> 00:54:11,190 OK? 734 00:54:11,190 --> 00:54:13,168 Any other questions? 735 00:54:13,168 --> 00:54:15,144 AUDIENCE: So, if I'm multiplying [INAUDIBLE] 736 00:54:19,100 --> 00:54:19,790 PROFESSOR: Right, right. 737 00:54:19,790 --> 00:54:22,263 I'm assuming the fixed alpha. 738 00:54:22,263 --> 00:54:24,190 AUDIENCE: [UNINTELLIGIBLE] 739 00:54:24,190 --> 00:54:27,350 PROFESSOR: I mean, basically, you will see that alpha is a 740 00:54:27,350 --> 00:54:28,895 function of Es, right? 741 00:54:28,895 --> 00:54:30,145 And I want to -- 742 00:54:36,890 --> 00:54:38,870 so why do I want -- the question is, why do I want to 743 00:54:38,870 --> 00:54:41,880 keep alpha fixed, right? 744 00:54:41,880 --> 00:54:42,140 AUDIENCE: Yeah. 745 00:54:42,140 --> 00:54:44,300 PROFESSOR: OK, so in order to understand that, I have to 746 00:54:44,300 --> 00:54:47,840 look at energy for two dimensions. 747 00:54:47,840 --> 00:54:50,040 If I normalize by M squared minus one -- 748 00:54:52,650 --> 00:54:53,900 that doesn't work out. 749 00:55:01,340 --> 00:55:02,590 AUDIENCE: [UNINTELLIGIBLE] 750 00:55:08,430 --> 00:55:10,875 so the constellation cannot be expanding [UNINTELLIGIBLE] 751 00:55:13,810 --> 00:55:15,600 PROFESSOR: Right, so I wanted to always plot by keeping 752 00:55:15,600 --> 00:55:16,370 alpha fixed. 753 00:55:16,370 --> 00:55:19,610 And I mean, the point is, typically you want to plot the 754 00:55:19,610 --> 00:55:22,190 probability of symbol error -- 755 00:55:22,190 --> 00:55:25,460 do I have that expression anywhere? 756 00:55:25,460 --> 00:55:26,160 Right there. 757 00:55:26,160 --> 00:55:29,030 Ps of p is a function of SNR norm, right? 758 00:55:29,030 --> 00:55:31,890 And that will be -- if I increase SNR norm, so it will 759 00:55:31,890 --> 00:55:33,310 be some kind of a graph, right? 760 00:55:33,310 --> 00:55:34,370 A trade-off. 761 00:55:34,370 --> 00:55:37,380 Basically, alpha will define the trade-off between SNR norm 762 00:55:37,380 --> 00:55:39,640 and Ps of p. 763 00:55:39,640 --> 00:55:43,810 So as I slip over larger and larger values of alpha, for 764 00:55:43,810 --> 00:55:48,160 different values of M, then I will have a trade-off as Ps of 765 00:55:48,160 --> 00:55:50,840 p is a function of SNR norm. 766 00:55:50,840 --> 00:55:52,990 Does that make sense? 767 00:55:52,990 --> 00:55:56,370 So say I fixed a value of M, OK? 768 00:55:56,370 --> 00:55:58,740 I define SNR norm, which is alpha squared 769 00:55:58,740 --> 00:56:00,390 over 3 sigma squared. 770 00:56:00,390 --> 00:56:03,450 For this, I will get a certain probability of error now. 771 00:56:03,450 --> 00:56:05,920 So if I fix this value of SNR norm, I get a certain 772 00:56:05,920 --> 00:56:07,410 probability of error. 773 00:56:07,410 --> 00:56:09,220 What if I want to increase my SNR norm? 774 00:56:09,220 --> 00:56:11,430 The only way to do that will be to increase alpha. 775 00:56:15,340 --> 00:56:18,020 Does that make sense? 776 00:56:18,020 --> 00:56:19,140 Yeah. 777 00:56:19,140 --> 00:56:20,390 AUDIENCE: [UNINTELLIGIBLE] 778 00:56:22,605 --> 00:56:28,980 You're defining SNR norm as the gap to capacity, but it 779 00:56:28,980 --> 00:56:32,516 seems like, I mean, obviously as you increase alpha, as you 780 00:56:32,516 --> 00:56:34,350 increase your signal energy, you're going to do better and 781 00:56:34,350 --> 00:56:35,020 better, right? 782 00:56:35,020 --> 00:56:35,930 PROFESSOR: Right. 783 00:56:35,930 --> 00:56:37,420 AUDIENCE: Well, in terms of what? 784 00:56:37,420 --> 00:56:39,630 In terms of probability of error, like achievable 785 00:56:39,630 --> 00:56:41,300 probability of error, or? 786 00:56:41,300 --> 00:56:42,220 PROFESSOR: Right, exactly. 787 00:56:42,220 --> 00:56:48,460 So basically, the point is say I plot Ps of E as a function 788 00:56:48,460 --> 00:56:50,610 of SNR norm, or as a function of alpha squared 789 00:56:50,610 --> 00:56:52,700 over 3 sigma squared. 790 00:56:52,700 --> 00:56:54,868 So my core will look like this. 791 00:56:54,868 --> 00:56:58,272 AUDIENCE: But the gap to capacity is defined all the 792 00:56:58,272 --> 00:56:59,760 way over here on the left, is that what--? 793 00:56:59,760 --> 00:57:01,100 PROFESSOR: Right, that's a very good question. 794 00:57:01,100 --> 00:57:03,560 You are going much ahead then what I thought. 795 00:57:03,560 --> 00:57:09,250 So basically, at zero, seven, SNR norm is zero, right? 796 00:57:09,250 --> 00:57:11,270 This is in linear scale, so when I have 797 00:57:11,270 --> 00:57:13,500 one, SNR norm is one. 798 00:57:13,500 --> 00:57:15,940 I have the Shannon system. 799 00:57:15,940 --> 00:57:19,890 Basically, for any SNR norm greater than 1, what Shannon 800 00:57:19,890 --> 00:57:21,990 says is your probability of error will be arbitrarily 801 00:57:21,990 --> 00:57:25,290 small, and here, it will be large. 802 00:57:25,290 --> 00:57:27,500 So this here is the Shannon limit. 803 00:57:27,500 --> 00:57:30,120 And now, in a practical system, what I want to do is I 804 00:57:30,120 --> 00:57:32,080 want to fix the probability of error. 805 00:57:32,080 --> 00:57:33,840 So say I like probability of error of ten 806 00:57:33,840 --> 00:57:35,620 to the minus five. 807 00:57:35,620 --> 00:57:38,520 So that's something that the system specifies. 808 00:57:38,520 --> 00:57:41,970 And from that, I know the gap to the capacity. 809 00:57:41,970 --> 00:57:44,800 So if here, if I require, this is my SNR norm, then this is 810 00:57:44,800 --> 00:57:45,980 going to be my gap. 811 00:57:45,980 --> 00:57:47,630 I wanted to cover it, but later. 812 00:57:47,630 --> 00:57:50,372 That's a good point. 813 00:57:50,372 --> 00:57:51,622 AUDIENCE: [UNINTELLIGIBLE] 814 00:58:00,725 --> 00:58:04,150 PROFESSOR: Right, this is a certain, specific system, the 815 00:58:04,150 --> 00:58:07,110 M-PAM system. 816 00:58:07,110 --> 00:58:10,440 What Shannon says that, if you're anywhere here, you 817 00:58:10,440 --> 00:58:12,030 should be able to make your probability of error 818 00:58:12,030 --> 00:58:14,540 arbitrarily small. 819 00:58:14,540 --> 00:58:17,627 So your code should basically look something like this, if 820 00:58:17,627 --> 00:58:19,170 you will, or even steeper. 821 00:58:21,850 --> 00:58:24,176 AUDIENCE: What's the difference between that curve 822 00:58:24,176 --> 00:58:26,030 and the wider curve? 823 00:58:26,030 --> 00:58:28,030 PROFESSOR: This curve and this curve? 824 00:58:28,030 --> 00:58:28,860 AUDIENCE: Yeah. 825 00:58:28,860 --> 00:58:29,890 PROFESSOR: This is basically what we 826 00:58:29,890 --> 00:58:32,390 achieved by an M-PAM system. 827 00:58:32,390 --> 00:58:34,240 I don't want to go into too much of this, because we'll be 828 00:58:34,240 --> 00:58:35,070 doing this later. 829 00:58:35,070 --> 00:58:38,190 This is the next topic. 830 00:58:38,190 --> 00:58:41,370 Ideally, what we would want is something, basically, this is 831 00:58:41,370 --> 00:58:43,450 my Shannon limit, I want something such as the 832 00:58:43,450 --> 00:58:46,590 probability of error decreases right away as soon as I am 833 00:58:46,590 --> 00:58:47,890 just away from the Shannon limit. 834 00:58:52,000 --> 00:58:52,590 All right. 835 00:58:52,590 --> 00:58:54,680 AUDIENCE: [UNINTELLIGIBLE] 836 00:58:54,680 --> 00:58:55,010 PROFESSOR: Right. 837 00:58:55,010 --> 00:58:56,380 So that's a good point. 838 00:58:56,380 --> 00:58:58,700 So if I did not do coding, this is what I get. 839 00:58:58,700 --> 00:59:00,630 If I did coding, this is what I would get. 840 00:59:00,630 --> 00:59:04,238 So this is how much I can expect a gain from coding. 841 00:59:04,238 --> 00:59:06,310 AUDIENCE: So in other words, basically, this bound is 842 00:59:06,310 --> 00:59:10,550 basically in my channel, the SNR is such that -- 843 00:59:10,550 --> 00:59:14,785 I mean, I'm so energy limited that my SNR norm is lower than 844 00:59:14,785 --> 00:59:17,670 a certain amount, there's nothing I can do, basically, 845 00:59:17,670 --> 00:59:17,930 is what it's saying. 846 00:59:17,930 --> 00:59:18,210 PROFESSOR: Right. 847 00:59:18,210 --> 00:59:20,970 Because if you want a certain spectral efficiency, there's 848 00:59:20,970 --> 00:59:24,010 also spectral efficiency, right? 849 00:59:24,010 --> 00:59:26,700 And if your SNR is much lower, you're out of luck. 850 00:59:26,700 --> 00:59:28,398 AUDIENCE: So basically, you have to make your spectral 851 00:59:28,398 --> 00:59:29,790 efficiency higher -- 852 00:59:29,790 --> 00:59:30,720 or, lower. 853 00:59:30,720 --> 00:59:31,560 PROFESSOR: Lower, right. 854 00:59:31,560 --> 00:59:32,810 Exactly. 855 01:00:13,380 --> 01:00:16,280 So now let us start with the probability of error analysis, 856 01:00:16,280 --> 01:00:19,200 and we'll start in limited regime. 857 01:00:19,200 --> 01:00:22,840 So we have Eb of E as a function of Eb/N_0, we want to 858 01:00:22,840 --> 01:00:24,370 quantify this trade-off. 859 01:00:24,370 --> 01:00:27,110 Basically, what we are doing now is trying to quantify how 860 01:00:27,110 --> 01:00:29,430 this graph will look like, at least for the 861 01:00:29,430 --> 01:00:30,970 uncoded systems today. 862 01:00:30,970 --> 01:00:34,270 We'll do coding next class, or two classes from 863 01:00:34,270 --> 01:00:36,780 now, as we get time. 864 01:00:36,780 --> 01:00:41,440 So say I have a constellation, A, as binary PAM. 865 01:00:41,440 --> 01:00:45,230 So it takes only two values, minus alpha and alpha. 866 01:00:45,230 --> 01:00:49,430 So I have these two points here, alpha and minus alpha. 867 01:00:49,430 --> 01:00:56,030 My receiveds are Y, symbol Y is X plus n, where X belongs 868 01:00:56,030 --> 01:01:02,570 to A, and N is Gaussian zero mean, variance sigma squared, 869 01:01:02,570 --> 01:01:07,990 where sigma squared is N_0 over 2. 870 01:01:07,990 --> 01:01:11,130 And now what want to do at the receiver is given Y, you want 871 01:01:11,130 --> 01:01:14,610 to decide whether X was alpha or minus alpha. 872 01:01:14,610 --> 01:01:17,110 That's a standard detection problem. 873 01:01:17,110 --> 01:01:20,590 So how will the probability of error look like? 874 01:01:20,590 --> 01:01:24,070 So suppose I transmit X equals alpha, my conditional 875 01:01:24,070 --> 01:01:26,140 density of Y -- 876 01:01:26,140 --> 01:01:27,390 let me make some room here -- 877 01:01:32,270 --> 01:01:36,180 will be a bell shaped curve, because of the Gaussian noise. 878 01:01:36,180 --> 01:01:40,670 So this is P of Y given X equals alpha. 879 01:01:40,670 --> 01:01:43,830 Similarly, if I have set X equals minus alpha, what I get 880 01:01:43,830 --> 01:01:45,700 is something like this. 881 01:01:45,700 --> 01:01:50,170 This is p of Y given X equals minus alpha. 882 01:01:50,170 --> 01:01:52,132 Excuse my handwriting there. 883 01:01:52,132 --> 01:01:57,350 And the decision region is at the midpoint, Y equals zero. 884 01:01:57,350 --> 01:01:59,880 This is a standard binary detection problem. 885 01:01:59,880 --> 01:02:02,880 If Y is positive, you will say X equals alpha was sent. 886 01:02:02,880 --> 01:02:07,330 If Y is negative, you will say X equals minus alpha was sent. 887 01:02:07,330 --> 01:02:10,570 And your probability of error is simply the probability of 888 01:02:10,570 --> 01:02:12,840 error under this -- 889 01:02:12,840 --> 01:02:15,420 graph under these two curves here. 890 01:02:15,420 --> 01:02:19,080 So we want to find what the probability of error is. 891 01:02:19,080 --> 01:02:21,980 Just let me just do quickly the calculation, in order to 892 01:02:21,980 --> 01:02:22,890 remind you. 893 01:02:22,890 --> 01:02:24,575 We'll be doing this over and over again. 894 01:02:24,575 --> 01:02:25,825 We'll just do it once. 895 01:02:28,860 --> 01:02:32,260 Without loss in generality, I can say this is probability of 896 01:02:32,260 --> 01:02:35,900 error, given X equals minus alpha, where both the points 897 01:02:35,900 --> 01:02:37,220 are equally likely. 898 01:02:37,220 --> 01:02:39,490 So by symmetry, I can say that. 899 01:02:39,490 --> 01:02:44,100 So this is the same as probability that Y is greater 900 01:02:44,100 --> 01:02:46,820 than zero, given X is minus alpha. 901 01:02:46,820 --> 01:02:52,320 Now Y is X plus N, so this is same as probability that N -- 902 01:02:55,610 --> 01:02:58,350 yeah capital N -- 903 01:02:58,350 --> 01:02:59,310 is greater than alpha. 904 01:02:59,310 --> 01:03:02,170 Since N has zero mean variance sigma squared, this is a 905 01:03:02,170 --> 01:03:06,618 standard Q function of alpha over sigma. 906 01:03:06,618 --> 01:03:09,230 OK, so that's the probability of error. 907 01:03:09,230 --> 01:03:12,310 Now, there is one bit per symbol for each X that is one 908 01:03:12,310 --> 01:03:15,200 bit, so probability of error is also same as probability of 909 01:03:15,200 --> 01:03:23,490 bit error, so this is also Pb of E. So I have Pb of E equals 910 01:03:23,490 --> 01:03:34,720 Q of alpha over sigma, and now, I want to basically 911 01:03:34,720 --> 01:03:37,400 express it as a function of Eb/N_0, right. 912 01:03:37,400 --> 01:03:39,200 So what is Eb for the system? 913 01:03:42,540 --> 01:03:45,890 It's going to be alpha squared. 914 01:03:45,890 --> 01:03:50,540 Sigma squared is now N_0 over 2, so alpha squared over sigma 915 01:03:50,540 --> 01:03:55,500 squared is 2 Eb over N_0. 916 01:03:55,500 --> 01:04:00,920 So now my probability of bit error is this Q function of 917 01:04:00,920 --> 01:04:02,170 root 2 Eb over N_0. 918 01:04:07,820 --> 01:04:08,580 OK? 919 01:04:08,580 --> 01:04:09,850 So let us plot this. 920 01:04:12,570 --> 01:04:16,680 So I'm on the x-axis, I'm plotting Eb/N_0, 921 01:04:16,680 --> 01:04:19,640 which is in dB scale. 922 01:04:19,640 --> 01:04:21,920 On the y-axis, I'm going to plot the 923 01:04:21,920 --> 01:04:26,070 probability of bit error. 924 01:04:26,070 --> 01:04:28,820 Typically, what you do is you plot the y-axis 925 01:04:28,820 --> 01:04:30,690 on a semi-log scale. 926 01:04:30,690 --> 01:04:35,410 So this is ten to the minus six, ten to the minus five. 927 01:04:35,410 --> 01:04:37,390 If you want to do this in MATLAB, you can use this 928 01:04:37,390 --> 01:04:39,920 command semi-log y, and that does it. 929 01:04:43,130 --> 01:04:44,570 And so on. 930 01:04:44,570 --> 01:04:49,070 So can anybody say what will be a good candidate for the 931 01:04:49,070 --> 01:04:50,570 x-value at this point here? 932 01:04:53,840 --> 01:04:55,590 So what should be the Eb/N_0 at this point? 933 01:05:00,430 --> 01:05:01,890 AUDIENCE: [UNINTELLIGIBLE] 934 01:05:01,890 --> 01:05:04,210 PROFESSOR: It should be the Shannon limit, right? 935 01:05:04,210 --> 01:05:07,350 You cannot hope to go below the Shannon limit. 936 01:05:07,350 --> 01:05:12,050 So now, what Shannon says is that -- 937 01:05:12,050 --> 01:05:14,580 so I'm going to plot this as my Shannon limit here. 938 01:05:19,650 --> 01:05:22,460 This probability of error will basically use the standard 939 01:05:22,460 --> 01:05:25,100 waterfall curve. 940 01:05:25,100 --> 01:05:28,670 This is 2-PAM. 941 01:05:31,970 --> 01:05:34,460 And if you look at the x-coordinates, then the value 942 01:05:34,460 --> 01:05:38,350 at ten to the minus five is 9.6 dB. 943 01:05:47,840 --> 01:05:51,370 So as a system designer, if you care about probability of 944 01:05:51,370 --> 01:05:54,780 error at ten to the negative five, then your gap to 945 01:05:54,780 --> 01:05:57,800 capacity, or gap to the ultimate limit, if you will, 946 01:05:57,800 --> 01:06:02,665 will be this, 9.6 minus of minus 1.59 dB. 947 01:06:02,665 --> 01:06:04,260 OK, I should not erase this. 948 01:07:02,980 --> 01:07:21,770 OK, so the first thing is at N to the minus five, our gap to 949 01:07:21,770 --> 01:07:31,590 the ultimate limit is 9.6 plus 1.59 dB, and that's 950 01:07:31,590 --> 01:07:37,570 approximately 11.2 dB. 951 01:07:37,570 --> 01:07:38,650 OK? 952 01:07:38,650 --> 01:07:40,990 But there is one catch to this. 953 01:07:40,990 --> 01:07:44,025 This particular system has a spectral efficiency of two 954 01:07:44,025 --> 01:07:47,480 bits for two dimensions, whereas if you want to achieve 955 01:07:47,480 --> 01:07:50,150 something close to the Shanon limit, you have to drive the 956 01:07:50,150 --> 01:07:52,630 spectral efficiency down to zero. 957 01:07:52,630 --> 01:07:55,750 So you might say that this is not a fair comparison. 958 01:07:55,750 --> 01:07:58,770 So if you do want to make a fair comparison, you want to 959 01:07:58,770 --> 01:08:02,030 fix rho to be two bits for two dimensions. 960 01:08:02,030 --> 01:08:06,540 So if you fix rho for two bits per two dimensions, you will 961 01:08:06,540 --> 01:08:10,460 get a limit somewhere, not at 1.59 dB, but 962 01:08:10,460 --> 01:08:12,190 at some other point. 963 01:08:12,190 --> 01:08:18,895 This is if you fix rho2, two bits for two dimensions. 964 01:08:18,895 --> 01:08:20,970 Can anybody say what that point will be? 965 01:08:29,934 --> 01:08:31,939 AUDIENCE: [UNINTELLIGIBLE] 966 01:08:31,939 --> 01:08:32,990 PROFESSOR: 3 over 2. 967 01:08:32,990 --> 01:08:33,939 1.5. 968 01:08:33,939 --> 01:08:37,304 What will it be in the log scale? 969 01:08:37,304 --> 01:08:38,290 AUDIENCE: [UNINTELLIGIBLE] 970 01:08:38,290 --> 01:08:39,450 PROFESSOR: 1.76. 971 01:08:39,450 --> 01:08:40,380 Good. 972 01:08:40,380 --> 01:08:41,810 So what we do know -- 973 01:08:41,810 --> 01:08:43,074 if we do the calculation here. 974 01:08:45,819 --> 01:08:54,160 If you fix rho2, two bits for two dimensions, your Eb/N_0, 975 01:08:54,160 --> 01:08:58,670 we know, is greater than 2 to the rho minus 1 over rho, 976 01:08:58,670 --> 01:09:00,840 which is 3 over 2. 977 01:09:00,840 --> 01:09:06,260 And that is, if you remember your dB tables, 1.76 dB. 978 01:09:06,260 --> 01:09:16,430 Log of 3 is like 4.7, log of 2 is like 3, so it's 1.76 dB. 979 01:09:16,430 --> 01:09:31,470 So in this case, your gap to the ultimate limit is going to 980 01:09:31,470 --> 01:09:39,950 be 9.6 minus 1.76, which comes out to be 7.8 dB. 981 01:09:42,896 --> 01:09:45,359 OK? 982 01:09:45,359 --> 01:09:48,159 So now, let us do the bandwidth-limited regime. 983 01:10:07,160 --> 01:10:10,380 Now in bandwidth-limited regime, the trade-off is Ps of 984 01:10:10,380 --> 01:10:13,290 E as a function of SNR norm. 985 01:10:17,950 --> 01:10:20,010 OK, so that's the trade-off we seek for. 986 01:10:20,010 --> 01:10:21,320 And the baseline system we will be 987 01:10:21,320 --> 01:10:22,660 using is an M-PAM system. 988 01:10:28,810 --> 01:10:38,890 So now, your constellation is going to be minus alpha, 989 01:10:38,890 --> 01:10:43,420 alpha, 3 alpha, minus 3 alpha, and so on, up to M 990 01:10:43,420 --> 01:10:45,720 minus 1 alpha -- 991 01:10:45,720 --> 01:10:48,170 I always run out of room here -- 992 01:10:48,170 --> 01:10:50,810 minus M minus 1 alpha. 993 01:10:50,810 --> 01:10:53,170 This is your constellation. 994 01:10:53,170 --> 01:10:55,990 Now what's the probability of error going to be for this 995 01:10:55,990 --> 01:10:57,240 constellation? 996 01:11:01,280 --> 01:11:03,450 Well, what's the probability of error for each 997 01:11:03,450 --> 01:11:05,710 intermediate point? 998 01:11:05,710 --> 01:11:07,440 There are two ways you can make an error. 999 01:11:07,440 --> 01:11:11,300 Say we say alpha, your noise is either too small, so it 1000 01:11:11,300 --> 01:11:13,330 takes you to minus alpha. 1001 01:11:13,330 --> 01:11:17,710 Your noise is too high, so it takes you to 3 alpha. 1002 01:11:17,710 --> 01:11:21,410 For each one, the probability will be Q of -- 1003 01:11:21,410 --> 01:11:24,030 this distance is 2 alpha, it would be Q of alpha over 1004 01:11:24,030 --> 01:11:26,820 sigma, because you will be having your decision regions 1005 01:11:26,820 --> 01:11:27,870 right here. 1006 01:11:27,870 --> 01:11:30,260 So in other words, for each intermediate point -- and 1007 01:11:30,260 --> 01:11:32,170 there are M minus two of this -- 1008 01:11:32,170 --> 01:11:35,520 your probability of error would be 2 times Q of alpha 1009 01:11:35,520 --> 01:11:37,710 over sigma. 1010 01:11:37,710 --> 01:11:38,160 OK? 1011 01:11:38,160 --> 01:11:39,670 And how many of them there are? 1012 01:11:39,670 --> 01:11:42,230 There are M minus 2 of these. 1013 01:11:42,230 --> 01:11:44,860 And if all the points are equally likely, and you want 1014 01:11:44,860 --> 01:11:47,450 to find the probability of error, you divide by M. 1015 01:11:47,450 --> 01:11:50,420 For the two end points, the noise can only make an error 1016 01:11:50,420 --> 01:11:52,080 in one direction. 1017 01:11:52,080 --> 01:11:55,820 So you get two over M times Q of alpha over sigma. 1018 01:11:55,820 --> 01:11:59,340 You work this out, and you have two times M minus 1 over 1019 01:11:59,340 --> 01:12:02,465 M, times Q of alpha over sigma. 1020 01:12:05,260 --> 01:12:08,310 Now, we want to find probability of error for two 1021 01:12:08,310 --> 01:12:12,830 symbols, because that's what Ps of E is, OK? 1022 01:12:12,830 --> 01:12:17,435 So what's Ps of E going to be? 1023 01:12:17,435 --> 01:12:20,980 Well, it's -- in terms of Pr of E, it's going to be one 1024 01:12:20,980 --> 01:12:25,290 minus one minus Pr of E squared. 1025 01:12:25,290 --> 01:12:28,977 This is the probability you make error in none of the two 1026 01:12:28,977 --> 01:12:31,310 symbols, and 1 minus that will be the probability of error 1027 01:12:31,310 --> 01:12:33,730 you make in at least one symbol. 1028 01:12:33,730 --> 01:12:39,890 And that's going to be equal to two Pr of E, or it's four 1029 01:12:39,890 --> 01:12:45,270 times M minus 1 over M, Q of alpha over sigma. 1030 01:12:51,880 --> 01:12:53,395 Good, I did not write on this board. 1031 01:13:01,500 --> 01:13:04,130 So now what remains to do is to relate alpha over 1032 01:13:04,130 --> 01:13:06,980 sigma to SNR norm. 1033 01:13:06,980 --> 01:13:10,360 And I had it on this board here. 1034 01:13:10,360 --> 01:13:16,370 SNR norm, we just did in the previous example, for an M-PAM 1035 01:13:16,370 --> 01:13:21,940 system is alpha squared over 3 sigma squared. 1036 01:13:21,940 --> 01:13:22,770 OK? 1037 01:13:22,770 --> 01:13:29,120 So I plug that in here, so I get Ps of E is 4 times M minus 1038 01:13:29,120 --> 01:13:35,475 1 over M times Q of root 3 SNR norm. 1039 01:13:40,430 --> 01:13:46,320 And if M is large, this is approximately 4 times Q of 1040 01:13:46,320 --> 01:13:48,302 root 3 SNR norm. 1041 01:13:51,440 --> 01:13:53,720 So we are plotting now probability of error as a 1042 01:13:53,720 --> 01:13:58,600 function of SNR norm, similar to what we did in that part in 1043 01:13:58,600 --> 01:14:01,590 the power-limited regime there. 1044 01:14:01,590 --> 01:14:05,810 So this is Ps of E as a function of SNR norm. 1045 01:14:12,000 --> 01:14:16,260 Now my Shannon limit would be at 0 dB, so this is my Shannon 1046 01:14:16,260 --> 01:14:17,510 limit point. 1047 01:14:20,610 --> 01:14:24,912 This is ten to the minus six, ten to the minus five, ten to 1048 01:14:24,912 --> 01:14:28,630 the minus four, ten to the minus three, ten to the minus 1049 01:14:28,630 --> 01:14:29,880 two, and so on. 1050 01:14:33,010 --> 01:14:36,490 This is going to be the performance of M-PAM. 1051 01:14:36,490 --> 01:14:40,020 And again, I turn to the minus five, which will be the kind 1052 01:14:40,020 --> 01:14:41,840 of performance criteria we'll be using 1053 01:14:41,840 --> 01:14:43,090 throughout this course. 1054 01:14:45,690 --> 01:14:51,340 You'll see that this y-axis is 8.4 dB. 1055 01:14:51,340 --> 01:15:01,730 So in this case, the gap to capacity is 1056 01:15:01,730 --> 01:15:06,000 going to be 8.4 dB. 1057 01:15:06,000 --> 01:15:08,400 So if I want a certain spectral efficiency and I use 1058 01:15:08,400 --> 01:15:13,540 M-PAM, I'm 8.4 dB away from the Shannon limit. 1059 01:15:13,540 --> 01:15:17,490 The idea behind coding, as someone pointed out, was to 1060 01:15:17,490 --> 01:15:21,170 start from here and do coding to come closer and closer to 1061 01:15:21,170 --> 01:15:23,420 bridge this gap. 1062 01:15:23,420 --> 01:15:24,670 OK? 1063 01:15:27,010 --> 01:15:28,330 So are there any questions? 1064 01:15:28,330 --> 01:15:29,580 We are almost end time. 1065 01:15:34,390 --> 01:15:34,960 OK. 1066 01:15:34,960 --> 01:15:36,022 I think -- yes? 1067 01:15:36,022 --> 01:15:37,870 AUDIENCE: A perfect code would just be literally like a step 1068 01:15:37,870 --> 01:15:38,170 function -- 1069 01:15:38,170 --> 01:15:38,750 PROFESSOR: Right. 1070 01:15:38,750 --> 01:15:40,025 AUDIENCE: -- all the way down. 1071 01:15:40,025 --> 01:15:43,200 PROFESSOR: That's what Shannon coded. 1072 01:15:43,200 --> 01:15:46,070 OK, I think this is a natural point to stop. 1073 01:15:46,070 --> 01:15:47,320 We'll continue next class.