1 00:00:00,000 --> 00:00:13,560 PROFESSOR: So the idea behind the encoder is increase the 2 00:00:13,560 --> 00:00:23,760 minimum distance at the cost of spectral efficiency. 3 00:00:32,170 --> 00:00:38,070 The idea behind the decoder was the following. 4 00:00:38,070 --> 00:00:41,710 You have a received signal, y. 5 00:00:41,710 --> 00:00:49,930 And the job of the decoder is to find x hat, which belongs 6 00:00:49,930 --> 00:00:54,660 to the signal set so that you minimize your 7 00:00:54,660 --> 00:00:55,920 probability of error. 8 00:00:55,920 --> 00:01:01,840 And the probability of error x is not equal to x-hat. 9 00:01:01,840 --> 00:01:05,780 That's the probability of error for your decoder. 10 00:01:05,780 --> 00:01:09,090 And we saw a bunch of equivalence rules -- we said 11 00:01:09,090 --> 00:01:12,080 that the minimum probability of error rule, which is this 12 00:01:12,080 --> 00:01:18,900 rule here, is the same as the maximum a posteriori rule. 13 00:01:18,900 --> 00:01:22,410 And this just follows by the definition. 14 00:01:22,410 --> 00:01:28,160 This was the same as the maximum likelihood rule. 15 00:01:28,160 --> 00:01:30,350 And here we made one assumption that all the 16 00:01:30,350 --> 00:01:35,840 signals in your signal set A are equally likely. 17 00:01:35,840 --> 00:01:39,660 And this was shown to be equivalent to your minimum 18 00:01:39,660 --> 00:01:42,130 distance decision rule. 19 00:01:42,130 --> 00:01:45,580 And in order to show this equivalence, we use the fact 20 00:01:45,580 --> 00:01:48,630 that the noise was White and Gaussian. 21 00:01:48,630 --> 00:01:52,570 So this assumes equi-probable signals. 22 00:01:59,290 --> 00:02:01,265 And this assumes AWGN channel. 23 00:02:01,265 --> 00:02:02,515 OK. 24 00:02:08,259 --> 00:02:11,840 In fact, the minimum distance decisions has very nice 25 00:02:11,840 --> 00:02:13,090 geometrical properties. 26 00:02:17,720 --> 00:02:20,190 We basically said that the decision regions had a 27 00:02:20,190 --> 00:02:21,250 particular shape. 28 00:02:21,250 --> 00:02:23,100 They were convex polytopes. 29 00:02:23,100 --> 00:02:27,720 And they were also known as Voronoi regions. 30 00:02:27,720 --> 00:02:30,370 However, finding the exact expression for the probability 31 00:02:30,370 --> 00:02:33,850 of error seemed quite tedious so we decided 32 00:02:33,850 --> 00:02:35,640 to settle with bounds. 33 00:02:35,640 --> 00:02:38,200 And we developed some bounds for the probability of error. 34 00:02:42,530 --> 00:02:46,140 And the main bounds were -- we had a strict upper bound, 35 00:02:46,140 --> 00:02:46,937 which was the summation. 36 00:02:46,937 --> 00:02:48,187 OK. 37 00:03:01,220 --> 00:03:03,230 This is a strict upper bound. 38 00:03:03,230 --> 00:03:06,890 Well, the inequality here, say it's a strict upper bound. 39 00:03:06,890 --> 00:03:10,450 We also had a union bound estimate. 40 00:03:10,450 --> 00:03:12,910 Because this is a sum of various Q functions, this is 41 00:03:12,910 --> 00:03:16,270 still a very cumbersome expression to evaluate. 42 00:03:16,270 --> 00:03:18,440 So we wanted a simpler expression. 43 00:03:18,440 --> 00:03:28,650 So we then approximated this by K_min of aj times Q of 44 00:03:28,650 --> 00:03:33,410 d_min over 2 sigma. 45 00:03:33,410 --> 00:03:37,750 OK, and now if we take the average on both sides, what we 46 00:03:37,750 --> 00:03:43,650 get is the probability of error is approximately equal 47 00:03:43,650 --> 00:03:47,980 to K_min of the constellation, which is the average number of 48 00:03:47,980 --> 00:03:55,890 nearest neighbors, times Q of d_min over 2 sigma. 49 00:03:58,690 --> 00:04:02,600 And this expression is known as the union bound estimate. 50 00:04:10,280 --> 00:04:13,120 We call it an estimate because it's not a bound anymore. 51 00:04:13,120 --> 00:04:16,370 We have made an approximation going from here to here. 52 00:04:16,370 --> 00:04:18,910 So this is what we came up with last time. 53 00:04:18,910 --> 00:04:23,076 Throughout the analysis today, whenever we have a probability 54 00:04:23,076 --> 00:04:25,970 of error expression, we will be estimating it by the union 55 00:04:25,970 --> 00:04:27,170 bound estimate. 56 00:04:27,170 --> 00:04:31,310 So this union bound estimate is quite important. 57 00:04:31,310 --> 00:04:34,420 Are there any questions on this? 58 00:04:34,420 --> 00:04:40,700 OK, so today we will start by introducing the notion of 59 00:04:40,700 --> 00:04:41,950 effective coding gain. 60 00:04:58,240 --> 00:05:03,330 And we'll use the symbol gamma sub f for that. 61 00:05:03,330 --> 00:05:05,790 What do we mean by effective coding gain? 62 00:05:05,790 --> 00:05:08,630 First you have to decide what region you're operating in. 63 00:05:08,630 --> 00:05:11,300 It's defined for both bandwidth-limited regime and 64 00:05:11,300 --> 00:05:12,800 power-limited regimes. 65 00:05:12,800 --> 00:05:15,240 So let us start with the power-limited regime. 66 00:05:24,610 --> 00:05:27,600 Remember in the power-limited regime, the trade-off we care 67 00:05:27,600 --> 00:05:35,740 about is the probability of bit error versus Eb over N_0. 68 00:05:35,740 --> 00:05:37,110 The baseline scheme is 2-PAM. 69 00:05:45,100 --> 00:05:49,630 And for 2-PAM, we showed two lectures ago that the 70 00:05:49,630 --> 00:05:56,650 probability of bit error is given by Q of root 2 Eb N_0. 71 00:06:01,160 --> 00:06:04,310 OK, it should be -- 72 00:06:04,310 --> 00:06:10,870 so now the idea is to plot your probability of bit error 73 00:06:10,870 --> 00:06:12,120 as a function of Eb N_0. 74 00:06:16,150 --> 00:06:19,760 Eb N_0 is always plotted in dB on the x-axis. 75 00:06:19,760 --> 00:06:23,790 The probability of bit error we also plot on 76 00:06:23,790 --> 00:06:25,750 a logarithmic scale. 77 00:06:25,750 --> 00:06:27,590 So this is ten to the minus six. 78 00:06:27,590 --> 00:06:30,390 This is ten to the minus five, ten to the 79 00:06:30,390 --> 00:06:33,840 minus four and so on. 80 00:06:33,840 --> 00:06:37,420 The Shannon limit is at -- 81 00:06:37,420 --> 00:06:39,412 this is the Shannon limit -- 82 00:06:39,412 --> 00:06:45,830 is at minus 1.59 dB. 83 00:06:45,830 --> 00:06:46,580 OK? 84 00:06:46,580 --> 00:06:52,750 For the power-limited regime, the performance of 2-PAM, 85 00:06:52,750 --> 00:06:57,350 which is given by this expression, is here. 86 00:06:57,350 --> 00:07:01,260 And we basically go to say that when the probability of 87 00:07:01,260 --> 00:07:05,600 bit error is ten to the minus five, the EbN _0 that you 88 00:07:05,600 --> 00:07:11,310 require is 9.6 dB. 89 00:07:11,310 --> 00:07:12,560 This is a nine. 90 00:07:15,510 --> 00:07:19,540 So what's the idea behind the effective coding gain? 91 00:07:19,540 --> 00:07:23,440 Well, suppose I give you a certain constellation, OK, a 92 00:07:23,440 --> 00:07:24,940 certain signal set. 93 00:07:24,940 --> 00:07:28,370 And you find the probability of bit error for that, 94 00:07:28,370 --> 00:07:29,070 and you plot it. 95 00:07:29,070 --> 00:07:31,970 And it comes out to be something like this. 96 00:07:31,970 --> 00:07:37,470 So this is the performance for some given signal set A. This 97 00:07:37,470 --> 00:07:38,720 is your 2-PAM. 98 00:07:43,150 --> 00:07:46,470 And now you want to quantify how much better is the signal 99 00:07:46,470 --> 00:07:49,390 set over the baseline system. 100 00:07:49,390 --> 00:07:51,510 Now, so in other words you want to look at the gap 101 00:07:51,510 --> 00:07:53,490 between the two curves. 102 00:07:53,490 --> 00:07:55,580 Now clearly the gap is going to be a function of the 103 00:07:55,580 --> 00:07:58,100 probability of bit error, right? 104 00:07:58,100 --> 00:08:01,990 But the basic idea now is you want to fix a certain 105 00:08:01,990 --> 00:08:04,530 probability of bit error because if you're designing a 106 00:08:04,530 --> 00:08:07,070 system, the probability of bit error is something that the 107 00:08:07,070 --> 00:08:08,920 system tolerates and is going to be fixed 108 00:08:08,920 --> 00:08:10,480 throughout the analysis. 109 00:08:10,480 --> 00:08:12,320 So in this course, we'll be fixing it at ten 110 00:08:12,320 --> 00:08:13,920 to the minus five. 111 00:08:13,920 --> 00:08:16,510 You fix this probability of bit error to ten to the minus 112 00:08:16,510 --> 00:08:18,610 five, and you look at how much a gap you have 113 00:08:18,610 --> 00:08:21,520 between the two curves. 114 00:08:21,520 --> 00:08:23,630 And this gap is going to be your effective 115 00:08:23,630 --> 00:08:26,520 coding gain in dB. 116 00:08:33,049 --> 00:08:37,010 So in other words, effective coding gain, I can write here, 117 00:08:37,010 --> 00:08:54,340 is the gap between a given signal set and concordant 118 00:08:54,340 --> 00:09:01,415 system, 2-PAM, at a fixed probability of bit error. 119 00:09:05,300 --> 00:09:06,060 OK? 120 00:09:06,060 --> 00:09:09,470 So now let us try to quantify this effective coding gain 121 00:09:09,470 --> 00:09:12,690 using the union bound estimate that we have here. 122 00:09:12,690 --> 00:09:18,120 So if I have a certain signal set, A, then I know from the 123 00:09:18,120 --> 00:09:23,370 union bound estimate that the probability of error is given 124 00:09:23,370 --> 00:09:33,060 by K_min of A times approximately Q of 125 00:09:33,060 --> 00:09:37,160 d_min over 2 sigma. 126 00:09:37,160 --> 00:09:39,200 So since I want that expression in terms of 127 00:09:39,200 --> 00:09:43,360 probability of bit error, I will use the fact that 128 00:09:43,360 --> 00:09:46,620 probability of bit error is probability of error per 129 00:09:46,620 --> 00:09:51,130 symbols or the number of bits per symobl. 130 00:09:51,130 --> 00:09:55,730 Since I have endpoints in my signal set, I have logM bits 131 00:09:55,730 --> 00:09:56,980 per symbol. 132 00:09:59,450 --> 00:10:15,980 And this is then K_min of A over logM base 2 times Q of 133 00:10:15,980 --> 00:10:17,780 d_min over 2 sigma. 134 00:10:33,910 --> 00:10:37,100 So I'm going to rewrite my probability of bit error 135 00:10:37,100 --> 00:10:40,620 expression now in this -- 136 00:10:40,620 --> 00:10:44,080 in fact the right side in this quad. 137 00:10:44,080 --> 00:10:56,390 K_b of A times Q of root 2 gamma_c of A 138 00:10:56,390 --> 00:10:58,870 times Eb over N_0. 139 00:10:58,870 --> 00:11:00,520 So I'm just going to define the right hand 140 00:11:00,520 --> 00:11:01,880 side in this way. 141 00:11:01,880 --> 00:11:05,290 And I'm going to relate gamma_c of A and K_b of A to 142 00:11:05,290 --> 00:11:08,620 the parameters I have on the right hand side there. 143 00:11:08,620 --> 00:11:10,690 And why do I want to do it this way? 144 00:11:10,690 --> 00:11:13,620 Because I want to get an expression as close as 145 00:11:13,620 --> 00:11:17,330 possible to the performance of an uncoded 2-PAM system 146 00:11:17,330 --> 00:11:20,900 because that way things will be easier to do if I want to 147 00:11:20,900 --> 00:11:24,530 calculate the effective coding gain, OK? 148 00:11:24,530 --> 00:11:27,240 So how do the parameters relate? 149 00:11:27,240 --> 00:11:40,810 Well, K_b of A is K_min of A over logM to the base 2. 150 00:11:40,810 --> 00:11:56,430 And this is the average number of nearest neighbors per 151 00:11:56,430 --> 00:12:04,110 information bit. 152 00:12:04,110 --> 00:12:08,060 If I compare the arguments inside the Q function, what I 153 00:12:08,060 --> 00:12:20,570 have is 2 gamma_c of A times Eb N_0 is the argument d_min 154 00:12:20,570 --> 00:12:25,350 over 2 sigma squared, which I will write as d_min squared 155 00:12:25,350 --> 00:12:28,960 over 4 sigma squared. 156 00:12:28,960 --> 00:12:32,950 So remember sigma squared is N_0 over 2. 157 00:12:32,950 --> 00:12:36,397 So if I simplify this expression, what I will end up 158 00:12:36,397 --> 00:12:45,525 getting is gamma_c of A is d_min squared over 4 Eb. 159 00:12:48,860 --> 00:12:51,500 Gamma_c of A is known as the nominal coding gain. 160 00:13:03,680 --> 00:13:06,060 It's not the same as the effective coding gain, which 161 00:13:06,060 --> 00:13:08,880 is the distance between the two curves. 162 00:13:08,880 --> 00:13:13,730 OK, are there any questions so far? 163 00:13:13,730 --> 00:13:18,440 So far what I have done is given a constellation A, I can 164 00:13:18,440 --> 00:13:20,870 find two parameters. 165 00:13:20,870 --> 00:13:25,490 I can find the nominal coding gain, and I can find the 166 00:13:25,490 --> 00:13:30,000 average number of nearest neighbors per information bit. 167 00:13:30,000 --> 00:13:32,630 And using these two parameters, we will try to get 168 00:13:32,630 --> 00:13:35,360 a handle over the effective coding gain. 169 00:13:35,360 --> 00:13:36,990 So that's the idea. 170 00:13:36,990 --> 00:13:38,490 So how can we plot -- 171 00:13:38,490 --> 00:13:41,420 so given this constellation A, we want to plot the 172 00:13:41,420 --> 00:13:44,050 probability of bit error curve like this. 173 00:13:44,050 --> 00:13:46,430 But instead of plotting the exact curve we will be 174 00:13:46,430 --> 00:13:49,600 plotting this curve here, which is the union bound 175 00:13:49,600 --> 00:13:52,270 estimate of your probability of bit error. 176 00:13:52,270 --> 00:13:57,290 So we will always be plotting the union bound estimate. 177 00:13:57,290 --> 00:13:59,200 So how do we do that? 178 00:13:59,200 --> 00:14:02,270 So we'll again start with a curve like that. 179 00:14:02,270 --> 00:14:07,930 Probability of bit error as a function of Eb N_0. 180 00:14:07,930 --> 00:14:11,020 This is in dB. 181 00:14:11,020 --> 00:14:13,980 The y-axis is also in dB. 182 00:14:13,980 --> 00:14:15,870 Sorry, not in dB but on the log scale. 183 00:14:22,995 --> 00:14:24,760 And so on. 184 00:14:24,760 --> 00:14:28,870 This is the Shannon limit at minus 1. 185 00:14:28,870 --> 00:14:30,120 59 dB. 186 00:14:34,270 --> 00:14:36,000 This is the 2-PAM performance. 187 00:14:39,420 --> 00:14:43,770 So if we want to plot this union bound estimate, you will 188 00:14:43,770 --> 00:14:45,250 be doing two steps. 189 00:14:45,250 --> 00:14:48,170 First, we'll be plotting this Q function, a curve 190 00:14:48,170 --> 00:14:49,780 corresponding to the Q function. 191 00:14:49,780 --> 00:14:52,110 And then we'll be scaling it. 192 00:14:52,110 --> 00:14:56,400 So if I just want to plot this Q function, how will it 193 00:14:56,400 --> 00:14:58,070 compare to this curve here? 194 00:15:05,620 --> 00:15:08,660 Well, you're just scaling the argument inside the Q 195 00:15:08,660 --> 00:15:10,130 function, right? 196 00:15:10,130 --> 00:15:13,280 But now because we are plotting the x-axis on a dB 197 00:15:13,280 --> 00:15:17,770 scale, it simply means that we are translating to the left by 198 00:15:17,770 --> 00:15:20,670 an amount of gamma_c in dB. 199 00:15:20,670 --> 00:15:22,920 Is that clear? 200 00:15:22,920 --> 00:15:23,850 Well, let's see. 201 00:15:23,850 --> 00:15:28,500 What we are doing is we are going to scale the x-axis by a 202 00:15:28,500 --> 00:15:29,790 certain constant. 203 00:15:29,790 --> 00:15:33,210 So this means that we will have to scale x-axis here. 204 00:15:33,210 --> 00:15:37,480 But, our Eb N_0 is being plotted on a dB scale, right? 205 00:15:37,480 --> 00:15:41,610 So multiplying in the linear scale corresponds to an 206 00:15:41,610 --> 00:15:43,680 addition on the logarithmic scale. 207 00:15:43,680 --> 00:15:47,240 An addition on the logarithmic scale simply implies a shift 208 00:15:47,240 --> 00:15:52,170 on the x-axis so that's the constant shift on the x-axis. 209 00:15:52,170 --> 00:15:53,800 I'm going to plot it here. 210 00:15:53,800 --> 00:15:58,020 This curve is going to be parallel to the original 2-PAM 211 00:15:58,020 --> 00:16:01,310 curve to the best of my abilities. 212 00:16:01,310 --> 00:16:05,010 OK, so this is what we get as your Q function. 213 00:16:05,010 --> 00:16:08,800 Now we are going to scale the y-axis by this factor here, 214 00:16:08,800 --> 00:16:14,300 K_b of A. But remember the y-axis is also on a 215 00:16:14,300 --> 00:16:16,970 logarithmic scale, rather than the linear scale. 216 00:16:16,970 --> 00:16:21,010 So multiply [INAUDIBLE] by a factor of K_b of A implies a 217 00:16:21,010 --> 00:16:24,590 vertical shift by a certain factor. 218 00:16:24,590 --> 00:16:27,760 So I'm going to do a vertical shift here. 219 00:16:34,600 --> 00:16:38,510 So let me make that darker now. 220 00:16:38,510 --> 00:16:40,060 So this is what I end up getting. 221 00:16:43,700 --> 00:16:46,250 OK, now note that the distance between these two curves is 222 00:16:46,250 --> 00:16:47,960 not fixed anymore. 223 00:16:47,960 --> 00:16:51,460 If the slope is steeper, then this distance is small, 224 00:16:51,460 --> 00:16:53,790 meaning that this distance here is large. 225 00:16:53,790 --> 00:16:57,830 On the other hand, if the slope is going to be smaller 226 00:16:57,830 --> 00:17:01,330 here, then this distance is large. 227 00:17:01,330 --> 00:17:03,770 So this means that a distance between these two curves is no 228 00:17:03,770 --> 00:17:05,099 longer fixed. 229 00:17:05,099 --> 00:17:07,859 And basically what we're saying here is that if the 230 00:17:07,859 --> 00:17:11,109 probability of error is large, then the number of nearest 231 00:17:11,109 --> 00:17:13,069 neighbors matters much more. 232 00:17:13,069 --> 00:17:15,280 But if the probability of error is going to be quite 233 00:17:15,280 --> 00:17:19,400 small, then the more important factor is your exponent, how 234 00:17:19,400 --> 00:17:21,750 fast the slope decays as opposed to the number of 235 00:17:21,750 --> 00:17:26,349 nearest neighbors, OK? 236 00:17:26,349 --> 00:17:31,630 This distance here is your effective coding gain. 237 00:17:31,630 --> 00:17:37,420 And this constant horizontal distance here is your nominal 238 00:17:37,420 --> 00:17:39,364 coding gain in dB. 239 00:17:45,570 --> 00:17:49,280 Are there any questions on this curve? 240 00:17:49,280 --> 00:17:51,842 AUDIENCE: [INAUDIBLE] 241 00:17:51,842 --> 00:17:57,800 on the right of four 2-PAM because these are the logM. 242 00:17:57,800 --> 00:18:01,340 PROFESSOR: So so what I'm really assuming is that 243 00:18:01,340 --> 00:18:03,350 gamma_c of A is greater than 1. 244 00:18:03,350 --> 00:18:05,250 So if I'm adding it, then I'm going to shift 245 00:18:05,250 --> 00:18:06,500 to the left, right? 246 00:18:19,600 --> 00:18:21,035 OK, so a couple of remarks. 247 00:18:31,980 --> 00:18:34,970 I'm just summarizing a few remarks I made while 248 00:18:34,970 --> 00:18:36,430 plotting the curve. 249 00:18:36,430 --> 00:18:53,430 First is the log-log scale is very convenient because any 250 00:18:53,430 --> 00:18:56,280 multiplicative factor simply involves a translation along 251 00:18:56,280 --> 00:19:00,040 the x and y directions. 252 00:19:00,040 --> 00:19:07,510 The second point is that the accuracy we have in the 253 00:19:07,510 --> 00:19:21,330 effective coding gain in this way is determined by the union 254 00:19:21,330 --> 00:19:26,020 bound estimate because that's all we are plotting. 255 00:19:26,020 --> 00:19:28,160 We are not really plotting the exact curve. 256 00:19:32,210 --> 00:19:35,380 And the third point is that we have a rule of thumb. 257 00:19:41,770 --> 00:19:45,190 Because the distance between the curves is not fixed, it's 258 00:19:45,190 --> 00:19:47,720 still too tedious to find the exact numerical value of 259 00:19:47,720 --> 00:19:49,610 effective coding gain. 260 00:19:49,610 --> 00:19:52,750 So what we can find is the value of the nominal coding 261 00:19:52,750 --> 00:19:57,390 gain and K_b of A exactly, given a constellation. 262 00:19:57,390 --> 00:20:00,320 So we want to have a rule of thumb for the 263 00:20:00,320 --> 00:20:02,200 effective coding gain. 264 00:20:02,200 --> 00:20:06,280 And the rule of thumb is that the effective coding gain is 265 00:20:06,280 --> 00:20:16,050 given by the nominal coding gain in dB minus 0.2 log2 of 266 00:20:16,050 --> 00:20:20,620 K_b of A. This is in dB. 267 00:20:20,620 --> 00:20:25,450 What this means is every factor of two increase is K_b 268 00:20:25,450 --> 00:20:29,590 results in a loss of 0.2 dB in the effective coding gain. 269 00:20:29,590 --> 00:20:35,520 OK, and this is our probability of bit error of 270 00:20:35,520 --> 00:20:37,600 ten to the minus five. 271 00:20:37,600 --> 00:20:42,060 So that's the region that we will be doing our analysis in. 272 00:20:42,060 --> 00:20:44,610 So this rule of thumb is quite helpful. 273 00:20:44,610 --> 00:20:46,594 Are there any questions? 274 00:20:46,594 --> 00:20:49,340 AUDIENCE: [INAUDIBLE] 275 00:20:49,340 --> 00:20:50,530 PROFESSOR: This rule of thumb? 276 00:20:50,530 --> 00:20:53,360 Well, it works quite well in practice so you 277 00:20:53,360 --> 00:20:54,860 want to use that rule. 278 00:20:54,860 --> 00:20:57,360 AUDIENCE: [INAUDIBLE] 279 00:20:57,360 --> 00:20:58,610 PROFESSOR: Right. 280 00:21:06,360 --> 00:21:09,150 If things are clear, let us do some examples. 281 00:21:17,090 --> 00:21:19,515 So let us start with a 2-PAM for examples. 282 00:21:24,810 --> 00:21:26,480 The first is 2-PAM system. 283 00:21:29,760 --> 00:21:32,950 What's the effective coding gain going to be for this 284 00:21:32,950 --> 00:21:35,790 constellation? 285 00:21:35,790 --> 00:21:38,250 It's going to be 0 dB, right? 286 00:21:38,250 --> 00:21:39,500 It just follows from the definition. 287 00:21:42,718 --> 00:21:46,790 I mean the effective coding gain tells us how much gap we 288 00:21:46,790 --> 00:21:50,000 have between this new constellation and 2-PAM. 289 00:21:50,000 --> 00:21:52,990 If your new constellation is 2-PAM itself, we don't have 290 00:21:52,990 --> 00:21:56,100 any effective coding gain. 291 00:21:56,100 --> 00:21:57,350 What about the 4-QAM? 292 00:22:01,120 --> 00:22:04,230 It's going to be still 0, because a 4-QAM is a Cartesian 293 00:22:04,230 --> 00:22:06,820 product of two 2-PAM constellations. 294 00:22:06,820 --> 00:22:09,290 And we argued last time that there is no coding gain if you 295 00:22:09,290 --> 00:22:11,970 use a Cartesian product. 296 00:22:11,970 --> 00:22:16,290 If you want to verify it using the framework that we have 297 00:22:16,290 --> 00:22:21,420 just developed, say we have four points. 298 00:22:21,420 --> 00:22:24,070 This point is alpha, alpha. 299 00:22:24,070 --> 00:22:30,580 This point is minus alpha, alpha, minus alpha, minus 300 00:22:30,580 --> 00:22:33,600 alpha, and alpha minus alpha. 301 00:22:33,600 --> 00:22:39,010 The nominal coding gain is given by d_min 302 00:22:39,010 --> 00:22:40,580 squared over 4 Eb. 303 00:22:45,080 --> 00:22:49,820 The minimum distance is 3 alpha between any two points. 304 00:22:49,820 --> 00:22:52,250 So we have 4 alpha squared. 305 00:22:52,250 --> 00:22:54,040 Then what's the energy per bit? 306 00:22:54,040 --> 00:22:57,210 Well, the energy per symbol is 2 alpha squared. 307 00:22:57,210 --> 00:23:00,240 Then we have two bits per symbol, so the energy per bit 308 00:23:00,240 --> 00:23:01,520 is alpha squared. 309 00:23:01,520 --> 00:23:05,080 So we have 4 alpha squared, and so that's 1. 310 00:23:05,080 --> 00:23:08,060 So we do not have any nominal coding gain. 311 00:23:08,060 --> 00:23:10,600 K_b of A, what's that going to be? 312 00:23:10,600 --> 00:23:13,870 Well, we have two nearest neighbors per symbol, but we 313 00:23:13,870 --> 00:23:15,830 also have two bits per symbol. 314 00:23:15,830 --> 00:23:17,910 So the number of nearest neighbors per bit 315 00:23:17,910 --> 00:23:18,940 is going to be one. 316 00:23:18,940 --> 00:23:23,960 So your nominal coding gain is 1, K_b of A is going to be 1. 317 00:23:23,960 --> 00:23:28,220 And so we do not see any coding gain 318 00:23:28,220 --> 00:23:30,880 for the 4-QAM system. 319 00:23:30,880 --> 00:23:33,660 Now let me do a slightly different case. 320 00:23:33,660 --> 00:23:37,770 Let me start with a 4-QAM, but I remove two points. 321 00:23:37,770 --> 00:23:40,860 I remove this point, and I remove this point. 322 00:23:40,860 --> 00:23:43,610 And I only keep the two points here. 323 00:23:43,610 --> 00:23:48,710 So I have a constellation, say, A prime, which only has 324 00:23:48,710 --> 00:23:52,490 two points, one point here and one point here. 325 00:23:52,490 --> 00:23:55,750 So this point is alpha, alpha, and this point is minus alpha, 326 00:23:55,750 --> 00:23:58,180 minus alpha. 327 00:23:58,180 --> 00:24:00,480 Any idea what the effective coding gain 328 00:24:00,480 --> 00:24:01,730 will be for this case? 329 00:24:06,000 --> 00:24:07,920 AUDIENCE: I think it's the same. 330 00:24:07,920 --> 00:24:08,900 PROFESSOR: It's still zero, right? 331 00:24:08,900 --> 00:24:12,130 AUDIENCE: Yeah, because it's saying that's 2-PAM. 332 00:24:12,130 --> 00:24:12,820 PROFESSOR: Exactly. 333 00:24:12,820 --> 00:24:17,600 All this is a 2-PAM constellation embedded in two 334 00:24:17,600 --> 00:24:18,910 dimensions. 335 00:24:18,910 --> 00:24:22,130 It's 2-PAM along this line, right? 336 00:24:22,130 --> 00:24:24,780 So I cannot hope to have a higher coding gain for this 337 00:24:24,780 --> 00:24:27,620 constellation over the original one. 338 00:24:27,620 --> 00:24:30,340 I mean I told you the story that we can take a subset of 339 00:24:30,340 --> 00:24:33,860 points, and we can increase the minimum distance by 340 00:24:33,860 --> 00:24:35,360 throwing away some points. 341 00:24:35,360 --> 00:24:38,700 But one has to be careful in that analysis. 342 00:24:38,700 --> 00:24:41,280 There could be examples where you're strictly improving the 343 00:24:41,280 --> 00:24:42,760 minimum distance. 344 00:24:42,760 --> 00:24:49,460 Note that the minimum distance here is going to 8 alpha 345 00:24:49,460 --> 00:24:52,230 squared now. 346 00:24:52,230 --> 00:24:54,970 OK, so the minimum distance is strictly improved. 347 00:24:54,970 --> 00:24:58,530 But what happens to be your energy per bit? 348 00:24:58,530 --> 00:25:01,780 Well, the energy per symbol is 2 alpha squared. 349 00:25:01,780 --> 00:25:05,650 But you only have one bit per symbol, so the energy per bit 350 00:25:05,650 --> 00:25:07,760 is 2 alpha squared. 351 00:25:07,760 --> 00:25:13,710 So your nominal coding gain is d_min squared over 4 Eb, which 352 00:25:13,710 --> 00:25:17,130 follows from the relation we have here. 353 00:25:17,130 --> 00:25:19,310 And d_min squared is 8 alpha squared. 354 00:25:19,310 --> 00:25:23,040 4 Eb is also 8 alpha squared, so the nominal coding gain is 355 00:25:23,040 --> 00:25:24,410 1, or 0 dB. 356 00:25:27,890 --> 00:25:30,810 The average number of nearest neighbors, well, we only have 357 00:25:30,810 --> 00:25:34,120 one nearest neighbor, and we have one bit per symbol, so 358 00:25:34,120 --> 00:25:37,950 that's -- and this is A prime by the way -- 359 00:25:37,950 --> 00:25:39,270 is going to be 1. 360 00:25:39,270 --> 00:25:42,627 And so the effective coding gain is still 0 dB. 361 00:25:46,920 --> 00:25:48,170 AUDIENCE: [INAUDIBLE] 362 00:25:51,220 --> 00:25:54,170 PROFESSOR: So probability of symbol error as a function of 363 00:25:54,170 --> 00:25:56,250 alpha has definitely decreased, right. 364 00:25:56,250 --> 00:25:58,530 But the trade-off we care about is probability of bit 365 00:25:58,530 --> 00:26:00,740 error as a function of Eb N_0. 366 00:26:00,740 --> 00:26:02,620 In other words, this curve is still here. 367 00:26:02,620 --> 00:26:04,370 It's not changed. 368 00:26:04,370 --> 00:26:07,450 I just shifted right by reducing my spectral 369 00:26:07,450 --> 00:26:07,622 efficiency. 370 00:26:07,622 --> 00:26:08,872 OK? 371 00:26:15,560 --> 00:26:19,810 So let's do one last example just to prove that it's not 372 00:26:19,810 --> 00:26:23,290 always the case that you get nothing by removing points. 373 00:27:35,990 --> 00:27:42,770 OK, so the contellation that I have in mind is the one I 374 00:27:42,770 --> 00:27:45,420 introduced last time. 375 00:27:45,420 --> 00:27:50,170 We'll start with a 3, 4 Cartesian product of 2-PAM. 376 00:27:50,170 --> 00:27:54,980 OK, so in three dimensions it's a cube, and the 3,4 377 00:27:54,980 --> 00:27:57,840 Cartesian product will have a point on each 378 00:27:57,840 --> 00:27:59,260 vertex of the cube. 379 00:27:59,260 --> 00:28:01,150 And we only keep four points. 380 00:28:01,150 --> 00:28:03,377 So the points we keep are these four points. 381 00:28:03,377 --> 00:28:04,627 OK? 382 00:28:09,800 --> 00:28:19,300 So I write B, the coordinates are alpha, alpha, alpha, minus 383 00:28:19,300 --> 00:28:24,540 alpha, minus alpha, alpha, minus alpha, alpha, minus 384 00:28:24,540 --> 00:28:31,770 alpha, and alpha, minus alpha, minus alpha. 385 00:28:31,770 --> 00:28:33,650 So these are the vertices. 386 00:28:33,650 --> 00:28:37,575 These are the coordinates of the four points here where we 387 00:28:37,575 --> 00:28:40,190 have the standard x, y and z-axis, which I'm not drawing 388 00:28:40,190 --> 00:28:42,360 because it will just look confusing. 389 00:28:42,360 --> 00:28:46,320 So now we'll see that for this particular signal set, we do 390 00:28:46,320 --> 00:28:49,170 have a coding gain, OK? 391 00:28:49,170 --> 00:28:50,595 Let's find the minimum distance. 392 00:28:54,160 --> 00:28:56,740 The minimum distance here is what? 393 00:28:56,740 --> 00:29:00,010 It's the distance along a diagonal. 394 00:29:00,010 --> 00:29:04,390 And the diagonal is of length to -- each side of the cube is 395 00:29:04,390 --> 00:29:05,640 off-length to alpha. 396 00:29:08,790 --> 00:29:13,930 So the diagonal is of length to alpha. 397 00:29:13,930 --> 00:29:16,275 So d_min squared is 8 alpha squared. 398 00:29:19,450 --> 00:29:21,950 What is the energy per bit going to be in this case? 399 00:29:33,030 --> 00:29:35,226 Well, what's the energy per symbol now? 400 00:29:35,226 --> 00:29:37,520 AUDIENCE: [INAUDIBLE] 401 00:29:37,520 --> 00:29:40,140 PROFESSOR: 3 alpha squared over 2 is the Eb, right? 402 00:29:40,140 --> 00:29:43,890 We have 3 alpha squared units of energy per symbol. 403 00:29:43,890 --> 00:29:48,230 We have four points, so we have two bits per symbol. 404 00:29:48,230 --> 00:29:54,900 So the energy per bit is 3 alpha squared over 2. 405 00:29:54,900 --> 00:30:01,470 So now if I look at my nominal coding gain, that's going to 406 00:30:01,470 --> 00:30:07,210 be d_min squared over 4 Eb. 407 00:30:07,210 --> 00:30:10,640 d_min squared is 8 alpha squared. 408 00:30:10,640 --> 00:30:14,170 4 dB is 6 alpha squared. 409 00:30:14,170 --> 00:30:17,080 So that's 4/3. 410 00:30:17,080 --> 00:30:21,780 On a linear scale and on a dB scale, this is going to be? 411 00:30:32,680 --> 00:30:34,666 You guys should remember your dB tables. 412 00:30:37,534 --> 00:30:39,446 AUDIENCE: 0.2? 413 00:30:39,446 --> 00:30:40,410 PROFESSOR: 12. 414 00:30:40,410 --> 00:30:42,510 You have to multiply by ten. 415 00:30:42,510 --> 00:30:46,150 So actually I think it's 1.3, more like 1.3 dB if you look 416 00:30:46,150 --> 00:30:47,880 at the last significant digit. 417 00:30:50,730 --> 00:30:56,760 OK, so what's K_b of A going to be in this case? 418 00:30:56,760 --> 00:30:59,055 Or K_b of B I should say. 419 00:30:59,055 --> 00:31:01,650 I'm calling the signal set B here. 420 00:31:01,650 --> 00:31:06,140 So what is K_b if B going to be? 421 00:31:13,820 --> 00:31:15,260 AUDIENCE: [INAUDIBLE] 422 00:31:15,260 --> 00:31:17,180 PROFESSOR: 3 by 2, right? 423 00:31:17,180 --> 00:31:19,760 You have three nearest neighbors per point, for each 424 00:31:19,760 --> 00:31:21,150 point has three nearest neighbors. 425 00:31:21,150 --> 00:31:22,840 They're all equidistant. 426 00:31:22,840 --> 00:31:23,906 And you have two bits per symbol. 427 00:31:23,906 --> 00:31:25,156 OK? 428 00:31:26,680 --> 00:31:29,650 So if you work out your effective coding again using 429 00:31:29,650 --> 00:31:33,745 this rule of thumb here, take gamma_c of A in dB and 430 00:31:33,745 --> 00:31:37,460 subtract 0.2 log2 you will see that the effective coding gain 431 00:31:37,460 --> 00:31:39,220 is approximately 1 dB. 432 00:31:55,730 --> 00:31:56,980 Are there any questions? 433 00:32:00,970 --> 00:32:02,800 OK, so it might seem that this is still not a 434 00:32:02,800 --> 00:32:03,760 very impressive number. 435 00:32:03,760 --> 00:32:05,100 Yes, you had a question. 436 00:32:05,100 --> 00:32:07,810 AUDIENCE: What was it about this constellation that gave 437 00:32:07,810 --> 00:32:09,082 us coding gain? 438 00:32:09,082 --> 00:32:12,390 Was is the face that we went to higher dimensions? 439 00:32:12,390 --> 00:32:15,240 PROFESSOR: Right, so usually you do get a coding gain if 440 00:32:15,240 --> 00:32:20,400 you trade-off minimum distance with spectral efficiency. 441 00:32:20,400 --> 00:32:22,820 So that's usually the case. 442 00:32:22,820 --> 00:32:26,200 So that's why I presented you an example of that case. 443 00:32:26,200 --> 00:32:28,180 It's just that this was a very special case where you end up 444 00:32:28,180 --> 00:32:30,766 with a 2-PAM constellation to start with. 445 00:32:36,010 --> 00:32:42,440 OK, so the comment I was going to make is that 1.2 dB still 446 00:32:42,440 --> 00:32:44,510 might not be like a very impressive number. 447 00:32:44,510 --> 00:32:47,230 I mean after all the hard work, you just get 1 dB 448 00:32:47,230 --> 00:32:48,660 effective coding gain. 449 00:32:48,660 --> 00:32:51,290 So the question to ask at this point is are there any 450 00:32:51,290 --> 00:32:54,980 constellations that have much higher coding gains? 451 00:32:54,980 --> 00:32:58,440 In particular, other constellations that come close 452 00:32:58,440 --> 00:33:01,340 to the Shannon limit at all. 453 00:33:01,340 --> 00:33:04,280 And we'll look at one interesting class of signal 454 00:33:04,280 --> 00:33:06,640 sets, which are known as orthogonal signal sets. 455 00:33:18,540 --> 00:33:22,140 And you probably saw these examples back in 6.450, but 456 00:33:22,140 --> 00:33:24,070 we're just reviewing it again here. 457 00:33:24,070 --> 00:33:27,620 And these signal sets have a property that as you increase 458 00:33:27,620 --> 00:33:30,960 the number of dimensions, you come arbitrarily close to the 459 00:33:30,960 --> 00:33:32,770 Shannon limit. 460 00:33:32,770 --> 00:33:37,110 So the definition of the signal sets is your A -- 461 00:33:37,110 --> 00:33:43,450 it's a collection of signals aj, where j goes from 1 to M 462 00:33:43,450 --> 00:33:52,450 such that the inner product between ai and aj is E A times 463 00:33:52,450 --> 00:33:56,030 delta_i,j, meaning that if i is not equal to j, the two 464 00:33:56,030 --> 00:33:56,920 signals are orthogonal. 465 00:33:56,920 --> 00:34:01,560 And if i equals j, the energy is E(A). 466 00:34:01,560 --> 00:34:05,420 Geometrically, we have two points. 467 00:34:05,420 --> 00:34:07,220 The two points can be thought of as two 468 00:34:07,220 --> 00:34:09,870 points on the two axes. 469 00:34:09,870 --> 00:34:12,650 So this is a case when M equals 2. 470 00:34:12,650 --> 00:34:16,139 When M equals 3, the three points will lie on those three 471 00:34:16,139 --> 00:34:17,389 corresponding axes. 472 00:34:19,650 --> 00:34:22,870 So this is the case when M equals 3 and so on. 473 00:34:22,870 --> 00:34:25,120 So generally when you have M points, your signal 474 00:34:25,120 --> 00:34:28,020 spatializes in M dimensions, and each point can be thought 475 00:34:28,020 --> 00:34:31,860 of as a point on one of the axes. 476 00:34:31,860 --> 00:34:36,760 So if you look at the distance between any two points, it's 477 00:34:36,760 --> 00:34:41,850 the norm of ai minus aj squared. 478 00:34:41,850 --> 00:34:46,210 I could simplify this simply as the norm of ai squared plus 479 00:34:46,210 --> 00:34:50,050 the norm of aj squared because the end product 480 00:34:50,050 --> 00:34:52,630 is going to be zero. 481 00:34:52,630 --> 00:34:55,760 OK, now ai squared is going to be E(A). 482 00:34:55,760 --> 00:34:57,810 aj squared is going to be E(A). 483 00:34:57,810 --> 00:35:01,280 So this is 2E(A). 484 00:35:01,280 --> 00:35:03,440 So what we're saying here is that every point is 485 00:35:03,440 --> 00:35:06,880 equidistant from every other point, and the square of the 486 00:35:06,880 --> 00:35:09,940 distance is 2 E(A). 487 00:35:09,940 --> 00:35:11,190 OK? 488 00:35:13,440 --> 00:35:18,230 So in other words, the average number of nearest neighbors is 489 00:35:18,230 --> 00:35:20,160 always going to be a M minus 1. 490 00:35:24,310 --> 00:35:27,700 So let us calculate the nominal coding gain for this 491 00:35:27,700 --> 00:35:28,950 constellation. 492 00:35:32,190 --> 00:35:38,460 That's going to be d_min squared over 4 Eb. 493 00:35:41,590 --> 00:35:49,160 d_min squared is 2E(A) over 4 Eb. 494 00:35:49,160 --> 00:35:53,090 Well, the energy per bit is the energy per symbol, which 495 00:35:53,090 --> 00:35:56,585 is E(A) over the number of bits per symbol, which is log 496 00:35:56,585 --> 00:35:59,710 M to the base 2. 497 00:35:59,710 --> 00:36:09,520 And this I'm going to write here is 1/2 log 498 00:36:09,520 --> 00:36:11,890 M to the base 2. 499 00:36:11,890 --> 00:36:15,622 So as I increase my M, my number of dimensions, my 500 00:36:15,622 --> 00:36:17,155 nominal coding gain goes to infinity. 501 00:36:17,155 --> 00:36:18,405 OK? 502 00:36:21,240 --> 00:36:23,310 What do we expect for the effective coding gain? 503 00:36:23,310 --> 00:36:24,560 Will it also go to infinity? 504 00:36:30,974 --> 00:36:32,408 AUDIENCE: It approaches the Shannon limit. 505 00:36:32,408 --> 00:36:34,300 PROFESSOR: Well, we don't know whether it approaches the 506 00:36:34,300 --> 00:36:36,115 Shannon limit as of yet, but we know it 507 00:36:36,115 --> 00:36:37,250 cannot go to infinity. 508 00:36:37,250 --> 00:36:38,546 It's upper bounded by the Shannon limit. 509 00:36:38,546 --> 00:36:40,220 Right? 510 00:36:40,220 --> 00:36:44,090 So the effective coding gain cannot really become 511 00:36:44,090 --> 00:36:48,220 unbounded, and what's really happening here? 512 00:36:48,220 --> 00:36:49,470 AUDIENCE: [INAUDIBLE] 513 00:36:51,890 --> 00:36:54,280 PROFESSOR: Right, but why does the effective coding gain stay 514 00:36:54,280 --> 00:36:56,610 bounded and the nominal coding gain blow up? 515 00:36:56,610 --> 00:36:58,680 AUDIENCE: [INAUDIBLE] 516 00:36:58,680 --> 00:37:01,010 PROFESSOR: Right, the number of nearest neighbors also 517 00:37:01,010 --> 00:37:02,440 increases with M, right? 518 00:37:02,440 --> 00:37:04,359 Not just the nominal coding gain. 519 00:38:32,330 --> 00:38:34,570 So let's write that down. 520 00:38:34,570 --> 00:38:37,460 If I look at the number of nearest neighbors per 521 00:38:37,460 --> 00:38:40,690 information bit -- 522 00:38:40,690 --> 00:38:41,510 yes? 523 00:38:41,510 --> 00:38:44,480 AUDIENCE: These don't have 0 mean, right? 524 00:38:44,480 --> 00:38:45,790 PROFESSOR: Right, they don't have 0 mean. 525 00:38:45,790 --> 00:38:47,040 That's a good point. 526 00:38:51,420 --> 00:38:56,170 PROFESSOR: Let's see, what happens to the mean as M goes 527 00:38:56,170 --> 00:38:57,420 to infinity? 528 00:39:00,140 --> 00:39:02,470 If we look at what happens to the mean, it will be 1 over M. 529 00:39:02,470 --> 00:39:05,410 You find the mean of this guy, it's 1 over 2 times the 530 00:39:05,410 --> 00:39:09,300 distance here times distance here is 1 over 2 E(A), E(A). 531 00:39:09,300 --> 00:39:10,260 OK? 532 00:39:10,260 --> 00:39:12,340 If you look at three dimensions, it's 1 over 3 533 00:39:12,340 --> 00:39:14,050 E(A), E(A), E(A). 534 00:39:14,050 --> 00:39:16,560 So in M dimensions, it's going to be 1 over M times all these 535 00:39:16,560 --> 00:39:17,180 coordinates. 536 00:39:17,180 --> 00:39:19,930 If we look at the mean squared, it goes to 0 as M 537 00:39:19,930 --> 00:39:21,042 goes to infinity. 538 00:39:21,042 --> 00:39:24,390 So that's a good point. 539 00:39:24,390 --> 00:39:26,276 So the mean does go to 0, and so we 540 00:39:26,276 --> 00:39:27,910 approach the Shannon limit. 541 00:39:30,410 --> 00:39:38,260 So we have K_b of A over logM What's the 542 00:39:38,260 --> 00:39:39,610 number of nearest neighbors? 543 00:39:39,610 --> 00:39:47,990 We saw it was M minus 1 over log M. And that goes to 544 00:39:47,990 --> 00:39:51,990 infinity as M goes to infinity. 545 00:39:51,990 --> 00:39:55,980 OK, so the average number of nearest neighbors per 546 00:39:55,980 --> 00:40:00,590 information bit also go to infinity 547 00:40:00,590 --> 00:40:02,930 as M goes to infinity. 548 00:40:02,930 --> 00:40:07,060 And if we look at the mean -- just to write down the comment 549 00:40:07,060 --> 00:40:09,100 that was made. 550 00:40:09,100 --> 00:40:13,120 If we look at the mean of this constellation, that's going to 551 00:40:13,120 --> 00:40:21,960 be 1 over the number of points times root E(A) root 552 00:40:21,960 --> 00:40:26,590 E(A) and root E(A). 553 00:40:26,590 --> 00:40:28,170 So there are M coordinates. 554 00:40:28,170 --> 00:40:30,760 So that's going to be the mean if you believe this is the 555 00:40:30,760 --> 00:40:33,300 coordinate axis here. 556 00:40:33,300 --> 00:40:39,550 And if you we look at the norm of M(A) squared, then it's 557 00:40:39,550 --> 00:40:46,310 going to be E(A) over M. And as M goes to infinity, 558 00:40:46,310 --> 00:40:47,560 this goes to 0. 559 00:40:54,508 --> 00:40:57,590 And because the mean goes to 0, it's not unreasonable to 560 00:40:57,590 --> 00:41:00,825 expect that the performance does approach to the ultimate 561 00:41:00,825 --> 00:41:03,200 Shanon limit. 562 00:41:03,200 --> 00:41:09,060 OK, so so far we have looked at K_b of A and gamma_c of A. 563 00:41:09,060 --> 00:41:10,620 If you look at the effective coding 564 00:41:10,620 --> 00:41:13,640 gain, it is still bounded. 565 00:41:16,930 --> 00:41:18,360 It's always bounded. 566 00:41:18,360 --> 00:41:22,280 It is not clear at this point what really happens to it. 567 00:41:22,280 --> 00:41:25,750 And in fact, it's not quite straightforward to actually 568 00:41:25,750 --> 00:41:28,440 show what happens to the effective coding gain. 569 00:41:28,440 --> 00:41:32,260 If you have taken 6.450, then Professor Bob Gallager really 570 00:41:32,260 --> 00:41:35,470 goes into the details of proving how the effective 571 00:41:35,470 --> 00:41:38,490 coding gain for this signal set indeed does approach the 572 00:41:38,490 --> 00:41:40,560 Shannon limit. 573 00:41:40,560 --> 00:41:50,690 But it can be shown that the effective coding gain does 574 00:41:50,690 --> 00:41:56,790 approach 11.2 dB as M goes to infinity. 575 00:41:56,790 --> 00:41:59,560 The main trick here is that union bound is usually quite 576 00:41:59,560 --> 00:42:01,770 weak if the probability of error is small. 577 00:42:01,770 --> 00:42:04,450 So we replace the probability of error just by 1, instead of 578 00:42:04,450 --> 00:42:06,240 the union bound, at some point. 579 00:42:06,240 --> 00:42:10,300 And if you do that change, then things work out nicely. 580 00:42:10,300 --> 00:42:13,520 But in this course, we will just assert that the 581 00:42:13,520 --> 00:42:16,040 orthogonal signal sets do achieve the 582 00:42:16,040 --> 00:42:17,580 ultimate Shannon limit. 583 00:42:17,580 --> 00:42:22,180 And you can see 6.450 notes for the proof. 584 00:42:30,560 --> 00:42:33,570 So that's the assertion that we are making right now. 585 00:42:33,570 --> 00:42:36,910 So let's step back and take a look at what's 586 00:42:36,910 --> 00:42:38,740 happening so far. 587 00:42:38,740 --> 00:42:41,990 We have a class of signal sets which are conceptually quite 588 00:42:41,990 --> 00:42:43,290 easy to describe. 589 00:42:43,290 --> 00:42:45,430 They are just orthogonal signal sets. 590 00:42:45,430 --> 00:42:47,870 They are not too hard to analyze, and they 591 00:42:47,870 --> 00:42:51,460 asymptotically approach the ultimate Shannon limit. 592 00:42:51,460 --> 00:42:52,730 So this seems quite nice. 593 00:42:52,730 --> 00:42:56,010 So why not we just implement them? 594 00:42:56,010 --> 00:42:59,180 Now it turns that these are not very common in practice. 595 00:42:59,180 --> 00:43:01,850 They are not implemented as often as you might think when 596 00:43:01,850 --> 00:43:03,740 you first look at them and because 597 00:43:03,740 --> 00:43:05,840 they have some drawbacks. 598 00:43:05,840 --> 00:43:09,610 And there are a couple of drawbacks which make them very 599 00:43:09,610 --> 00:43:12,530 less appealing than what you might otherwise think. 600 00:43:15,450 --> 00:43:18,820 And first of all, what's the spectral efficiency for these 601 00:43:18,820 --> 00:43:20,070 signal sets? 602 00:43:23,316 --> 00:43:24,230 AUDIENCE:0, right? 603 00:43:24,230 --> 00:43:25,150 PROFESSOR: It goes to 0. 604 00:43:25,150 --> 00:43:27,050 And how does it go to 0? 605 00:43:27,050 --> 00:43:33,370 It's 2log M to the base 2 over M. As M goes to infinity, 606 00:43:33,370 --> 00:43:34,960 this goes to 0. 607 00:43:34,960 --> 00:43:39,150 And in fact, it goes to 0 quite sharply. 608 00:43:39,150 --> 00:43:41,740 Now that's fine, but if you want to approach Shannon 609 00:43:41,740 --> 00:43:45,300 limit, our spectral efficiency should indeed go to 0. 610 00:43:45,300 --> 00:43:48,520 But if you're looking at the system design problem, what do 611 00:43:48,520 --> 00:43:50,050 you really want? 612 00:43:50,050 --> 00:43:52,330 You have a certain effective coding gain that you have in 613 00:43:52,330 --> 00:43:55,030 mind, and suppose you are presented with a list of 614 00:43:55,030 --> 00:43:57,960 several different signal sets that achieve this effective 615 00:43:57,960 --> 00:43:59,290 coding gain. 616 00:43:59,290 --> 00:44:02,700 And your job is to pick one of these possible signal sets. 617 00:44:02,700 --> 00:44:04,320 Which one will you pick? 618 00:44:04,320 --> 00:44:06,430 Well, there are two things you will look for, right? 619 00:44:06,430 --> 00:44:08,220 You will look for the spectral efficiency that these 620 00:44:08,220 --> 00:44:12,020 different signal sets require or achieve in order to get 621 00:44:12,020 --> 00:44:13,800 this effective coding gain. 622 00:44:13,800 --> 00:44:16,650 And the higher the spectral efficiency the better. 623 00:44:16,650 --> 00:44:18,660 And secondly, you will also want to look at the 624 00:44:18,660 --> 00:44:20,600 implementation complexity. 625 00:44:20,600 --> 00:44:22,760 You do not want something which really takes a lot of 626 00:44:22,760 --> 00:44:25,110 time to decode. 627 00:44:25,110 --> 00:44:25,195 OK? 628 00:44:25,195 --> 00:44:27,770 So it turns out that because the spectral efficiency goes 629 00:44:27,770 --> 00:44:31,180 to 0 so sharply, it's not very surprising that there are 630 00:44:31,180 --> 00:44:34,380 other codes that have the same effective coding gain but 631 00:44:34,380 --> 00:44:36,480 higher spectral efficiency. 632 00:44:36,480 --> 00:44:55,460 So there are binary codes that have a higher spectral 633 00:44:55,460 --> 00:45:03,050 efficiency for same effective coding gain. 634 00:45:03,050 --> 00:45:05,860 And so they would be a natural choice over the orthogonal 635 00:45:05,860 --> 00:45:06,960 signal set. 636 00:45:06,960 --> 00:45:08,130 Yes? 637 00:45:08,130 --> 00:45:09,740 AUDIENCE: [INAUDIBLE] high-spectrum efficiency to 638 00:45:09,740 --> 00:45:11,350 mean bandwidth? 639 00:45:11,350 --> 00:45:12,990 PROFESSOR: Right, because in practice, bandwidth is never 640 00:45:12,990 --> 00:45:14,320 really infinite, right? 641 00:45:14,320 --> 00:45:16,850 So if you are achieving the same -- 642 00:45:16,850 --> 00:45:18,440 the first objective is to have a certain 643 00:45:18,440 --> 00:45:19,530 effective coding gain. 644 00:45:19,530 --> 00:45:21,680 If you do have that, you want to pick something which has a 645 00:45:21,680 --> 00:45:24,090 higher spectral efficiency. 646 00:45:24,090 --> 00:45:28,120 And indeed, the subject of chapters six and eight is to 647 00:45:28,120 --> 00:45:30,850 come up with binary codes that have a much better performance 648 00:45:30,850 --> 00:45:32,970 than the orthogonal signal set for a given 649 00:45:32,970 --> 00:45:34,430 effective coding gain. 650 00:45:34,430 --> 00:45:37,240 Chapter six deals with the the block codes, whereas chapter 651 00:45:37,240 --> 00:45:39,880 eight deals with convolutional codes. 652 00:45:39,880 --> 00:45:43,220 And chapter seven basically develops all the finite field 653 00:45:43,220 --> 00:45:46,410 theory that you require to study convolutional codes. 654 00:45:46,410 --> 00:45:49,430 So the point is, in the subsequent lectures we will be 655 00:45:49,430 --> 00:45:52,510 focusing a lot on improving the performance over 656 00:45:52,510 --> 00:45:54,940 orthogonal signal sets. 657 00:45:54,940 --> 00:45:57,775 The second point I mentioned was implementation complexity. 658 00:46:07,600 --> 00:46:10,670 Now one particular way of implementing orthogonal codes, 659 00:46:10,670 --> 00:46:15,100 which you saw in the first homework, was using this idea 660 00:46:15,100 --> 00:46:17,100 of Hadamard transforms, right? 661 00:46:17,100 --> 00:46:19,940 You have a Hardamard matrix, which is an M by M matrix. 662 00:46:19,940 --> 00:46:22,260 The rows of the Hadamard matrix are your orthogonal 663 00:46:22,260 --> 00:46:25,400 signal sets, which are elements aj. 664 00:46:25,400 --> 00:46:27,950 And at the receiver, what you would do is when you get the 665 00:46:27,950 --> 00:46:31,730 signal y, you simply multiply by the Hadamard matrix and 666 00:46:31,730 --> 00:46:34,050 look at the coordinate which is the largest. 667 00:46:34,050 --> 00:46:35,180 So this is something you did in the 668 00:46:35,180 --> 00:46:37,120 first homework exercise. 669 00:46:37,120 --> 00:46:40,680 And how many computations did you require for an orthogonal 670 00:46:40,680 --> 00:46:41,960 signal set of size M? 671 00:46:46,010 --> 00:46:47,260 AUDIENCE: [INAUDIBLE] 672 00:46:51,420 --> 00:46:51,506 PROFESSOR: Sorry? 673 00:46:51,506 --> 00:46:54,350 OK, so I meant M being the size of the 674 00:46:54,350 --> 00:46:55,792 orthogonal signal set. 675 00:46:55,792 --> 00:46:59,030 AUDIENCE: Yeah, [INAUDIBLE] 676 00:46:59,030 --> 00:47:01,778 PROFESSOR: Right. 677 00:47:01,778 --> 00:47:03,590 AUDIENCE: 2 to the M by 2 to the M. 678 00:47:03,590 --> 00:47:05,330 PROFESSOR: If it is 2 to the M by 2 to the M, it is M times 2 679 00:47:05,330 --> 00:47:09,470 to the M. But there are M orthogonal signal sets, then 680 00:47:09,470 --> 00:47:17,950 it will be M log M, where M is the number of signal sets. 681 00:47:17,950 --> 00:47:19,440 So that's the number of computations 682 00:47:19,440 --> 00:47:20,970 that you would require. 683 00:47:20,970 --> 00:47:26,640 Now, the question to ask is, is this fast or is this slow? 684 00:47:26,640 --> 00:47:28,670 Is this too many computations or is this too little 685 00:47:28,670 --> 00:47:30,090 computations? 686 00:47:30,090 --> 00:47:32,790 And if you're looking in the power-limited regime, what we 687 00:47:32,790 --> 00:47:36,270 really want is to see the complexity per information bit 688 00:47:36,270 --> 00:47:38,810 because we normalize things per information bit. 689 00:47:38,810 --> 00:47:41,730 Now, how many bits are sent when you have orthogonal 690 00:47:41,730 --> 00:47:43,510 signal set of size M? 691 00:47:43,510 --> 00:47:45,050 We have log M bits, right? 692 00:47:45,050 --> 00:47:47,290 So this quantity is actually exponential in the number of 693 00:47:47,290 --> 00:47:50,360 transmitted bits that we are sending, and so in fact this 694 00:47:50,360 --> 00:47:52,385 is quite slow. 695 00:47:52,385 --> 00:47:54,480 Or in other words we are saying it is we require M 696 00:47:54,480 --> 00:47:57,640 times 2 to the M computations, where M is the number of 697 00:47:57,640 --> 00:48:01,295 information bits that we are sending. 698 00:48:01,295 --> 00:48:04,220 AUDIENCE: So if M is the size of your [INAUDIBLE] 699 00:48:04,220 --> 00:48:06,127 then you have log M bits? 700 00:48:06,127 --> 00:48:07,558 PROFESSOR: Right. 701 00:48:07,558 --> 00:48:09,470 AUDIENCE: So M log M over log M is M. 702 00:48:09,470 --> 00:48:11,195 PROFESSOR: Why are you adding by M? 703 00:48:11,195 --> 00:48:13,690 AUDIENCE: You need M log M operations to decode a symbol. 704 00:48:13,690 --> 00:48:15,190 PROFESSOR: To decode a symbol. 705 00:48:15,190 --> 00:48:18,020 AUDIENCE: And each symbol gives you log M bits. 706 00:48:18,020 --> 00:48:21,650 So you have M operations per bit. 707 00:48:21,650 --> 00:48:24,080 PROFESSOR: Well, when you send, say, log M bits, right, 708 00:48:24,080 --> 00:48:26,290 you can only send one of the M possible symbols. 709 00:48:29,860 --> 00:48:32,100 And when you want to decode the symbol, you would end up 710 00:48:32,100 --> 00:48:33,350 recovering log M bits. 711 00:48:35,630 --> 00:48:38,928 So you should not be dividing by M here. 712 00:48:38,928 --> 00:48:42,030 AUDIENCE: You're dividing by log M because you want to do 713 00:48:42,030 --> 00:48:43,730 it per bit. 714 00:48:43,730 --> 00:48:44,710 PROFESSOR: All right. 715 00:48:44,710 --> 00:48:44,960 You're right. 716 00:48:44,960 --> 00:48:47,330 So you're dividing by log M. You have M log M bits. 717 00:48:47,330 --> 00:48:51,276 You're dividing by log M, so you still have -- 718 00:48:51,276 --> 00:48:55,240 AUDIENCE: You have M calculations per bit. 719 00:48:55,240 --> 00:48:56,490 [INAUDIBLE] 720 00:49:02,870 --> 00:49:06,040 PROFESSOR: So we are sending -- so I need this many 721 00:49:06,040 --> 00:49:09,990 computations to get log M bits. 722 00:49:12,590 --> 00:49:17,359 So is this expression exponential in log M or not? 723 00:49:20,670 --> 00:49:23,310 So that's all I was saying. 724 00:49:23,310 --> 00:49:25,350 AUDIENCE: I just put it another way that you could 725 00:49:25,350 --> 00:49:27,170 divide by log M. 726 00:49:27,170 --> 00:49:28,440 PROFESSOR: M for information, right. 727 00:49:32,900 --> 00:49:35,050 OK, are there any questions? 728 00:49:35,050 --> 00:49:37,510 I mean we are both agreeing to the fact that this is too many 729 00:49:37,510 --> 00:49:39,030 computations than you would want. 730 00:49:39,030 --> 00:49:43,730 And there exist other decoding algorithms for which the 731 00:49:43,730 --> 00:49:46,605 number of computations is much smaller than exponential. 732 00:49:54,290 --> 00:49:58,280 OK, so let's next look at the bandwidth-limited regime. 733 00:50:12,110 --> 00:50:15,500 Well, actually before that I wanted to mention a couple of 734 00:50:15,500 --> 00:50:18,370 other signal sets as well. 735 00:50:18,370 --> 00:50:20,780 So there are two other important class of signal sets 736 00:50:20,780 --> 00:50:24,050 which are related to the orthogonal signal sets. 737 00:50:24,050 --> 00:50:28,090 The first is this class of simplex signal sets. 738 00:50:31,580 --> 00:50:41,065 And the other class is bi-orthogonal signal sets. 739 00:50:46,960 --> 00:50:49,910 So the idea behind simplex signal sets goes back to the 740 00:50:49,910 --> 00:50:53,300 observation that was made that we have a non-zero mean for 741 00:50:53,300 --> 00:50:54,960 this signal set here. 742 00:50:54,960 --> 00:50:57,850 So if we do, we can indeed subtract of the mean and get a 743 00:50:57,850 --> 00:51:00,430 performance which is better than the 744 00:51:00,430 --> 00:51:02,070 orthogonal signal set. 745 00:51:02,070 --> 00:51:05,450 So in particular if I subtract of the mean here, I get a 746 00:51:05,450 --> 00:51:09,560 signal set, which is like this. 747 00:51:09,560 --> 00:51:14,090 This is the simplex signal set when M equals 2. 748 00:51:14,090 --> 00:51:18,730 When M equals 3, I'm going to subtract of the mean from this 749 00:51:18,730 --> 00:51:19,390 plane here. 750 00:51:19,390 --> 00:51:21,980 The points lie on this particular plane. 751 00:51:21,980 --> 00:51:25,570 And what I end up with is an equilateral triangle. 752 00:51:25,570 --> 00:51:29,580 So this is the case when M equals 3. 753 00:51:29,580 --> 00:51:32,960 What do you think M equals 4 would look like? 754 00:51:32,960 --> 00:51:36,020 A tetrahedron right here. 755 00:51:36,020 --> 00:51:39,930 That's a simplex signal set. 756 00:51:39,930 --> 00:51:45,410 So basically, it's also not too hard to write an algebraic 757 00:51:45,410 --> 00:51:47,820 expression for these signal sets. 758 00:51:47,820 --> 00:51:55,720 E of A prime is going to be E of A minus M of A. You 759 00:51:55,720 --> 00:52:00,590 subtract off the mean from your orthogonal signal set. 760 00:52:00,590 --> 00:52:03,460 If I want to write it in terms of inner products, the inner 761 00:52:03,460 --> 00:52:13,450 product between ai and aj is given by E(A) if i equals j. 762 00:52:13,450 --> 00:52:21,610 And it's I believe minus of 1 over M minus 1 times E(A) if i 763 00:52:21,610 --> 00:52:23,220 is not equal to j. 764 00:52:23,220 --> 00:52:26,340 So this is the inner product between ai and aj. 765 00:52:26,340 --> 00:52:28,850 And I mean there are also many properties of this simplex 766 00:52:28,850 --> 00:52:31,652 signal set, but I believe that that's going to be in your 767 00:52:31,652 --> 00:52:33,960 next homework exercise so I'm not going 768 00:52:33,960 --> 00:52:35,780 into too many details. 769 00:52:35,780 --> 00:52:38,360 What' the spectral efficiency in this case going to be? 770 00:52:45,346 --> 00:52:46,850 AUDIENCE: I have a question. 771 00:52:46,850 --> 00:52:49,040 So what is a simplex? 772 00:52:49,040 --> 00:52:52,270 Is a simplex just a reference to anything with [INAUDIBLE]. 773 00:52:52,270 --> 00:52:54,990 PROFESSOR: Simplex signal set is derived from the orthogonal 774 00:52:54,990 --> 00:52:57,560 signal set by subtracting off its mean. 775 00:52:57,560 --> 00:52:59,935 AUDIENCE: Right, but in general simplex means anything 776 00:52:59,935 --> 00:53:01,840 [INAUDIBLE]? 777 00:53:01,840 --> 00:53:03,170 PROFESSOR: I'm not sure about that. 778 00:53:08,940 --> 00:53:12,188 What's the spectral efficiency going to be? 779 00:53:12,188 --> 00:53:13,438 AUDIENCE: [INAUDIBLE] 780 00:53:15,840 --> 00:53:18,536 PROFESSOR: Well, almost. 781 00:53:18,536 --> 00:53:19,030 AUDIENCE: You use one dimension. 782 00:53:19,030 --> 00:53:21,285 PROFESSOR: Right, you just use one dimension. 783 00:53:27,230 --> 00:53:30,650 OK, and there are other properties that you'll be 784 00:53:30,650 --> 00:53:32,420 looking at it in the homework. 785 00:53:32,420 --> 00:53:34,692 Now the idea behind this -- 786 00:53:34,692 --> 00:53:35,942 AUDIENCE: [INAUDIBLE] 787 00:53:39,660 --> 00:53:44,390 PROFESSOR: This is 2 log M over [INAUDIBLE]. 788 00:53:44,390 --> 00:53:48,510 So the idea behind the bi-orthogonal signal set is 789 00:53:48,510 --> 00:53:51,870 you start with an orthogonal signal set, and for each 790 00:53:51,870 --> 00:53:54,240 point, you also put the negative signal of it. 791 00:53:54,240 --> 00:53:56,830 So in addition to this, you will have a point here, and 792 00:53:56,830 --> 00:53:58,520 you will have a point here. 793 00:53:58,520 --> 00:54:03,770 So the case in two dimensions, M equals 2, you will have four 794 00:54:03,770 --> 00:54:09,270 points like this. 795 00:54:09,270 --> 00:54:16,510 So in this case you are increasing the number of 796 00:54:16,510 --> 00:54:25,830 signal points by a factor of two. 797 00:54:30,440 --> 00:54:32,380 But this does come at a price, right? 798 00:54:32,380 --> 00:54:35,280 Because if you're increasing the number of signal points, 799 00:54:35,280 --> 00:54:37,450 the number of nearest neighbors is also going to 800 00:54:37,450 --> 00:54:39,300 increase by a factor of two. 801 00:54:39,300 --> 00:54:41,410 In this case, you have two nearest neighbors now. 802 00:55:07,210 --> 00:55:09,540 So you have both of the effects going on. 803 00:55:09,540 --> 00:55:12,680 I mean ultimately, all the signal sets, as M goes to 804 00:55:12,680 --> 00:55:15,570 infinity, are going to approach the Shannon limit. 805 00:55:15,570 --> 00:55:18,510 So again, but they do suffer from the same kind of 806 00:55:18,510 --> 00:55:21,270 drawbacks that we saw for the orthogonal signal sets. 807 00:55:21,270 --> 00:55:23,830 So these are more of theoretical interest as 808 00:55:23,830 --> 00:55:28,130 opposed to real, practical implementation points. 809 00:55:28,130 --> 00:55:29,380 Any questions? 810 00:55:33,540 --> 00:55:34,790 AUDIENCE: Minus [INAUDIBLE] 811 00:55:37,310 --> 00:55:38,570 PROFESSOR: It's a bit involved to show. 812 00:55:38,570 --> 00:55:42,760 I think you'll be showing it in the homework exercise or 813 00:55:42,760 --> 00:55:46,530 for an exam problem at one point. 814 00:55:46,530 --> 00:55:50,710 You basically start with the orthogonal signal sets here, 815 00:55:50,710 --> 00:55:54,590 look at the inner product here, and use the relation 816 00:55:54,590 --> 00:55:57,110 between the simplex signal sets and the orthogonal signal 817 00:55:57,110 --> 00:55:59,770 sets to do a subtraction of the mean. 818 00:55:59,770 --> 00:56:01,445 And then you can derive in a product expression. 819 00:56:59,690 --> 00:57:02,265 OK, so let's now move on to the bandwidth-limited regime. 820 00:57:18,548 --> 00:57:20,990 OK, so the main difference between -- 821 00:57:20,990 --> 00:57:22,946 yes? 822 00:57:22,946 --> 00:57:24,196 AUDIENCE: [INAUDIBLE] 823 00:57:29,220 --> 00:57:31,040 PROFESSOR: You mean here? 824 00:57:31,040 --> 00:57:32,120 For each -- 825 00:57:32,120 --> 00:57:33,503 AUDIENCE: How far can you [INAUDIBLE]? 826 00:57:33,503 --> 00:57:36,270 PROFESSOR: A factor of two. 827 00:57:36,270 --> 00:57:37,550 I said a factor right? 828 00:57:37,550 --> 00:57:39,850 So if you have M points in the orthogonal signal set, you 829 00:57:39,850 --> 00:57:44,690 have 2M points in the bi-orthogonal signal set 830 00:57:44,690 --> 00:57:45,940 because for each -- 831 00:57:48,460 --> 00:57:52,470 AUDIENCE: In the orthogonal sets you have 3. 832 00:57:52,470 --> 00:57:56,050 PROFESSOR: And in this you have 6. 833 00:57:56,050 --> 00:58:00,290 So let's look at the bandwidth-limited regime. 834 00:58:00,290 --> 00:58:03,480 The trade-off we care about is the probability of error for 835 00:58:03,480 --> 00:58:06,260 two dimensions as a function of SNR norm. 836 00:58:09,410 --> 00:58:10,960 OK? 837 00:58:10,960 --> 00:58:15,530 The baseline scheme here is your M-PAM scheme. 838 00:58:15,530 --> 00:58:18,210 And we showed a couple of lectures ago that the 839 00:58:18,210 --> 00:58:22,460 probability of error for two dimensions is approximately 4 840 00:58:22,460 --> 00:58:24,542 Q times root 3 SNR norm. 841 00:58:24,542 --> 00:58:25,792 OK? 842 00:58:30,420 --> 00:58:32,450 So let's plot the performance graph. 843 00:58:40,576 --> 00:58:43,810 So on the x-axis I plot SNR norm. 844 00:58:43,810 --> 00:58:47,980 On the y-axis I plot Ps of E again on a log-log scale. 845 00:58:53,310 --> 00:58:55,770 This is ten to the negative six, ten to the negative five, 846 00:58:55,770 --> 00:58:57,010 ten to the negative four, ten to the negative 847 00:58:57,010 --> 00:58:59,140 three and so on. 848 00:58:59,140 --> 00:59:02,900 The intercept is at 0 dB, this point here. 849 00:59:02,900 --> 00:59:05,420 So this is your Shannon limit in the 850 00:59:05,420 --> 00:59:08,320 bandwidth-limited regime. 851 00:59:08,320 --> 00:59:12,560 Now your performance curve is going to be something like 852 00:59:12,560 --> 00:59:14,010 this for the M-PAM system. 853 00:59:22,710 --> 00:59:28,480 And we want to basically quantify now the effective 854 00:59:28,480 --> 00:59:30,650 coding gain in this regime. 855 00:59:30,650 --> 00:59:33,900 So it's the same story as in the power-limited regime. 856 00:59:33,900 --> 00:59:39,610 We will start off with the probability of error using the 857 00:59:39,610 --> 00:59:51,000 union bound estimate, which is K_min of A times Q of d_min 858 00:59:51,000 --> 00:59:52,575 over 2 sigma. 859 00:59:55,620 --> 01:00:00,460 Now Ps of E is the probability of error for two dimensions. 860 01:00:00,460 --> 01:00:04,740 Assuming we have N dimensions here, it is two times the 861 01:00:04,740 --> 01:00:10,400 probability of error over N. So this is probability of 862 01:00:10,400 --> 01:00:12,140 error per symbol. 863 01:00:12,140 --> 01:00:15,120 We are dividing by the number of dimensions and multiplying 864 01:00:15,120 --> 01:00:18,770 by two because it is for two dimensions. 865 01:00:18,770 --> 01:00:28,980 And this is going to be 2 times K_min of A over N, times 866 01:00:28,980 --> 01:00:32,230 Q of d_min over 2 sigma. 867 01:00:39,530 --> 01:00:41,900 So now let's do the same trick that we did in the 868 01:00:41,900 --> 01:00:43,150 power-limited regime. 869 01:00:46,110 --> 01:00:49,860 We will write Ps of E be approximately -- 870 01:00:49,860 --> 01:00:53,980 and instead of the right hand side, I will write it as Ks of 871 01:00:53,980 --> 01:01:01,950 A times Q of root 3 SNR norm. 872 01:01:06,390 --> 01:01:08,900 But because I have a factor of four up there, I will also 873 01:01:08,900 --> 01:01:12,790 multiply and divide by 4. 874 01:01:12,790 --> 01:01:16,024 That's not going to change anything. 875 01:01:16,024 --> 01:01:17,888 AUDIENCE: [INAUDIBLE] 876 01:01:17,888 --> 01:01:19,470 per two dimensions or by symbol. 877 01:01:19,470 --> 01:01:20,925 PROFESSOR: Per two dimensions. 878 01:01:20,925 --> 01:01:23,510 And for bandwidth-limited regime, we normalize 879 01:01:23,510 --> 01:01:24,600 everything by two dimensions. 880 01:01:24,600 --> 01:01:28,222 AUDIENCE: But for N band we used it for symbols, right? 881 01:01:28,222 --> 01:01:30,790 The calculations represent -- 882 01:01:30,790 --> 01:01:32,040 PROFESSOR: No, it was for two dimensions. 883 01:01:37,720 --> 01:01:42,120 OK?, so now let's compare the two expressions. 884 01:01:42,120 --> 01:01:50,960 We have Ks of A, which we'll define by two times K_min of A 885 01:01:50,960 --> 01:02:05,810 over N. And this is the number of nearest neighbors per two 886 01:02:05,810 --> 01:02:08,740 dimensions. 887 01:02:08,740 --> 01:02:19,780 And I can write 3 times SNR norm is d_min squared over 4 888 01:02:19,780 --> 01:02:27,970 sigma squared, OK? 889 01:02:27,970 --> 01:02:30,550 Actually, I was missing this factor of gamma. 890 01:02:30,550 --> 01:02:32,070 I knew I was missing something. 891 01:02:32,070 --> 01:02:34,970 It's not just 3 SNR norm. 892 01:02:34,970 --> 01:02:39,090 It's 3 times this nominal coding gain times SNR norm. 893 01:02:42,080 --> 01:02:47,550 So 3 times SNR norm times gamma_c of A here. 894 01:02:47,550 --> 01:02:50,020 So this is d_min squared over 2N_0. 895 01:02:54,850 --> 01:03:02,470 So what do I have for the gamma_c c o A is d_min squared 896 01:03:02,470 --> 01:03:06,070 over 6 N_0 times SNR norm. 897 01:03:08,740 --> 01:03:12,480 Remember SNR norm is SNR over 2 to the rho minus 1. 898 01:03:12,480 --> 01:03:15,730 It's normalized signal-to-noise ratio. 899 01:03:15,730 --> 01:03:20,890 So this is the d_min squared times 2 to the rho minus 1 900 01:03:20,890 --> 01:03:24,300 over 6 times N_0 times SNR. 901 01:03:24,300 --> 01:03:27,390 But N_0 times SNR is just Es. 902 01:03:27,390 --> 01:03:30,180 So this is 6 times Es. 903 01:03:30,180 --> 01:03:31,740 So this is the expression you have for the 904 01:03:31,740 --> 01:03:32,990 nominal coding gain. 905 01:03:47,760 --> 01:03:52,140 So now again, given a signal set A, I can find those two 906 01:03:52,140 --> 01:03:59,370 parameters, the nominal coding gain and the number of nearest 907 01:03:59,370 --> 01:04:01,440 neighbors per two dimensions. 908 01:04:01,440 --> 01:04:05,686 And I can use those to plot on the Ps of E 909 01:04:05,686 --> 01:04:07,990 versus SNR norm curve. 910 01:04:07,990 --> 01:04:09,405 Again, the exact same story. 911 01:04:12,940 --> 01:04:17,920 If I have a certain nominal coding gain, I will simply 912 01:04:17,920 --> 01:04:23,460 shift my curve around the x-axis by that factor as a 913 01:04:23,460 --> 01:04:25,550 pure translation. 914 01:04:25,550 --> 01:04:32,120 And then because I have a factor of Ks A over 4, I'm 915 01:04:32,120 --> 01:04:33,396 going to plot -- 916 01:04:33,396 --> 01:04:35,340 how did that expression go? 917 01:04:35,340 --> 01:04:37,590 This expression on the top. 918 01:04:37,590 --> 01:04:41,440 Remember this curve here is 4 times root 3 SNR norm, right? 919 01:04:41,440 --> 01:04:44,740 So the multiplicative factor I have is Ks(A) over 4. 920 01:04:44,740 --> 01:04:46,880 So I will shift my curve up by that factor. 921 01:04:50,180 --> 01:04:52,360 And I will get something which is like this. 922 01:04:57,110 --> 01:05:00,430 And that's my union bound estimate for this new 923 01:05:00,430 --> 01:05:04,470 constellation A. 924 01:05:04,470 --> 01:05:06,340 The distance here is gamma effective. 925 01:05:08,990 --> 01:05:19,400 This distance here is gamma_c of A. Are there any questions? 926 01:05:19,400 --> 01:05:21,870 So now we also have a similar rule of thumb as in the 927 01:05:21,870 --> 01:05:24,450 power-limited regime. 928 01:05:24,450 --> 01:05:27,140 I will write that rule of thumb here. 929 01:05:27,140 --> 01:05:34,430 We have that gamma effective is going to be gamma_c in dB 930 01:05:34,430 --> 01:05:43,020 minus 0.2 times log2 times Ks of A over 4 in dB. 931 01:05:46,670 --> 01:05:51,400 OK, so again, a factor of 2NKs will decrease your nominal 932 01:05:51,400 --> 01:05:54,830 coding gain by a factor of 0.2 in dB. 933 01:05:58,620 --> 01:05:59,870 Are there any questions? 934 01:06:03,260 --> 01:06:06,260 Well, so I mean this is the end of chapter five. 935 01:06:06,260 --> 01:06:08,830 We still have like ten minutes remaining, so I thought I 936 01:06:08,830 --> 01:06:11,480 would give you a preview of the next chapter. 937 01:06:11,480 --> 01:06:13,510 Professor Forney will be coming next class, and he will 938 01:06:13,510 --> 01:06:15,970 be starting chapter six all over I believe. 939 01:06:19,030 --> 01:06:21,303 This chalk is much nicer to erase than that other one. 940 01:06:43,000 --> 01:06:48,650 So in chapter six, we'll be looking at the 941 01:06:48,650 --> 01:06:49,900 binary block codes. 942 01:07:00,130 --> 01:07:04,540 OK, and what is the main architecture of these codes? 943 01:07:04,540 --> 01:07:09,700 Well, for an encoder, remember we have K bits coming in. 944 01:07:09,700 --> 01:07:12,315 And we pass it now through a binary block code. 945 01:07:16,070 --> 01:07:23,220 And what we get out is a binary sequence 946 01:07:23,220 --> 01:07:25,425 of length N. OK? 947 01:07:25,425 --> 01:07:31,750 So x belongs to c, where c is a subset of binary sequences 948 01:07:31,750 --> 01:07:33,550 of length N. 949 01:07:33,550 --> 01:07:37,240 So we start with K bits, and we convert them to N bits, 950 01:07:37,240 --> 01:07:42,610 where N is going to be greater than or equal to K. Once we 951 01:07:42,610 --> 01:07:46,590 have this sequence of N bits, what we end up doing is we 952 01:07:46,590 --> 01:07:49,820 pass it through a binary signal constellation. 953 01:07:54,710 --> 01:07:56,690 So I'm writing in as binary signaling. 954 01:07:56,690 --> 01:08:02,200 And what we get out is S of x, which is a 955 01:08:02,200 --> 01:08:04,940 sequence of N symbols. 956 01:08:04,940 --> 01:08:07,110 Each symbol can either take value minus alpha or alpha. 957 01:08:07,110 --> 01:08:12,130 So this binary signalling is actually quite a trivial step. 958 01:08:12,130 --> 01:08:16,560 Basically, the relation is S of 0 is going to be alpha, S 959 01:08:16,560 --> 01:08:18,500 of 1 is going to be minus alpha. 960 01:08:18,500 --> 01:08:21,520 If I have a 0 coming in, I will produce an alpha. 961 01:08:21,520 --> 01:08:24,520 if I have a 1 coming in, I will produce a minus alpha. 962 01:08:24,520 --> 01:08:26,689 So this part here is trivial. 963 01:08:26,689 --> 01:08:29,253 All our energy will be focused on will be 964 01:08:29,253 --> 01:08:31,340 to find a good code. 965 01:08:31,340 --> 01:08:33,970 Given a sequence of input bits, we want to find a good 966 01:08:33,970 --> 01:08:39,080 binary code in order to optimize the minimum 967 01:08:39,080 --> 01:08:40,880 distance and so on. 968 01:08:40,880 --> 01:08:43,540 So that's the architecture that we will be using for the 969 01:08:43,540 --> 01:08:45,210 binary linear codes. 970 01:08:45,210 --> 01:08:46,800 Both chapters six and eight will be only 971 01:08:46,800 --> 01:08:48,250 focusing on this part. 972 01:08:48,250 --> 01:08:50,550 And this binary signalling scheme, for the most part, 973 01:08:50,550 --> 01:08:52,265 will be implicit in our architecture. 974 01:08:55,069 --> 01:08:58,399 So at this point, one might ask is there anything to lose 975 01:08:58,399 --> 01:09:01,359 in having imposed this type of architecture? 976 01:09:01,359 --> 01:09:03,899 Remember, the nice thing about this architecture is we have 977 01:09:03,899 --> 01:09:06,910 coding, which is going on here, and we have modulation 978 01:09:06,910 --> 01:09:08,460 that is happening here. 979 01:09:08,460 --> 01:09:10,960 And the modulation step is quite simple. 980 01:09:10,960 --> 01:09:23,156 So we have a separation between coding and modulation. 981 01:09:23,156 --> 01:09:24,406 OK? 982 01:09:29,689 --> 01:09:32,600 And the question is, is this the right thing to do? 983 01:09:32,600 --> 01:09:34,520 Now, it turns out that if we are going to live in this 984 01:09:34,520 --> 01:09:36,984 binary world where we are only constrained ourselves to 985 01:09:36,984 --> 01:09:40,134 sending binary signals over the channel, this is in fact 986 01:09:40,134 --> 01:09:41,800 an economical structure. 987 01:09:41,800 --> 01:09:44,310 There isn't much improvement we can get, and that is quite 988 01:09:44,310 --> 01:09:45,424 obvious, right? 989 01:09:45,424 --> 01:09:48,410 But it turns out that if you are going to go for non-binary 990 01:09:48,410 --> 01:09:51,870 signaling, particularly in the bandwidth-limited regime, 991 01:09:51,870 --> 01:09:55,460 there is a cost to be paid for this kind of separation. 992 01:09:55,460 --> 01:09:57,950 In fact, in the 1980's there was this whole idea of 993 01:09:57,950 --> 01:10:01,060 turbo-coded modulation or trellis-coded modulation. 994 01:10:01,060 --> 01:10:03,680 And the idea there was to combine coding and modulation 995 01:10:03,680 --> 01:10:05,110 in one step. 996 01:10:05,110 --> 01:10:19,440 So this is not optimal for non-binary signalling, but's 997 01:10:19,440 --> 01:10:25,270 it's OK for binary. 998 01:10:30,570 --> 01:10:33,820 So we will be focusing on this type of an architecture. 999 01:10:33,820 --> 01:10:37,130 The other issue is OK, nobody told us that you can only send 1000 01:10:37,130 --> 01:10:39,080 binary signals over the channel. 1001 01:10:39,080 --> 01:10:42,270 If you want to achieve the Shannon limit, when we derived 1002 01:10:42,270 --> 01:10:44,580 the capacity expression -- 1003 01:10:44,580 --> 01:10:46,970 or we just stated it but you can derive it. 1004 01:10:46,970 --> 01:10:49,660 We said that the Shannon limit is always less 1005 01:10:49,660 --> 01:10:52,830 than log2 1 plus SNR. 1006 01:10:52,830 --> 01:10:55,980 And we never said that we can only send binary signals over 1007 01:10:55,980 --> 01:10:56,840 the channel. 1008 01:10:56,840 --> 01:10:59,070 So it could be that this is some inherence of optimality 1009 01:10:59,070 --> 01:11:00,770 in this type of architecture. 1010 01:11:00,770 --> 01:11:03,730 Why send binary signals in the first place? 1011 01:11:03,730 --> 01:11:06,940 Again, it turns out that the regime that we are interested 1012 01:11:06,940 --> 01:11:10,130 in here is the power-limited regime because the spectral 1013 01:11:10,130 --> 01:11:12,220 efficiency can never be more than two bits per two 1014 01:11:12,220 --> 01:11:13,810 dimensions, right? 1015 01:11:13,810 --> 01:11:16,340 Remember, rho is 2K over N here, and K is going to be 1016 01:11:16,340 --> 01:11:17,290 less than that. 1017 01:11:17,290 --> 01:11:22,130 So we are inherently in the power-limited regime. 1018 01:11:22,130 --> 01:11:24,770 And so a natural thing to do in order to understand how 1019 01:11:24,770 --> 01:11:28,120 much fundamental loss we have in imposing this type of 1020 01:11:28,120 --> 01:11:32,360 binary constraint over the signals is to not look at this 1021 01:11:32,360 --> 01:11:36,470 expression, but to find the best possible performance we 1022 01:11:36,470 --> 01:11:39,390 can have if we constrain ourselves to binary signals 1023 01:11:39,390 --> 01:11:41,090 over the channel. 1024 01:11:41,090 --> 01:11:43,910 If we impose a certain signaling constraint, we can 1025 01:11:43,910 --> 01:11:49,330 find another fundamental limit on the spectral efficiency. 1026 01:11:49,330 --> 01:11:53,070 So the point being, the best possible binary code can only 1027 01:11:53,070 --> 01:11:55,450 achieve this upper bound here. 1028 01:11:55,450 --> 01:11:57,050 Now, this upper bound has to be less than 1029 01:11:57,050 --> 01:11:58,300 log2 1 plus SNR, right? 1030 01:12:03,170 --> 01:12:06,070 Because the Shannon code assumes the binary signaling 1031 01:12:06,070 --> 01:12:07,640 is a special case. 1032 01:12:07,640 --> 01:12:10,700 So it turns out that this expression is actually quite 1033 01:12:10,700 --> 01:12:12,310 tedious to write out. 1034 01:12:12,310 --> 01:12:15,090 I'm not even sure if the closed-form expression exists, 1035 01:12:15,090 --> 01:12:16,820 but you can calculate this numerically. 1036 01:12:16,820 --> 01:12:19,760 It's just some kind of a convex optimization problem. 1037 01:12:19,760 --> 01:12:22,350 And now, to understand how much loss we have, we can 1038 01:12:22,350 --> 01:12:26,090 compare the two expressions in the power-limited regime. 1039 01:12:26,090 --> 01:12:30,690 So I'm going to plot the curves for the two cases. 1040 01:12:30,690 --> 01:12:35,350 I'm going to plot the spectral efficiency on the y-axis as a 1041 01:12:35,350 --> 01:12:36,500 function of Eb N_0. 1042 01:12:36,500 --> 01:12:39,000 You can easily convert from SNR to Eb N_0 using the 1043 01:12:39,000 --> 01:12:42,210 relations we discussed in chapter four. 1044 01:12:42,210 --> 01:12:45,030 Now we first plot the Shannon limit. 1045 01:12:45,030 --> 01:12:48,920 The ultimate Shannon limit is minus 1.59 dB. 1046 01:12:48,920 --> 01:12:52,000 This is going to be 0 dB here. 1047 01:12:52,000 --> 01:12:54,100 And if I'm plot the Shannon limit, it will be 1048 01:12:54,100 --> 01:12:56,260 some code like this. 1049 01:12:56,260 --> 01:12:58,080 It increases with Eb N_0. 1050 01:12:58,080 --> 01:13:01,210 And minus 1.59 dB is 0. 1051 01:13:01,210 --> 01:13:03,130 This point here is 1. 1052 01:13:03,130 --> 01:13:05,070 Some point here is 2. 1053 01:13:05,070 --> 01:13:09,200 So actually I want to draw this horizontal asymptote at 1054 01:13:09,200 --> 01:13:13,030 two when rho Shannon equals 2. 1055 01:13:13,030 --> 01:13:15,720 So this is my rho Shannon. 1056 01:13:15,720 --> 01:13:19,200 Now, I can now also plot rho binary from 1057 01:13:19,200 --> 01:13:21,020 this expression here. 1058 01:13:21,020 --> 01:13:23,220 I can never exceed two bits per two dimensions. 1059 01:13:23,220 --> 01:13:25,720 So I had plotted this horizontal line. 1060 01:13:25,720 --> 01:13:28,720 So as Eb N_0 goes to infinity, the most I can get is two bits 1061 01:13:28,720 --> 01:13:30,250 per two dimensions. 1062 01:13:30,250 --> 01:13:35,380 If Eb N_0 is smaller, I cannot do better so I will be doing 1063 01:13:35,380 --> 01:13:38,210 worse and worse, and I will always be below this line. 1064 01:13:38,210 --> 01:13:41,330 Ultimately, it can be shown that I will have the same 1065 01:13:41,330 --> 01:13:43,210 Shannon limit here. 1066 01:13:43,210 --> 01:13:45,020 So this is rho binary. 1067 01:13:48,380 --> 01:13:51,830 And the main observation here is if I'm in the power-limited 1068 01:13:51,830 --> 01:13:55,150 regime, which is say, in this part of the curve -- 1069 01:13:55,150 --> 01:13:57,650 like, say around one bit per two dimension -- the gap here 1070 01:13:57,650 --> 01:13:59,390 is quite small. 1071 01:13:59,390 --> 01:14:02,620 In fact, this gap can be shown to be at most 0.2 dB. 1072 01:14:05,190 --> 01:14:08,130 So if I want to achieve a certain spectral efficiency, 1073 01:14:08,130 --> 01:14:10,855 if I impose a binary signaling constraint, the additional Eb 1074 01:14:10,855 --> 01:14:15,300 N_0 that I require when the spectral efficiency is 1 dB, 1075 01:14:15,300 --> 01:14:18,360 one bit per two dimensions, is at most 0.2 dB. 1076 01:14:18,360 --> 01:14:22,280 So there is not much to lose by imposing a binary signaling 1077 01:14:22,280 --> 01:14:23,870 constraint. 1078 01:14:23,870 --> 01:14:26,220 Said in different words words if you had the best possible 1079 01:14:26,220 --> 01:14:29,150 binary linear code, followed by a binary signaling 1080 01:14:29,150 --> 01:14:33,210 constellation, the most you can lose is 0.2 dB. 1081 01:14:33,210 --> 01:14:36,070 And so this type of an architecture does make sense. 1082 01:14:36,070 --> 01:14:39,783 There are a lot of details that the next powerpoint will 1083 01:14:39,783 --> 01:14:44,680 be to kind of formalize the notion of a binary block code, 1084 01:14:44,680 --> 01:14:46,850 then specialize it to the case of binary linear 1085 01:14:46,850 --> 01:14:48,450 codes and so on. 1086 01:14:48,450 --> 01:14:51,370 Unfortunately, the first thing to do is to start with some 1087 01:14:51,370 --> 01:14:54,970 finite field theory because there's a lot of finite field 1088 01:14:54,970 --> 01:14:57,780 algebra involved in this algebraic block codes. 1089 01:14:57,780 --> 01:15:02,580 So how many of you have seen finite field theory before? 1090 01:15:02,580 --> 01:15:05,810 OK, if you haven't seen it, don't worry about it. 1091 01:15:05,810 --> 01:15:08,570 We'll be starting right from the basics. 1092 01:15:08,570 --> 01:15:11,580 But I think I will be stopping here for today because I don't 1093 01:15:11,580 --> 01:15:13,390 want to go into those details today.