1 00:00:00,000 --> 00:00:00,499 2 00:00:00,499 --> 00:00:03,144 The following content is provided under a Creative 3 00:00:03,144 --> 00:00:03,810 Commons license. 4 00:00:03,810 --> 00:00:05,518 Your support will help MIT OpenCourseWare 5 00:00:05,518 --> 00:00:10,405 continue to offer high-quality educational resources for free. 6 00:00:10,405 --> 00:00:12,530 To make a donation, or to view additional materials 7 00:00:12,530 --> 00:00:17,080 from hundreds of MIT courses, visit MIT OpenCourseWare 8 00:00:17,080 --> 00:00:20,550 at ocw.mit.edu. 9 00:00:20,550 --> 00:00:22,950 PROFESSOR STRANG: So, thanks for coming today. 10 00:00:22,950 --> 00:00:30,370 This is a key lecture in the application of Fourier, 11 00:00:30,370 --> 00:00:33,090 you could say. 12 00:00:33,090 --> 00:00:35,640 So convolution is the big word. 13 00:00:35,640 --> 00:00:39,250 And a major application of convolution 14 00:00:39,250 --> 00:00:43,200 is filtering, signal processing. 15 00:00:43,200 --> 00:00:46,530 So we'll develop that application. 16 00:00:46,530 --> 00:00:48,940 But it's nothing but convolution. 17 00:00:48,940 --> 00:00:53,860 So the key idea is these convolution rules, 18 00:00:53,860 --> 00:00:55,340 where they come from. 19 00:00:55,340 --> 00:00:58,510 And what these new symbols, that's 20 00:00:58,510 --> 00:01:03,400 the symbol for the convolution of two functions. 21 00:01:03,400 --> 00:01:05,390 They could be functions here. 22 00:01:05,390 --> 00:01:09,240 Here they're long vectors of coefficients, 23 00:01:09,240 --> 00:01:14,240 and so these are the rules. 24 00:01:14,240 --> 00:01:16,240 So it's just a little bit of algebra. 25 00:01:16,240 --> 00:01:21,110 But it just is so central to all this subject. 26 00:01:21,110 --> 00:01:26,290 Signal processing is certainly the most important little thing 27 00:01:26,290 --> 00:01:27,310 to know. 28 00:01:27,310 --> 00:01:29,230 That if I multiply two functions, 29 00:01:29,230 --> 00:01:31,650 so that's where we started last time. 30 00:01:31,650 --> 00:01:33,790 If I have a function f with Fourier 31 00:01:33,790 --> 00:01:39,900 coefficients c, and a function g with coefficients d, then, 32 00:01:39,900 --> 00:01:44,060 oh, wrong. 33 00:01:44,060 --> 00:01:48,590 I convolve the coefficients, right. 34 00:01:48,590 --> 00:01:53,900 So f has coefficients c, g has coefficients d. 35 00:01:53,900 --> 00:01:57,920 So if I multiply the functions I definitely do not just 36 00:01:57,920 --> 00:02:00,580 multiply each c times the d. 37 00:02:00,580 --> 00:02:04,640 I do this convolution operation, which we have to remember. 38 00:02:04,640 --> 00:02:09,790 That's our main first step is to remember what that was about. 39 00:02:09,790 --> 00:02:13,550 And then this is the other direction. 40 00:02:13,550 --> 00:02:15,910 So we didn't see this before. 41 00:02:15,910 --> 00:02:20,980 That I mean, there's always this fantastic symmetry 42 00:02:20,980 --> 00:02:26,360 between physical space and frequency space. 43 00:02:26,360 --> 00:02:30,810 And convolution in one is multiplication in the other. 44 00:02:30,810 --> 00:02:33,240 It's so easy to remember. 45 00:02:33,240 --> 00:02:36,890 If I multiply in one space, I do a convolution 46 00:02:36,890 --> 00:02:38,240 in the other space. 47 00:02:38,240 --> 00:02:41,490 If I do a convolution of functions, 48 00:02:41,490 --> 00:02:45,500 I do a multiplication of coefficients. 49 00:02:45,500 --> 00:02:49,510 So that's the rule to know. 50 00:02:49,510 --> 00:02:52,440 And now if we expand on it, by sort 51 00:02:52,440 --> 00:02:54,750 of seeing again what it means. 52 00:02:54,750 --> 00:02:59,780 So let me do a quick repeat of this first step 53 00:02:59,780 --> 00:03:04,840 to remember what this symbol, convolution symbol, means. 54 00:03:04,840 --> 00:03:07,790 And then another thing I have to do. 55 00:03:07,790 --> 00:03:11,380 Here, I'm talking about the infinite case, functions, 56 00:03:11,380 --> 00:03:15,810 and with a whole infinite sequence of coefficients. 57 00:03:15,810 --> 00:03:20,530 I've got to do the cyclic case, too. 58 00:03:20,530 --> 00:03:23,590 Which goes with the discrete transform. 59 00:03:23,590 --> 00:03:26,630 OK, but let's start with the infinite case 60 00:03:26,630 --> 00:03:29,160 as we did last time. 61 00:03:29,160 --> 00:03:31,500 Maybe, here's something. 62 00:03:31,500 --> 00:03:32,780 Here's a suggestion. 63 00:03:32,780 --> 00:03:36,480 We had f(x). 64 00:03:36,480 --> 00:03:40,840 We started with f(x), as the sum -- and remember, 65 00:03:40,840 --> 00:03:45,050 to have nice formulas, we're doing the complex version -- 66 00:03:45,050 --> 00:03:47,470 c_k*e^(ikx). 67 00:03:47,470 --> 00:03:49,350 Let me suggest something. 68 00:03:49,350 --> 00:03:52,880 Let me write z for e^(ikx). 69 00:03:52,880 --> 00:04:00,060 So I'm going to just write that at the sum of c_k*z^k. 70 00:04:00,060 --> 00:04:01,600 So z is e^(ix). 71 00:04:01,600 --> 00:04:05,970 72 00:04:05,970 --> 00:04:09,300 This is like a good point to make anyway. 73 00:04:09,300 --> 00:04:14,220 When we have this e^(ikx), it's natural to think of that 74 00:04:14,220 --> 00:04:18,520 as a complex number on the unit circle. 75 00:04:18,520 --> 00:04:19,020 Right? 76 00:04:19,020 --> 00:04:21,180 That's always the message, e to the i 77 00:04:21,180 --> 00:04:23,390 real is on the unit circle. 78 00:04:23,390 --> 00:04:25,100 Absolute value one. 79 00:04:25,100 --> 00:04:29,760 And this is great for periodic functions. 80 00:04:29,760 --> 00:04:32,870 Because if these functions have period 2pi, 81 00:04:32,870 --> 00:04:39,750 and if we think of our function as being on the circle, 82 00:04:39,750 --> 00:04:42,290 it obviously has period 2pi. 83 00:04:42,290 --> 00:04:44,770 I mean, the picture says, yes. 84 00:04:44,770 --> 00:04:47,030 If I go 2pi, I come back. 85 00:04:47,030 --> 00:04:52,625 So x is the angle, right? x is the-- In this picture, 86 00:04:52,625 --> 00:04:59,050 a little bit unusual maybe, x there would be the angle. 87 00:04:59,050 --> 00:05:03,320 And it's just a little bit easier to write. 88 00:05:03,320 --> 00:05:06,630 And maybe it even has an official name, the Z-transform. 89 00:05:06,630 --> 00:05:07,660 How do you like that? 90 00:05:07,660 --> 00:05:13,791 You learn a transform just, in fourteen seconds. 91 00:05:13,791 --> 00:05:14,290 Z-transform. 92 00:05:14,290 --> 00:05:17,750 93 00:05:17,750 --> 00:05:20,980 It'll make it easier to multiply by g. 94 00:05:20,980 --> 00:05:26,070 So g is going to be the sum of d_k*z^k. 95 00:05:26,070 --> 00:05:30,100 So just think of these as long, well I'm 96 00:05:30,100 --> 00:05:32,400 tempted to say long polynomials. 97 00:05:32,400 --> 00:05:37,140 I mean, very long, because they can be infinite series. 98 00:05:37,140 --> 00:05:40,460 Infinite negative powers and positive powers. 99 00:05:40,460 --> 00:05:44,410 But just think of it as a bunch of powers of z times 100 00:05:44,410 --> 00:05:46,670 a bunch of powers of z. 101 00:05:46,670 --> 00:05:51,090 And if you multiply a couple of polynomials, 102 00:05:51,090 --> 00:05:54,230 you're doing convolution. 103 00:05:54,230 --> 00:05:56,890 I guess my message is, you've been doing convolution 104 00:05:56,890 --> 00:05:59,740 since the second grade. 105 00:05:59,740 --> 00:06:01,380 That's the real message. 106 00:06:01,380 --> 00:06:04,020 Here, let me show you. 107 00:06:04,020 --> 00:06:06,530 This is convolution too. 108 00:06:06,530 --> 00:06:14,550 Suppose I have to multiply a 123 times 456? 109 00:06:14,550 --> 00:06:17,910 OK, so what do you do? 110 00:06:17,910 --> 00:06:22,640 Remember back, it's a long way back but we can do this. 111 00:06:22,640 --> 00:06:25,470 One, two, three times four, five, six. 112 00:06:25,470 --> 00:06:30,460 So I multiply the three, oh there's a little point. 113 00:06:30,460 --> 00:06:33,170 Where that second-grade teacher's going to panic. 114 00:06:33,170 --> 00:06:37,010 I'm going to write that as 18. 115 00:06:37,010 --> 00:06:41,040 And that's 15, and that's 12. 116 00:06:41,040 --> 00:06:43,830 Sorry about that, yeah. 117 00:06:43,830 --> 00:06:48,930 And 12, 10 and 8, right? 118 00:06:48,930 --> 00:06:50,140 Four, five and six. 119 00:06:50,140 --> 00:06:53,970 So you see the nine multiplications 120 00:06:53,970 --> 00:06:55,740 that you have to do? 121 00:06:55,740 --> 00:06:58,650 Nine multiplications, three times three. 122 00:06:58,650 --> 00:07:04,994 OK, right now imagine those were, 123 00:07:04,994 --> 00:07:06,160 they could have been longer. 124 00:07:06,160 --> 00:07:13,810 But they were finite length filters, we could say. 125 00:07:13,810 --> 00:07:17,200 And now, what does convolution do? 126 00:07:17,200 --> 00:07:19,990 Just what you did in multiplication. 127 00:07:19,990 --> 00:07:23,690 When I add 12, 10 and 6, what am I doing? 128 00:07:23,690 --> 00:07:26,580 I'm putting together the three times the four, 129 00:07:26,580 --> 00:07:29,340 the two times the five, and the one times the six. 130 00:07:29,340 --> 00:07:31,510 That's what convolution, those are the things 131 00:07:31,510 --> 00:07:33,220 that convolution puts together. 132 00:07:33,220 --> 00:07:40,710 So we got an 18, a 27, 28. 133 00:07:40,710 --> 00:07:45,020 13, and 4. 134 00:07:45,020 --> 00:07:49,750 So I guess I'm saying that, alright, here's 135 00:07:49,750 --> 00:07:51,110 what I really want to say. 136 00:07:51,110 --> 00:07:56,600 I want to say that the convolution of those, 137 00:07:56,600 --> 00:08:04,010 with these, four, five, six, is this sequence here-- Oh. 138 00:08:04,010 --> 00:08:04,980 Yeah, that's right. 139 00:08:04,980 --> 00:08:09,430 4, 13, 28. 140 00:08:09,430 --> 00:08:13,020 27, and 18. 141 00:08:13,020 --> 00:08:15,060 If you just look at that. 142 00:08:15,060 --> 00:08:17,690 Where did that 13 come from? 143 00:08:17,690 --> 00:08:21,360 Let's just remember, where does that 13 come from? 144 00:08:21,360 --> 00:08:25,160 That came from, this was z^0. 145 00:08:25,160 --> 00:08:29,890 This 13 is 13 z to the first power, right? 146 00:08:29,890 --> 00:08:33,100 We're just checking all the powers here. 147 00:08:33,100 --> 00:08:35,520 There's 13 z to the first power. 148 00:08:35,520 --> 00:08:37,220 Where do we get a first power? 149 00:08:37,220 --> 00:08:41,620 We get a z^0 times 5z^1. 150 00:08:41,620 --> 00:08:43,460 So that's 5z^1. 151 00:08:43,460 --> 00:08:49,520 And we also have 2z^1, times 4z^0. 152 00:08:49,520 --> 00:08:50,040 Right? 153 00:08:50,040 --> 00:08:52,170 Two times four gave the eight. 154 00:08:52,170 --> 00:08:55,070 So the eight from there and the five from there 155 00:08:55,070 --> 00:08:56,650 produce that 13. 156 00:08:56,650 --> 00:09:00,120 And that's just what you did in multiplication. 157 00:09:00,120 --> 00:09:01,630 Right? 158 00:09:01,630 --> 00:09:06,620 So that's multiplication of two series. 159 00:09:06,620 --> 00:09:08,490 That's not cyclic. 160 00:09:08,490 --> 00:09:11,010 This is definitely not yet cyclic, 161 00:09:11,010 --> 00:09:13,750 but we'll make it cyclic in a minute. 162 00:09:13,750 --> 00:09:20,720 This is the infinite one, except that we had all zeroes beyond. 163 00:09:20,720 --> 00:09:24,500 OK, so that if you, in non-cyclic convolution like 164 00:09:24,500 --> 00:09:29,280 this, if I have length m and length n, 165 00:09:29,280 --> 00:09:34,790 then I get length m+n, maybe m+n-1. 166 00:09:34,790 --> 00:09:37,830 OK, so that's convolution. 167 00:09:37,830 --> 00:09:40,590 Without carrying numbers. 168 00:09:40,590 --> 00:09:42,190 Without doing it right. 169 00:09:42,190 --> 00:09:46,360 OK, so and what does that correspond to? 170 00:09:46,360 --> 00:09:48,770 Let me just, so you see it every way. 171 00:09:48,770 --> 00:09:57,410 That corresponds to 1+2z+3z^2, multiplying 4+5z+6z^2. 172 00:09:57,410 --> 00:10:00,160 173 00:10:00,160 --> 00:10:02,830 And it gave-- That's times. 174 00:10:02,830 --> 00:10:10,160 And it gave this thing up to 18z^4. 175 00:10:10,160 --> 00:10:14,400 Just, exactly the multiplication that you've always done. 176 00:10:14,400 --> 00:10:17,380 OK, so that's the idea. 177 00:10:17,380 --> 00:10:19,510 Over on that board I'm going to put a formula 178 00:10:19,510 --> 00:10:22,230 for this convolution operation. 179 00:10:22,230 --> 00:10:27,740 But my point on this board is, you've done it always. 180 00:10:27,740 --> 00:10:29,900 When you multiply a couple of polynomials, 181 00:10:29,900 --> 00:10:31,930 you collect powers. 182 00:10:31,930 --> 00:10:34,760 And that's all convolution is doing, collecting 183 00:10:34,760 --> 00:10:36,820 each power separately. 184 00:10:36,820 --> 00:10:38,120 OK, let's do it. 185 00:10:38,120 --> 00:10:46,950 So then f(x)g(x) is, when I multiply that polynomial 186 00:10:46,950 --> 00:10:53,330 or series by that polynomial, I get some polynomial in, with 187 00:10:53,330 --> 00:10:56,890 coefficients, oh I was changed to l, 188 00:10:56,890 --> 00:11:00,640 just to have a different symbol there. 189 00:11:00,640 --> 00:11:03,270 And what was the formula for l? 190 00:11:03,270 --> 00:11:05,620 For h_l? 191 00:11:05,620 --> 00:11:10,780 What is the coefficient of z^l? 192 00:11:10,780 --> 00:11:15,230 If I multiply that by that, do you remember the story there? 193 00:11:15,230 --> 00:11:21,100 When I multiply that by that and I looked for the terms that 194 00:11:21,100 --> 00:11:23,580 gave me z^l? 195 00:11:23,580 --> 00:11:27,350 OK, well that means that this power times this power is going 196 00:11:27,350 --> 00:11:29,220 to be z^l. 197 00:11:29,220 --> 00:11:33,030 So that the index, do you remember what happened? 198 00:11:33,030 --> 00:11:38,160 It was a lot of different, just the way-- This h_l is here. 199 00:11:38,160 --> 00:11:41,750 Here's h_2 or something. 200 00:11:41,750 --> 00:11:43,990 I've got to do an addition. 201 00:11:43,990 --> 00:11:49,730 Because a bunch of c's come in with different d's, and what's 202 00:11:49,730 --> 00:11:53,820 the deal then? c_k comes in with which d? 203 00:11:53,820 --> 00:11:56,820 This is the magic number there. 204 00:11:56,820 --> 00:12:04,420 What's the subscript that if I look at the coefficient of z^l, 205 00:12:04,420 --> 00:12:07,760 I look at each of these. 206 00:12:07,760 --> 00:12:13,240 And then I pick out the one of these that will give me a z^l. 207 00:12:13,240 --> 00:12:16,690 And which one is it? 208 00:12:16,690 --> 00:12:18,550 d_(l-k). 209 00:12:18,550 --> 00:12:22,690 It's that magic quantity that the eye spots perfectly. 210 00:12:22,690 --> 00:12:32,850 k and l-k, adding to l simply because z^k z^(l-k) multiplies 211 00:12:32,850 --> 00:12:35,600 to z^l. 212 00:12:35,600 --> 00:12:36,880 Same thing. 213 00:12:36,880 --> 00:12:37,540 Right? 214 00:12:37,540 --> 00:12:43,930 OK, this is, now I'll use that notation. 215 00:12:43,930 --> 00:12:47,800 This is c convolved with the d. c 216 00:12:47,800 --> 00:12:50,640 convolved with d is h. c convolved with d is h. 217 00:12:50,640 --> 00:12:55,810 So this is the l-th component. c convolved with d is my symbol. 218 00:12:55,810 --> 00:12:58,450 This h is the convolution. 219 00:12:58,450 --> 00:13:01,660 And that's the convolution rule. 220 00:13:01,660 --> 00:13:04,720 It's just whatever operation you have 221 00:13:04,720 --> 00:13:06,990 to do to get the right answer. 222 00:13:06,990 --> 00:13:11,460 The right answer when you multiply. 223 00:13:11,460 --> 00:13:13,540 So that one. 224 00:13:13,540 --> 00:13:16,640 Ready for the discrete case? 225 00:13:16,640 --> 00:13:18,200 The finite case? 226 00:13:18,200 --> 00:13:25,270 The case when, you have power, when z becomes w. 227 00:13:25,270 --> 00:13:27,040 The discrete case. 228 00:13:27,040 --> 00:13:28,850 The cyclic case, sorry. 229 00:13:28,850 --> 00:13:33,950 Maybe emphasize the cyclic case, meaning it circles around, 230 00:13:33,950 --> 00:13:39,730 is the case when z becomes this very special z, on the unit 231 00:13:39,730 --> 00:13:42,390 circle that we know as w. 232 00:13:42,390 --> 00:13:47,510 OK, and we have to say, and of course it's w_N, 233 00:13:47,510 --> 00:13:54,730 I have to tell you in the cyclic case how long the cycle is. 234 00:13:54,730 --> 00:13:57,840 So this would be a case. 235 00:13:57,840 --> 00:14:02,590 Watch what you do here to make this cyclic. 236 00:14:02,590 --> 00:14:09,990 OK, I have three inputs, so N is three here. 237 00:14:09,990 --> 00:14:13,150 Now I'm going to do the cyclic. 238 00:14:13,150 --> 00:14:16,190 So instead of z's, I should be putting w's. 239 00:14:16,190 --> 00:14:18,420 I will. 240 00:14:18,420 --> 00:14:23,240 Just to emphasize, it's good to think of it with the w there, 241 00:14:23,240 --> 00:14:27,510 because the w has this special property that's critical 242 00:14:27,510 --> 00:14:31,750 to everything, OK so now I'm going to, I think of this 243 00:14:31,750 --> 00:14:38,230 as 1w^0, 2w^1, and 3 w squared. 244 00:14:38,230 --> 00:14:40,800 I'm thinking of the same multiplication here, 245 00:14:40,800 --> 00:14:45,570 but now w's. 246 00:14:45,570 --> 00:14:51,540 OK, so I'll end up with 18w^4, and four was the constant 247 00:14:51,540 --> 00:14:55,700 and 13 w's and so on. 248 00:14:55,700 --> 00:14:59,910 All these numbers. 249 00:14:59,910 --> 00:15:01,940 What's the difference? 250 00:15:01,940 --> 00:15:04,070 Ready for the key point? 251 00:15:04,070 --> 00:15:07,340 Now, what's happened in this cyclic case? 252 00:15:07,340 --> 00:15:12,010 Well, the difference is what is w^4? 253 00:15:12,010 --> 00:15:15,740 If we're in the cyclic case, N is three now, 254 00:15:15,740 --> 00:15:19,420 our guys have length three, our circle 255 00:15:19,420 --> 00:15:24,270 is, w now is 1/3 of the way around. 256 00:15:24,270 --> 00:15:27,860 So that's my w, here's my w squared. 257 00:15:27,860 --> 00:15:29,640 Here is my one. 258 00:15:29,640 --> 00:15:32,850 But here it is also w cubed. 259 00:15:32,850 --> 00:15:39,900 So w is the same as w^4, w^2 is the same as w^5. 260 00:15:39,900 --> 00:15:44,320 So, what's the difference? 261 00:15:44,320 --> 00:15:48,440 What can I do now? 262 00:15:48,440 --> 00:15:54,300 If I'm in this discrete case, then my inputs 263 00:15:54,300 --> 00:15:58,030 are a vector of length N, three. 264 00:15:58,030 --> 00:16:00,380 A vector of length N, three. 265 00:16:00,380 --> 00:16:06,180 And I want to get out to a vector of length three. 266 00:16:06,180 --> 00:16:10,180 I'm not happy with that in the cyclic case, 267 00:16:10,180 --> 00:16:13,920 because I'm not happy with w^4. 268 00:16:13,920 --> 00:16:17,790 So now tell me again that last, when I do the multiplication 269 00:16:17,790 --> 00:16:21,920 and I just do it, there's no difference except in how 270 00:16:21,920 --> 00:16:23,710 I write the answer. 271 00:16:23,710 --> 00:16:35,010 18w^4 is the same as? w. w^4 is the same as w, when N is three. 272 00:16:35,010 --> 00:16:41,290 So that 18w^4 cycles back in, with this 13. 273 00:16:41,290 --> 00:16:45,650 So now if I do the, can I show you the symbol? 274 00:16:45,650 --> 00:16:47,730 I'll just do a little circle there. 275 00:16:47,730 --> 00:16:52,130 To say this is now the cyclic convolution. 276 00:16:52,130 --> 00:16:54,620 Then this isn't the answer any more. 277 00:16:54,620 --> 00:16:57,090 The answer now for cyclic convolution, 278 00:16:57,090 --> 00:16:59,190 I only want three numbers. 279 00:16:59,190 --> 00:17:01,600 And if you can tell me what those three numbers are, 280 00:17:01,600 --> 00:17:04,250 we've got it. 281 00:17:04,250 --> 00:17:07,240 Move those over a little to make room for the three numbers. 282 00:17:07,240 --> 00:17:11,940 So there's the answer, the space for the answer. 283 00:17:11,940 --> 00:17:14,430 What do I write in? 284 00:17:14,430 --> 00:17:19,750 How many, what's the constant term? 285 00:17:19,750 --> 00:17:21,460 It's 31, right. 286 00:17:21,460 --> 00:17:22,410 It's 31. 287 00:17:22,410 --> 00:17:25,290 Where did 31 come from? 288 00:17:25,290 --> 00:17:31,540 It came from, you could say, cycling that 27w^3 back with 289 00:17:31,540 --> 00:17:33,000 the four. 290 00:17:33,000 --> 00:17:36,250 Because there's no, w^3 is the same as one. 291 00:17:36,250 --> 00:17:40,730 So I've gone around the circle when I come around to w^3. 292 00:17:40,730 --> 00:17:45,820 So that 27 and 4 combined into that 31. 293 00:17:45,820 --> 00:17:47,930 And let's see. 294 00:17:47,930 --> 00:17:50,130 So what multiplications am I doing here? 295 00:17:50,130 --> 00:17:53,280 One times four gives me the constant. 296 00:17:53,280 --> 00:18:00,520 Two times six, that's two w's and six w squareds, 297 00:18:00,520 --> 00:18:03,870 that's 12w^3, that's 12. 298 00:18:03,870 --> 00:18:10,880 And then 3 w squareds and five w's is 15w^3. 299 00:18:10,880 --> 00:18:13,330 But that's the same as 15. 300 00:18:13,330 --> 00:18:15,330 So that's why we get 31. 301 00:18:15,330 --> 00:18:20,670 We got 4, we've got 12 and we've got 15. 302 00:18:20,670 --> 00:18:25,350 OK, now what's the second component, the w component 303 00:18:25,350 --> 00:18:32,510 of the cyclic convolution? 304 00:18:32,510 --> 00:18:36,120 Tell me what number do I write in, in that middle position? 305 00:18:36,120 --> 00:18:39,100 How many w's do I have? 306 00:18:39,100 --> 00:18:41,300 31 again. 307 00:18:41,300 --> 00:18:47,050 This 28, this 18 is coming back by three. 308 00:18:47,050 --> 00:18:48,920 To 13. 309 00:18:48,920 --> 00:18:53,010 You see, I could have done that multiplication. 310 00:18:53,010 --> 00:18:57,540 So coming back to 31, am I going to get another 31? 311 00:18:57,540 --> 00:19:01,030 No, what's the w^2 guy? 312 00:19:01,030 --> 00:19:01,830 28. 313 00:19:01,830 --> 00:19:02,330 Yeah. 314 00:19:02,330 --> 00:19:09,730 28, because there's no w to the fifth to come back. 315 00:19:09,730 --> 00:19:15,890 So 28 uses three multiplications. 316 00:19:15,890 --> 00:19:21,320 The 31 there used the four and these two. 317 00:19:21,320 --> 00:19:24,630 This came back over to here. 318 00:19:24,630 --> 00:19:28,640 And this 31 used, this 18 came back. 319 00:19:28,640 --> 00:19:30,920 I could have put the 18 here. 320 00:19:30,920 --> 00:19:33,450 You know, I could have lined it up just three. 321 00:19:33,450 --> 00:19:35,470 So I'll write a formula for it. 322 00:19:35,470 --> 00:19:37,790 So that's the answer. 323 00:19:37,790 --> 00:19:41,610 31, 31, 28. 324 00:19:41,610 --> 00:19:44,740 Could I just suggest a little check on that? 325 00:19:44,740 --> 00:19:46,460 Just to check on the numbers. 326 00:19:46,460 --> 00:19:52,590 I think that if I add up these numbers, I get six. 327 00:19:52,590 --> 00:19:57,770 And if I add up those numbers I get 15. 328 00:19:57,770 --> 00:20:01,180 And if I multiply that, I get 90. 329 00:20:01,180 --> 00:20:04,410 And if I add those numbers I get 90, right? 330 00:20:04,410 --> 00:20:08,440 So those add to 90. 331 00:20:08,440 --> 00:20:12,820 So I'm just saying the miracle check is add these, 332 00:20:12,820 --> 00:20:15,040 multiply by the sum of those. 333 00:20:15,040 --> 00:20:17,720 And you get the sum of those. 334 00:20:17,720 --> 00:20:20,700 Why is that? 335 00:20:20,700 --> 00:20:23,480 Somehow seems right, doesn't it? 336 00:20:23,480 --> 00:20:29,790 Because somehow I've taken all nine products here, and so 337 00:20:29,790 --> 00:20:33,460 when I add all the results, I'll have the sum of all nine 338 00:20:33,460 --> 00:20:35,220 of these possible products. 339 00:20:35,220 --> 00:20:37,590 So I'll have 6 times 15. 340 00:20:37,590 --> 00:20:41,310 Actually, here's a good way to look at it. 341 00:20:41,310 --> 00:20:43,730 In doing this, I just set w=1. 342 00:20:43,730 --> 00:20:48,620 343 00:20:48,620 --> 00:20:51,840 I just set w=1 in the polynomial. 344 00:20:51,840 --> 00:20:56,220 When I set w to one, this becomes six, this becomes 15 345 00:20:56,220 --> 00:20:59,540 and the answer becomes 90, when w is one. 346 00:20:59,540 --> 00:21:00,040 Yeah. 347 00:21:00,040 --> 00:21:02,910 So that's another way to see. 348 00:21:02,910 --> 00:21:07,420 And actually, this multiplication, 349 00:21:07,420 --> 00:21:13,400 the second grade version, had w, well, 350 00:21:13,400 --> 00:21:19,600 I'm almost going to say w=10, but not quite that. 351 00:21:19,600 --> 00:21:22,540 Because it's written in the opposite order, right? 352 00:21:22,540 --> 00:21:25,290 If w was ten, this would be one. 353 00:21:25,290 --> 00:21:28,270 Well, anyway, w can't be ten, it's got 354 00:21:28,270 --> 00:21:31,580 to stay on the unit circle, so. 355 00:21:31,580 --> 00:21:35,760 So somewhere in the non-cyclic case, it's something like w=10, 356 00:21:35,760 --> 00:21:38,090 or w=1/10, maybe. 357 00:21:38,090 --> 00:21:39,040 Whatever. 358 00:21:39,040 --> 00:21:40,190 OK. 359 00:21:40,190 --> 00:21:42,670 Could you take the convolution now, 360 00:21:42,670 --> 00:21:45,270 let me give you just another example. 361 00:21:45,270 --> 00:21:49,700 Do it mentally. 362 00:21:49,700 --> 00:21:55,560 What's the convolution of [0, 1, 0, 0], 363 00:21:55,560 --> 00:21:57,990 let me make a little longer. 364 00:21:57,990 --> 00:22:02,200 So I take the, first of all, the non-cyclic convolution 365 00:22:02,200 --> 00:22:05,810 of [0, 0, 1, 0]. 366 00:22:05,810 --> 00:22:07,010 OK. 367 00:22:07,010 --> 00:22:11,780 What's the ordinary-- how long is the answer now? 368 00:22:11,780 --> 00:22:13,320 This is just practice. 369 00:22:13,320 --> 00:22:15,590 How many components am I going to have 370 00:22:15,590 --> 00:22:22,650 in the ordinary convolution of those two guys of length four? 371 00:22:22,650 --> 00:22:24,930 I think it's seven. 372 00:22:24,930 --> 00:22:26,610 I think it'll be seven. 373 00:22:26,610 --> 00:22:34,250 Because we'll have here one, we have no, we have z^0, z^1, z^2, 374 00:22:34,250 --> 00:22:35,910 z^3. 375 00:22:35,910 --> 00:22:39,450 And here we'll have again the same, it would go up to z^6, 376 00:22:39,450 --> 00:22:43,020 but remember there's a z^0, so that's why we have seven. 377 00:22:43,020 --> 00:22:47,440 And what will it be? 378 00:22:47,440 --> 00:22:49,600 What will it be? 379 00:22:49,600 --> 00:22:53,020 I guess, actually, this wasn't a brilliant example, was it? 380 00:22:53,020 --> 00:22:55,970 But let's finish it. 381 00:22:55,970 --> 00:23:01,330 So I'm just multiplying z by z squared, so 382 00:23:01,330 --> 00:23:02,650 what do you get for an answer? 383 00:23:02,650 --> 00:23:13,600 I think the one shows up in the z cubed. 384 00:23:13,600 --> 00:23:14,970 Is that right? 385 00:23:14,970 --> 00:23:22,000 Yeah. z to the first power here, z squared here, z cubed here, 386 00:23:22,000 --> 00:23:26,490 and we always have to remember everything in Chapter 4 starts 387 00:23:26,490 --> 00:23:27,640 at zero. 388 00:23:27,640 --> 00:23:30,340 Zero, z^0's the first one. 389 00:23:30,340 --> 00:23:33,030 OK, that's not too great an example, 390 00:23:33,030 --> 00:23:40,580 because what happens if I do the circular cyclic convolution? 391 00:23:40,580 --> 00:23:42,500 What would be the cyclic convolution? 392 00:23:42,500 --> 00:23:47,150 Now I'm expecting four guys only, right? 393 00:23:47,150 --> 00:23:50,250 The cyclic keeps the same length. 394 00:23:50,250 --> 00:23:55,150 And what would be the answer? 395 00:23:55,150 --> 00:23:58,330 Well, there's nobody to fold back, so it would be just 396 00:23:58,330 --> 00:23:59,910 [0, 0, 0, 1]. 397 00:23:59,910 --> 00:24:06,210 So let me update this a little bit with a one here. 398 00:24:06,210 --> 00:24:08,090 OK, just to practice. 399 00:24:08,090 --> 00:24:12,000 So suppose I do that convolution. 400 00:24:12,000 --> 00:24:15,850 Un-cyclic, first. 401 00:24:15,850 --> 00:24:18,220 What do I change here now? 402 00:24:18,220 --> 00:24:21,780 I've now got a z squared and I've also got a z cubed, 403 00:24:21,780 --> 00:24:25,170 but I only have a single one there. 404 00:24:25,170 --> 00:24:27,330 Let's make it a little more interesting. 405 00:24:27,330 --> 00:24:30,930 OK, make it like so. 406 00:24:30,930 --> 00:24:36,530 Alright, z+z^2 is what we're looking at, z+z^2 is 407 00:24:36,530 --> 00:24:39,820 multiplying z^2+z^3. 408 00:24:39,820 --> 00:24:43,870 And in the long form, what do I get? 409 00:24:43,870 --> 00:24:47,920 Let me make space, tell me what numbers to put in. 410 00:24:47,920 --> 00:24:52,140 If I multiply z+z^2 times z^2+z^3, I get what? 411 00:24:52,140 --> 00:24:54,420 1z^3, how many z^4? 412 00:24:54,420 --> 00:24:57,070 413 00:24:57,070 --> 00:24:58,220 Two of them. 414 00:24:58,220 --> 00:25:00,240 How many z^5? 415 00:25:00,240 --> 00:25:01,760 One, and nobody there. 416 00:25:01,760 --> 00:25:02,890 OK. 417 00:25:02,890 --> 00:25:09,600 And now the cyclic version would be what? 418 00:25:09,600 --> 00:25:12,540 What's my answer now for the cyclic version? 419 00:25:12,540 --> 00:25:15,850 Let me take those out. 420 00:25:15,850 --> 00:25:21,680 So the cyclic version would bring the two back to the zero. 421 00:25:21,680 --> 00:25:24,320 Would bring that one back so there. 422 00:25:24,320 --> 00:25:27,440 That zero will still be zero. 423 00:25:27,440 --> 00:25:32,070 And I checked that I haven't missed anything 424 00:25:32,070 --> 00:25:35,760 by adding those up to get four, and adding this up 425 00:25:35,760 --> 00:25:37,610 to get two times two. 426 00:25:37,610 --> 00:25:38,660 Yeah. 427 00:25:38,660 --> 00:25:42,700 OK, so that's the rule. 428 00:25:42,700 --> 00:25:47,330 And it's a lot cleaner to see these answers 429 00:25:47,330 --> 00:25:52,770 than to see this formula. 430 00:25:52,770 --> 00:25:57,390 And I need, actually of course, now mentioning that formula, 431 00:25:57,390 --> 00:25:59,640 I need a cyclic formula. 432 00:25:59,640 --> 00:26:03,080 So can I write above it the cyclic formula? 433 00:26:03,080 --> 00:26:12,580 What do I get when I'm, instead of this sum, which 434 00:26:12,580 --> 00:26:17,390 went from k equal minus infinity to infinity, 435 00:26:17,390 --> 00:26:25,000 in the cyclic case, h_k is just going to be a sum from zero 436 00:26:25,000 --> 00:26:33,720 to N-1, and there'll be a c_k, and a d something. 437 00:26:33,720 --> 00:26:42,620 And now this is the cyclic case, so I guess this makes us, 438 00:26:42,620 --> 00:26:45,460 I think what our situation now is we understand 439 00:26:45,460 --> 00:26:47,620 the cyclic case from examples. 440 00:26:47,620 --> 00:26:52,590 And now we just have the job of how do I put it into algebra. 441 00:26:52,590 --> 00:26:55,270 How do I put it into symbols? 442 00:26:55,270 --> 00:27:02,280 What's the point? c_k d_n, let's say. 443 00:27:02,280 --> 00:27:08,780 But oh no, I'm looking for h_l, yeah. 444 00:27:08,780 --> 00:27:10,260 So what's the deal? 445 00:27:10,260 --> 00:27:14,780 Here l was k. 446 00:27:14,780 --> 00:27:17,900 That one is the sum of that one and that one. 447 00:27:17,900 --> 00:27:21,630 So here, l is the sum of k and n, but. 448 00:27:21,630 --> 00:27:28,720 What's the but? 449 00:27:28,720 --> 00:27:35,360 I mean somehow I've got some wraparound to do, right? 450 00:27:35,360 --> 00:27:39,200 When I'm doing the cyclic multiplication and I'm doing 451 00:27:39,200 --> 00:27:47,890 the wraparound because w^N, the wraparound comes from that, 452 00:27:47,890 --> 00:27:49,490 right? 453 00:27:49,490 --> 00:27:52,710 That's why I never get as high as N, because when I get to N 454 00:27:52,710 --> 00:27:55,460 I go back to the zeroth power. 455 00:27:55,460 --> 00:28:02,960 OK, so what's the relation of k and n and l here? 456 00:28:02,960 --> 00:28:07,440 We just need the right word to express it. 457 00:28:07,440 --> 00:28:08,740 What's the word? 458 00:28:08,740 --> 00:28:10,770 Mod. 459 00:28:10,770 --> 00:28:17,360 So that's the word I'm looking for. k+n is l. 460 00:28:17,360 --> 00:28:21,320 With wraparound and wraparound means, 461 00:28:21,320 --> 00:28:35,590 the nice notation that people use is mod N. Let's practice. 462 00:28:35,590 --> 00:28:45,670 What is two plus two mod seven? 463 00:28:45,670 --> 00:28:47,400 Four. 464 00:28:47,400 --> 00:28:49,950 Two plus two is four, even in 18.085. 465 00:28:49,950 --> 00:28:51,360 Right, OK. 466 00:28:51,360 --> 00:29:03,570 But two plus two mod three is, two plus two mod three is? 467 00:29:03,570 --> 00:29:04,120 One. 468 00:29:04,120 --> 00:29:06,520 Everybody sees it? 469 00:29:06,520 --> 00:29:13,690 I'm taking z^2 times z^2, z^4, but I'm doing with N=3, 470 00:29:13,690 --> 00:29:18,020 so z^3 is one, so that z^4 is really just z to the first 471 00:29:18,020 --> 00:29:20,080 power. 472 00:29:20,080 --> 00:29:27,300 So this is the little nifty notation 473 00:29:27,300 --> 00:29:29,450 that says make it cyclic. 474 00:29:29,450 --> 00:29:34,860 Bring it back so that l only has the values here, 475 00:29:34,860 --> 00:29:38,750 zero up to N-1. 476 00:29:38,750 --> 00:29:41,410 And then stops. 477 00:29:41,410 --> 00:29:45,070 OK. 478 00:29:45,070 --> 00:29:50,830 OK, so we'll have more practice with examples 479 00:29:50,830 --> 00:29:56,410 when we do some filtering. 480 00:29:56,410 --> 00:29:58,770 Have you got that fundamental-- So we've 481 00:29:58,770 --> 00:30:03,490 talked about this rule one here. f times g 482 00:30:03,490 --> 00:30:07,530 goes to those coefficients. 483 00:30:07,530 --> 00:30:10,160 And if it's the cyclic case then I 484 00:30:10,160 --> 00:30:12,460 put a circle around that star. 485 00:30:12,460 --> 00:30:16,450 And I do the wraparound. 486 00:30:16,450 --> 00:30:23,750 But it's just, it's the Z-transform, 487 00:30:23,750 --> 00:30:28,590 it's polynomials in z or polynomials in w, 488 00:30:28,590 --> 00:30:32,360 and when it's polynomials in w, you use that special property 489 00:30:32,360 --> 00:30:34,700 that w^N is one. 490 00:30:34,700 --> 00:30:38,150 Yeah, OK. 491 00:30:38,150 --> 00:30:43,420 Now, I see I've written another line, there. 492 00:30:43,420 --> 00:30:46,390 That I could convolve functions. 493 00:30:46,390 --> 00:30:48,520 Let me do a couple more examples. 494 00:30:48,520 --> 00:30:50,390 Couple of examples. 495 00:30:50,390 --> 00:30:59,200 First, before I go to that line, OK. 496 00:30:59,200 --> 00:31:05,520 So I'm up to this line. 497 00:31:05,520 --> 00:31:12,640 A couple of examples here. 498 00:31:12,640 --> 00:31:18,840 Let's see, what example would I want to do? 499 00:31:18,840 --> 00:31:28,270 Let's see, OK, I want to do one example with a delta function. 500 00:31:28,270 --> 00:31:31,080 One example with a delta function. 501 00:31:31,080 --> 00:31:32,870 One example with the delta vector. 502 00:31:32,870 --> 00:31:41,890 Yeah, let me take the function g(x) identically one. 503 00:31:41,890 --> 00:31:45,820 OK, constant function. 504 00:31:45,820 --> 00:31:48,000 In this rule. 505 00:31:48,000 --> 00:31:50,300 I want to see what happens with the rule. 506 00:31:50,300 --> 00:32:00,140 OK, then f(x) g(x) is the same as f(x), right? 507 00:32:00,140 --> 00:32:03,770 Because this function g(x) is so simple, it's just one. 508 00:32:03,770 --> 00:32:09,520 Now, what about the coefficient? 509 00:32:09,520 --> 00:32:13,480 So I have the coefficients of c, what are the Fourier 510 00:32:13,480 --> 00:32:15,370 coefficients, what are the d's? 511 00:32:15,370 --> 00:32:18,900 Ah yes, what are the d's? 512 00:32:18,900 --> 00:32:22,840 So I'm testing my rule on a really really simple case, 513 00:32:22,840 --> 00:32:25,600 g(x) identically one. 514 00:32:25,600 --> 00:32:27,970 What, you have to tell me, in order 515 00:32:27,970 --> 00:32:30,490 to check the right side of the rule 516 00:32:30,490 --> 00:32:34,440 you have to tell me the Fourier coefficients for that very 517 00:32:34,440 --> 00:32:35,530 special function. 518 00:32:35,530 --> 00:32:38,270 What would be the Fourier coefficients? 519 00:32:38,270 --> 00:32:42,260 If I expand the function one in a Fourier series, 520 00:32:42,260 --> 00:32:44,000 what do I see? 521 00:32:44,000 --> 00:32:44,890 I see a one. 522 00:32:44,890 --> 00:32:47,240 Yeah, that's it, I see one. 523 00:32:47,240 --> 00:32:54,570 So what are its coefficients? d_0, right, is one? 524 00:32:54,570 --> 00:33:01,610 And the other d's are all zero, right? 525 00:33:01,610 --> 00:33:07,580 So my vector of d, my vector of d's is a whole lot 526 00:33:07,580 --> 00:33:09,600 of zeroes on the negative side. 527 00:33:09,600 --> 00:33:12,780 A one right there in the center, and then a lot of zeroes. 528 00:33:12,780 --> 00:33:24,770 And now I want to convolve that with c. 529 00:33:24,770 --> 00:33:27,450 I'm practicing the convolution rule 530 00:33:27,450 --> 00:33:34,590 on a case that's so simple it's confusing, right? 531 00:33:34,590 --> 00:33:45,570 I mean, it's a big mess, this multiplication. 532 00:33:45,570 --> 00:33:47,070 What do I get out of this? 533 00:33:47,070 --> 00:33:58,280 If d is this vector, if d has this property that d_0 is one 534 00:33:58,280 --> 00:34:03,450 and others are zero, all others are zero, 535 00:34:03,450 --> 00:34:06,250 so this is my little example, what 536 00:34:06,250 --> 00:34:10,940 does this sum boil down to? 537 00:34:10,940 --> 00:34:17,900 Well, I only get something when l=k, right? 538 00:34:17,900 --> 00:34:22,760 I only got something when l=k, because then I have d_0 539 00:34:22,760 --> 00:34:25,560 and that's the only d that's around. 540 00:34:25,560 --> 00:34:29,430 So in this sum, something happens only 541 00:34:29,430 --> 00:34:31,970 when k and l are the same. 542 00:34:31,970 --> 00:34:33,160 And then what happens? 543 00:34:33,160 --> 00:34:37,190 Then I have a one, I have c_l, and that's h_l, 544 00:34:37,190 --> 00:34:39,280 so that's all I'm concluding then. 545 00:34:39,280 --> 00:34:43,210 That this h is the same as c. 546 00:34:43,210 --> 00:34:48,210 I'm sorry, it's so dumb. 547 00:34:48,210 --> 00:34:51,570 My point is that in convolution, this is 548 00:34:51,570 --> 00:34:54,450 the thing that acts like one. 549 00:34:54,450 --> 00:34:56,480 Because in multiplication, that's 550 00:34:56,480 --> 00:34:59,380 the thing, that's the function that acts like one. 551 00:34:59,380 --> 00:35:01,890 That's the function that is one. 552 00:35:01,890 --> 00:35:11,640 So this is the one in-- Oh, would you allow me to do this? 553 00:35:11,640 --> 00:35:18,820 I'm going to create a matrix with these d's. 554 00:35:18,820 --> 00:35:24,560 There's another way to see convolution. 555 00:35:24,560 --> 00:35:26,680 Yeah, there's another way to see convolution 556 00:35:26,680 --> 00:35:28,490 and discrete convolution. 557 00:35:28,490 --> 00:35:31,090 Maybe the discrete one's the better. 558 00:35:31,090 --> 00:35:36,860 Yeah can you stand one more way to write the formula? 559 00:35:36,860 --> 00:35:40,720 One more way to write, now I'm going to do, 560 00:35:40,720 --> 00:35:47,940 I'm going to do discrete convolution. 561 00:35:47,940 --> 00:35:51,720 Discrete cyclic. 562 00:35:51,720 --> 00:35:53,010 So how am I going to write it? 563 00:35:53,010 --> 00:35:57,210 I'm going to write it by a matrix multiplication. 564 00:35:57,210 --> 00:35:58,905 Because you know that in this course 565 00:35:58,905 --> 00:36:01,190 a matrix was going to show up. 566 00:36:01,190 --> 00:36:06,290 So it's going to be a matrix multiplication. 567 00:36:06,290 --> 00:36:08,780 So I just have to tell you the matrix, 568 00:36:08,780 --> 00:36:12,300 so this is going to be some matrix. 569 00:36:12,300 --> 00:36:15,020 Let me take N to be four. 570 00:36:15,020 --> 00:36:20,240 So then I have, you watch. 571 00:36:20,240 --> 00:36:32,080 So I have four d's, and the output is the four h's. 572 00:36:32,080 --> 00:36:37,350 And the rule I'm following is this rule. 573 00:36:37,350 --> 00:36:43,660 Is this, the same old rule but with the cyclic part. 574 00:36:43,660 --> 00:36:47,070 And now I want to show you the matrix that'll just do this. 575 00:36:47,070 --> 00:36:54,120 Look, I've put the c's in the first column. 576 00:36:54,120 --> 00:37:00,340 And then I go, yeah, here's another. 577 00:37:00,340 --> 00:37:07,380 So it's a cyclic matrix. 578 00:37:07,380 --> 00:37:09,230 So let me finish it up. 579 00:37:09,230 --> 00:37:13,240 It's going to be four by four, it's going to be cyclic. 580 00:37:13,240 --> 00:37:18,340 So I have a c_0, c_0, c_0, c_0 on the diagonal. 581 00:37:18,340 --> 00:37:21,315 That's fine, that's because z to the zero 582 00:37:21,315 --> 00:37:24,740 is multiplying all the d's and leaving them in place. 583 00:37:24,740 --> 00:37:28,200 And then I have c_1's, and then I 584 00:37:28,200 --> 00:37:31,520 think I come around again here for a c_1. 585 00:37:31,520 --> 00:37:38,750 And I have c_2's, see where see c_3, c_2. 586 00:37:38,750 --> 00:37:43,510 And I come around again, to a c_2 and a c_2. 587 00:37:43,510 --> 00:37:49,490 And c_3 comes around to a c_3, a c_3 and a c_3. 588 00:37:49,490 --> 00:37:59,400 Well, can you, I hope you can see, this is cyclic matrix. 589 00:37:59,400 --> 00:38:05,150 It's only got one, it starts with a vector c, 590 00:38:05,150 --> 00:38:10,640 and those are on the diagonal and the diagonals wrap around. 591 00:38:10,640 --> 00:38:12,350 That's the other word that you often 592 00:38:12,350 --> 00:38:16,140 see when you see the word cyclic, wraparound. 593 00:38:16,140 --> 00:38:23,760 It's because you think of a circle. 594 00:38:23,760 --> 00:38:25,640 If you go a second time around, it's 595 00:38:25,640 --> 00:38:28,530 wrapped around the first time. 596 00:38:28,530 --> 00:38:32,000 OK, just can you look and see that this is the right formula 597 00:38:32,000 --> 00:38:36,300 for h_0? h_0 is c_0*d_0. 598 00:38:36,300 --> 00:38:37,950 Where does that come from? 599 00:38:37,950 --> 00:38:43,230 Remember, h_0 is the coefficient of z^0 in the answer. 600 00:38:43,230 --> 00:38:47,980 So it comes from c_0*d_0 to the zeroth power in the input. 601 00:38:47,980 --> 00:38:49,930 And then why is there is a c_3*d_1? 602 00:38:49,930 --> 00:38:52,960 603 00:38:52,960 --> 00:38:58,450 Why is there a c_3*d_1, and then a c_2*d_2 and then a c_1*d_3 604 00:38:58,450 --> 00:39:02,630 all piling up into h_0? 605 00:39:02,630 --> 00:39:04,380 Tell me now, why is there a c_3*d_1? 606 00:39:04,380 --> 00:39:07,800 607 00:39:07,800 --> 00:39:10,370 Because we're doing mod four is one way to say it. 608 00:39:10,370 --> 00:39:12,840 Three and one add to four. 609 00:39:12,840 --> 00:39:20,750 Because c_3 is the w cubed guy, and d_1 is the coefficient 610 00:39:20,750 --> 00:39:30,790 of w^1 and w^3 times w^1 piles back into the constant. 611 00:39:30,790 --> 00:39:33,840 And you see the pattern of that matrix? 612 00:39:33,840 --> 00:39:38,420 So these matrices are very important. 613 00:39:38,420 --> 00:39:40,620 So they circle around. 614 00:39:40,620 --> 00:39:45,570 Oh, we've actually met a matrix of this type, 615 00:39:45,570 --> 00:39:49,920 the first day of 18.085. 616 00:39:49,920 --> 00:39:51,160 What was that matrix? 617 00:39:51,160 --> 00:39:53,760 It was one of our four great matrices. 618 00:39:53,760 --> 00:39:56,650 And now here it is back again. 619 00:39:56,650 --> 00:40:00,700 Which one was it? 620 00:40:00,700 --> 00:40:03,430 Well, you remember the letter for it. 621 00:40:03,430 --> 00:40:05,170 Which isn't going to change. 622 00:40:05,170 --> 00:40:11,170 And do you remember the particular matrix? 623 00:40:11,170 --> 00:40:14,320 Well, everybody does remember that matrix, right? 624 00:40:14,320 --> 00:40:19,510 Twos were on the diagonal, minus ones were on the diagonal, 625 00:40:19,510 --> 00:40:22,180 and the diagonal curve continued. 626 00:40:22,180 --> 00:40:26,510 Minus one was on this diagonal and that continued and zeroes 627 00:40:26,510 --> 00:40:28,120 was on this diagonal. 628 00:40:28,120 --> 00:40:35,610 So this is cyclic convolution, the circulant matrix, 629 00:40:35,610 --> 00:40:42,050 cyclic convolution by c, what's the c 630 00:40:42,050 --> 00:40:46,920 that produces that convolution matrix? 631 00:40:46,920 --> 00:40:50,720 It's just, it's got-- well there it is. 632 00:40:50,720 --> 00:40:56,380 The first column is it. 633 00:40:56,380 --> 00:40:57,350 Right, right. 634 00:40:57,350 --> 00:41:09,530 And somehow I would say that that's an even vector. 635 00:41:09,530 --> 00:41:12,190 It's sort of, I associate it with cosine. 636 00:41:12,190 --> 00:41:13,300 It's an even vector. 637 00:41:13,300 --> 00:41:18,810 Here is the zero term, and then these are the same, 638 00:41:18,810 --> 00:41:21,030 not to worry about that part. 639 00:41:21,030 --> 00:41:25,210 Do you see that we've seen that matrix before? 640 00:41:25,210 --> 00:41:28,440 And the cyclic convolution means you 641 00:41:28,440 --> 00:41:32,400 take its second differences, of course. 642 00:41:32,400 --> 00:41:37,750 We're taking second differences, but everything in our world 643 00:41:37,750 --> 00:41:40,260 is cyclic. 644 00:41:40,260 --> 00:41:47,210 So the result, the x_4 is x_0. 645 00:41:47,210 --> 00:41:49,990 So we're taking second differences -- well, 646 00:41:49,990 --> 00:41:55,990 maybe I should say d -- we're taking second differences d_i, 647 00:41:55,990 --> 00:42:02,260 2d_i's, minus d_(i-1) minus d_(i+1). 648 00:42:02,260 --> 00:42:05,720 I don't know if this is-- So there's a minus one, 649 00:42:05,720 --> 00:42:07,070 two, minus one. 650 00:42:07,070 --> 00:42:11,760 And we're cycling around so that d_0 is d_4. 651 00:42:11,760 --> 00:42:20,180 And d_1 is d_5, and d_(-1) is d_3, whatever. 652 00:42:20,180 --> 00:42:25,140 OK, I'm just reminding you, we've seen these before. 653 00:42:25,140 --> 00:42:33,280 OK, so this is another way to remember the formula. 654 00:42:33,280 --> 00:42:37,950 OK now, can I ask you a practical question? 655 00:42:37,950 --> 00:42:39,910 A practical question. 656 00:42:39,910 --> 00:42:47,610 Let me bring back this second grade multiplication. 657 00:42:47,610 --> 00:42:49,810 Well, I have a granddaughter named Elizabeth, 658 00:42:49,810 --> 00:42:51,230 I'll have to admit I didn't think 659 00:42:51,230 --> 00:42:52,590 about mentioning Elizabeth. 660 00:42:52,590 --> 00:42:54,270 She's six. 661 00:42:54,270 --> 00:42:59,710 And she delights in sending me long multiplications. 662 00:42:59,710 --> 00:43:02,570 I mean, really long. 663 00:43:02,570 --> 00:43:04,570 And then every time I talk to her on the phone, 664 00:43:04,570 --> 00:43:06,330 she says have you done that one yet? 665 00:43:06,330 --> 00:43:07,880 And I say, I'm working on it. 666 00:43:07,880 --> 00:43:11,340 I've got MATLAB at work. 667 00:43:11,340 --> 00:43:15,260 Because they're ridiculous and I haven't figured out 668 00:43:15,260 --> 00:43:15,970 how to tell her. 669 00:43:15,970 --> 00:43:20,230 I mean, she just writes page after page. 670 00:43:20,230 --> 00:43:23,240 Times 100, plus three, minus seven, 671 00:43:23,240 --> 00:43:25,510 just whatever she things of. 672 00:43:25,510 --> 00:43:31,130 OK, now I need help from the convolution rule. here, 673 00:43:31,130 --> 00:43:31,960 actually. 674 00:43:31,960 --> 00:43:34,360 So let's suppose that Elizabeth has given me 675 00:43:34,360 --> 00:43:38,070 a multiplication in which I have a thousand 676 00:43:38,070 --> 00:43:41,470 digits times a thousand, right. 677 00:43:41,470 --> 00:43:44,880 Which Mathematica is prepared to do exactly, right? 678 00:43:44,880 --> 00:43:51,100 MATLAB will mess up, but Mathematica and Maple 679 00:43:51,100 --> 00:43:55,360 and symbolic packages will do exact computations. 680 00:43:55,360 --> 00:43:59,770 So what would be the right way, well let may make it 1,024. 681 00:43:59,770 --> 00:44:08,220 1,024 digits times 1,024 digits. 682 00:44:08,220 --> 00:44:11,260 Let's do the cyclic version first. 683 00:44:11,260 --> 00:44:13,340 Elizabeth doesn't know about cyclic. 684 00:44:13,340 --> 00:44:14,740 Maybe I could teach her that. 685 00:44:14,740 --> 00:44:18,650 That'll keep her busy while I'm doing the multiplication. 686 00:44:18,650 --> 00:44:23,040 OK, right. 687 00:44:23,040 --> 00:44:25,060 Only, her older brother would explain it 688 00:44:25,060 --> 00:44:26,320 to her, that's the trouble. 689 00:44:26,320 --> 00:44:30,080 OK, so how am I going to do, or how are you 690 00:44:30,080 --> 00:44:32,810 going to do on the quiz, multiplication 691 00:44:32,810 --> 00:44:37,070 of a 1,024 digits times 1,024? 692 00:44:37,070 --> 00:44:39,880 And I'll make it easy by making it cyclic, 693 00:44:39,880 --> 00:44:43,310 so I just want 1,024 digits in the answer. 694 00:44:43,310 --> 00:44:45,420 OK. 695 00:44:45,420 --> 00:44:48,390 How would you do it? 696 00:44:48,390 --> 00:44:55,930 Well, before today, you would have just multiplied, right? 697 00:44:55,930 --> 00:45:00,730 You would have written down 1,024, two lines of 1,024, 698 00:45:00,730 --> 00:45:03,490 done an addition. 699 00:45:03,490 --> 00:45:08,090 And you would have had a million multiplications to do. 700 00:45:08,090 --> 00:45:10,360 But how would you do it now? 701 00:45:10,360 --> 00:45:14,680 Apart from giving it to Mathematica. 702 00:45:14,680 --> 00:45:17,760 What's a faster way to do it? 703 00:45:17,760 --> 00:45:21,450 What's a faster way to do a convolution? 704 00:45:21,450 --> 00:45:23,460 The fast way to do a convolution is 705 00:45:23,460 --> 00:45:27,300 to use the convolution rule, go this way. 706 00:45:27,300 --> 00:45:32,000 So take these numbers, these 1,024 numbers, 707 00:45:32,000 --> 00:45:35,930 in c and these 1,024 numbers in d, 708 00:45:35,930 --> 00:45:40,250 and, well what do I have to do? 709 00:45:40,250 --> 00:45:41,820 I want to use the convolution rule, 710 00:45:41,820 --> 00:45:47,190 because multiplying is fast. 711 00:45:47,190 --> 00:45:50,200 Now I've got functions. 712 00:45:50,200 --> 00:45:52,110 But I'm in the cyclic case. 713 00:45:52,110 --> 00:45:58,520 So I'm in the cyclic case, so what should I do? 714 00:45:58,520 --> 00:46:03,020 How can I change this to be the cyclic case? 715 00:46:03,020 --> 00:46:08,830 This is like f_j g_j. 716 00:46:08,830 --> 00:46:15,690 So multiplication of components of things in function space 717 00:46:15,690 --> 00:46:18,170 is convolution of coefficients. 718 00:46:18,170 --> 00:46:22,130 So now, this is the cyclic. 719 00:46:22,130 --> 00:46:25,820 So let me make it cyclic. 720 00:46:25,820 --> 00:46:28,320 So again, what's your problem? 721 00:46:28,320 --> 00:46:31,740 The problem is to do this cyclic multiplication. 722 00:46:31,740 --> 00:46:33,440 What's the idea? 723 00:46:33,440 --> 00:46:41,270 The idea is to transform c back to f, to transform d back to g. 724 00:46:41,270 --> 00:46:43,510 Do the multiplications, now I have 725 00:46:43,510 --> 00:46:45,610 only 1,024 multiplications. 726 00:46:45,610 --> 00:46:47,690 Not 1,024 squared. 727 00:46:47,690 --> 00:46:49,350 That's the point. 728 00:46:49,350 --> 00:46:52,710 And if I do this directly, I've got 1,024 729 00:46:52,710 --> 00:46:55,380 squared multiplications to do. 730 00:46:55,380 --> 00:46:57,230 Much better. 731 00:46:57,230 --> 00:47:01,160 Transform back to here, do just 1,024-- 732 00:47:01,160 --> 00:47:04,430 what's the MATLAB command for that, when you're multiplying 733 00:47:04,430 --> 00:47:06,630 each component by itself? 734 00:47:06,630 --> 00:47:09,730 It's not the dot product, notice. 735 00:47:09,730 --> 00:47:13,300 It's not the dot product because I'm not summing. 736 00:47:13,300 --> 00:47:14,920 Do you know the MATLAB command, if I 737 00:47:14,920 --> 00:47:18,950 have a sequence of numbers of numbers, a vector 738 00:47:18,950 --> 00:47:25,520 f of length 1,024, and I want to get that result? 739 00:47:25,520 --> 00:47:26,940 What's the result? 740 00:47:26,940 --> 00:47:32,450 It's a vector of length 1,024 that takes each f times its g. 741 00:47:32,450 --> 00:47:34,600 But doesn't do any adds. 742 00:47:34,600 --> 00:47:36,120 That's what's there. 743 00:47:36,120 --> 00:47:38,880 What's the MATLAB command for that? 744 00:47:38,880 --> 00:47:40,370 Dot, yeah. 745 00:47:40,370 --> 00:47:42,620 Dot star, right. 746 00:47:42,620 --> 00:47:45,680 So that dot says component by component. 747 00:47:45,680 --> 00:47:48,980 OK, so what's the plan here? 748 00:47:48,980 --> 00:47:53,100 I do c's back to f. 749 00:47:53,100 --> 00:48:00,740 By the Fourier matrix. d back to g, by the Fourier matrix, 750 00:48:00,740 --> 00:48:04,470 then I do a very quick multiplication. 751 00:48:04,470 --> 00:48:05,520 And then what? 752 00:48:05,520 --> 00:48:07,440 Then I mustn't forget. 753 00:48:07,440 --> 00:48:11,830 That I'm in frequency space, and what do I have to do? 754 00:48:11,830 --> 00:48:16,680 I've got to get back into coefficient space. 755 00:48:16,680 --> 00:48:24,160 So I do the inverse transform of-- Here's the formula, then. 756 00:48:24,160 --> 00:48:32,550 I'm doing the inverse transform of, so the transform of c 757 00:48:32,550 --> 00:48:39,930 dot star, the transform of d. 758 00:48:39,930 --> 00:48:44,690 To get c, d. 759 00:48:44,690 --> 00:48:47,560 Is that right? 760 00:48:47,560 --> 00:48:51,630 So I took c, and I got back into the function. 761 00:48:51,630 --> 00:48:54,210 I took d, and got back to its function, 762 00:48:54,210 --> 00:48:55,920 with the Fourier matrix. 763 00:48:55,920 --> 00:48:59,740 OK, I'm in the Fourier and now I'm in this space. 764 00:48:59,740 --> 00:49:03,000 I've added up coefficients to get in this space. 765 00:49:03,000 --> 00:49:06,050 Now I do the dot star, the fast one. 766 00:49:06,050 --> 00:49:08,390 And then I transform back. 767 00:49:08,390 --> 00:49:11,850 So why is that faster? 768 00:49:11,850 --> 00:49:14,960 Than just doing it? 769 00:49:14,960 --> 00:49:20,850 Because what's the cost of F times c? 770 00:49:20,850 --> 00:49:23,180 And how am I going to do that? 771 00:49:23,180 --> 00:49:25,730 I'm going to do with the fast Fourier transform, right. 772 00:49:25,730 --> 00:49:26,650 That's the point. 773 00:49:26,650 --> 00:49:32,300 I can multiply by F, or by F inverse, faster. 774 00:49:32,300 --> 00:49:36,120 So I have three of these transforms. 775 00:49:36,120 --> 00:49:39,800 I've got to get two guys into the other space, 776 00:49:39,800 --> 00:49:42,480 and the answer back out. 777 00:49:42,480 --> 00:49:45,460 So I have sort of three of these N 778 00:49:45,460 --> 00:49:53,190 log N's but that will easily beat N squared. 779 00:49:53,190 --> 00:49:53,690 Right? 780 00:49:53,690 --> 00:50:01,560 So if you have a convolution to do, 781 00:50:01,560 --> 00:50:05,950 and it's possible to do this, get into the other space 782 00:50:05,950 --> 00:50:09,380 where it's just an element by element multiplication. 783 00:50:09,380 --> 00:50:11,710 And that would apply in either direction. 784 00:50:11,710 --> 00:50:13,910 Because the rule goes both ways. 785 00:50:13,910 --> 00:50:17,240 If I have this convolution to do, 786 00:50:17,240 --> 00:50:19,980 I would find the coefficients here, 787 00:50:19,980 --> 00:50:24,200 the c's, the coefficients of d of the g's. 788 00:50:24,200 --> 00:50:27,470 I would do this one by one multiplication, 789 00:50:27,470 --> 00:50:31,200 and then I have the Fourier coefficients 790 00:50:31,200 --> 00:50:32,860 of the convolution. 791 00:50:32,860 --> 00:50:35,770 Right, OK? 792 00:50:35,770 --> 00:50:37,110 Do I have a moment? 793 00:50:37,110 --> 00:50:37,870 Well, hardly. 794 00:50:37,870 --> 00:50:39,830 Just, can I write down what the formula 795 00:50:39,830 --> 00:50:44,380 for a convolution of two functions would look like? 796 00:50:44,380 --> 00:50:48,380 Sorry, f(x) convolved with g(x). 797 00:50:48,380 --> 00:50:50,610 Let me make it cyclic. 798 00:50:50,610 --> 00:50:53,300 Just to see what it would look like. 799 00:50:53,300 --> 00:50:56,140 What am I expecting for that convolution? 800 00:50:56,140 --> 00:50:58,570 I'm expecting a function, and somehow there's 801 00:50:58,570 --> 00:51:02,450 going to be an integral instead of a sum, where I had sums, 802 00:51:02,450 --> 00:51:04,140 but now I have integrals. 803 00:51:04,140 --> 00:51:05,890 And here's the point. 804 00:51:05,890 --> 00:51:13,000 I'll have f(t) times g(x-t) dt. 805 00:51:13,000 --> 00:51:16,180 806 00:51:16,180 --> 00:51:20,160 All I'm asking you to look at is the fact that the way here I 807 00:51:20,160 --> 00:51:23,480 had k, and l-k. 808 00:51:23,480 --> 00:51:28,190 For functions, your eye sees that right away as a t, 809 00:51:28,190 --> 00:51:30,420 and an (x-t), dt. 810 00:51:30,420 --> 00:51:33,290 These add to the answer. 811 00:51:33,290 --> 00:51:36,880 These add to the result, that x. 812 00:51:36,880 --> 00:51:39,670 That would be the cyclical one, yeah. 813 00:51:39,670 --> 00:51:41,930 So I could go zero-- These are all 814 00:51:41,930 --> 00:51:46,780 periodic functions, so all 2pi periods are the same. 815 00:51:46,780 --> 00:51:50,250 The book will do that properly. 816 00:51:50,250 --> 00:51:54,440 OK, we've got the filtering to discuss on Monday. 817 00:51:54,440 --> 00:51:59,030 You can see that this convolution stuff just 818 00:51:59,030 --> 00:52:03,220 takes a little new thinking, but it comes out nicely.