1 00:00:00,090 --> 00:00:02,490 The following content is provided under a Creative 2 00:00:02,490 --> 00:00:04,030 Commons license. 3 00:00:04,030 --> 00:00:06,330 Your support will help MIT OpenCourseWare 4 00:00:06,330 --> 00:00:10,720 continue to offer high quality educational resources for free. 5 00:00:10,720 --> 00:00:13,320 To make a donation or view additional materials 6 00:00:13,320 --> 00:00:17,280 from hundreds of MIT courses, visit MIT OpenCourseWare 7 00:00:17,280 --> 00:00:22,970 at ocw.mit.edu 8 00:00:22,970 --> 00:00:26,150 PROFESSOR: Hello and welcome. 9 00:00:26,150 --> 00:00:30,710 So today is mostly distinguished by what happens tomorrow. 10 00:00:30,710 --> 00:00:33,650 Not surprisingly, but of course, you all know that. 11 00:00:33,650 --> 00:00:37,100 So tomorrow we have our first quiz. 12 00:00:37,100 --> 00:00:40,407 Tomorrow evening 7:30 to 9:30, no recitation. 13 00:00:40,407 --> 00:00:41,990 We've been through this several times. 14 00:00:41,990 --> 00:00:45,320 I won't spend any time on it other than to ask if there are 15 00:00:45,320 --> 00:00:49,100 any questions, so if I don't hear any questions, 16 00:00:49,100 --> 00:00:51,260 the idea is going to be at the end of this lecture, 17 00:00:51,260 --> 00:00:54,470 the next time we'll see you is in office hours, 18 00:00:54,470 --> 00:01:00,260 and, or tomorrow, Wednesday, 7:30 on the third floor, 19 00:01:00,260 --> 00:01:01,070 building 26. 20 00:01:03,690 --> 00:01:06,380 Questions or comments about the exam? 21 00:01:06,380 --> 00:01:07,050 Yep. 22 00:01:07,050 --> 00:01:11,650 AUDIENCE: [INAUDIBLE] 23 00:01:11,650 --> 00:01:14,910 PROFESSOR: So that one page of notes, 8 and 1/2 24 00:01:14,910 --> 00:01:17,680 by 11, front and back. 25 00:01:17,680 --> 00:01:20,790 You can write as small as you like. 26 00:01:20,790 --> 00:01:22,780 In fact, later in today's lecture, 27 00:01:22,780 --> 00:01:24,631 I'll show you how a microscope works 28 00:01:24,631 --> 00:01:26,880 and you're welcome to use a microscope because they're 29 00:01:26,880 --> 00:01:29,370 completely non-electronic. 30 00:01:29,370 --> 00:01:33,780 Oh, as long as you use an optical microscope. 31 00:01:33,780 --> 00:01:35,150 Other questions about the exam? 32 00:01:38,290 --> 00:01:44,020 OK, then for today, so far, since the beginning 33 00:01:44,020 --> 00:01:46,090 of the term, we thought about a number 34 00:01:46,090 --> 00:01:47,710 of different kinds of representations 35 00:01:47,710 --> 00:01:49,540 for both DT systems-- 36 00:01:49,540 --> 00:01:53,110 discrete time-- and CT systems-- continuous time systems-- 37 00:01:53,110 --> 00:01:55,660 and we saw that we were interested 38 00:01:55,660 --> 00:01:57,670 in that large number of representations, 39 00:01:57,670 --> 00:02:01,720 because each of them had some particular aspect that 40 00:02:01,720 --> 00:02:04,630 made it particularly convenient sometimes. 41 00:02:04,630 --> 00:02:08,229 So, for example, in both CT and DT we looked at verbal, 42 00:02:08,229 --> 00:02:09,100 but not so much. 43 00:02:09,100 --> 00:02:11,500 That was mostly in the homework. 44 00:02:11,500 --> 00:02:14,380 We looked at difference in differential equations, 45 00:02:14,380 --> 00:02:19,660 mostly because they were so compact, so concise, so 46 00:02:19,660 --> 00:02:22,320 precise, they told you exactly what the system does. 47 00:02:22,320 --> 00:02:25,630 No fluff, this is it. 48 00:02:25,630 --> 00:02:27,040 So that was nice. 49 00:02:27,040 --> 00:02:31,226 Block diagrams, by contrast, are less concise, 50 00:02:31,226 --> 00:02:33,100 but they tell you the way a signal propagates 51 00:02:33,100 --> 00:02:36,560 through the system on its way from the input to the output, 52 00:02:36,560 --> 00:02:38,620 and that can be very helpful for understanding 53 00:02:38,620 --> 00:02:41,500 why certain behaviors occur, especially when we talk 54 00:02:41,500 --> 00:02:43,750 about things like feedback. 55 00:02:43,750 --> 00:02:45,340 We looked at operator representations. 56 00:02:45,340 --> 00:02:47,950 They were nice because we could transform 57 00:02:47,950 --> 00:02:50,980 the way we think about systems into the way 58 00:02:50,980 --> 00:02:55,522 we think about polynomials, so we reduced a college level 59 00:02:55,522 --> 00:02:56,980 thing to a high school level thing. 60 00:02:56,980 --> 00:03:00,820 That's always nice, and then we looked at transforms. 61 00:03:00,820 --> 00:03:03,670 The distinguishing feature of transform representation 62 00:03:03,670 --> 00:03:06,550 was that we took an entire function of time, 63 00:03:06,550 --> 00:03:08,890 and turned it into an algebraic expression. 64 00:03:08,890 --> 00:03:10,900 So we turned a differential system 65 00:03:10,900 --> 00:03:13,390 that described a system of differential equations 66 00:03:13,390 --> 00:03:16,480 that describes a system into a system of algebraic equations 67 00:03:16,480 --> 00:03:18,160 that describes a system. 68 00:03:18,160 --> 00:03:20,050 All of those were useful for different ways 69 00:03:20,050 --> 00:03:24,640 and what I wanted to talk about today is yet another way 70 00:03:24,640 --> 00:03:27,580 to represent a system and that is to represent 71 00:03:27,580 --> 00:03:30,180 a system by a single signal. 72 00:03:33,066 --> 00:03:34,690 So in some sense we're going backwards, 73 00:03:34,690 --> 00:03:36,850 because we're taking what we would normally 74 00:03:36,850 --> 00:03:41,429 think of as an entire system and reducing it, 75 00:03:41,429 --> 00:03:42,970 getting rid of the system altogether, 76 00:03:42,970 --> 00:03:44,830 so all we're going to have is signals. 77 00:03:44,830 --> 00:03:47,230 That turns out to be a particular powerful way 78 00:03:47,230 --> 00:03:50,110 to do some sorts of operations, and is actually 79 00:03:50,110 --> 00:03:52,240 the first instance, the first step, 80 00:03:52,240 --> 00:03:55,630 in that we will take in a major field 81 00:03:55,630 --> 00:03:57,820 thought of signal processing. 82 00:03:57,820 --> 00:04:00,670 When you reduce the entire behavior of a system 83 00:04:00,670 --> 00:04:03,970 to a signal, we then regard the whole processing task 84 00:04:03,970 --> 00:04:05,320 as a signal processing task. 85 00:04:05,320 --> 00:04:07,750 Signal processing task, not system. 86 00:04:10,930 --> 00:04:13,750 So far, we have focused on the responses 87 00:04:13,750 --> 00:04:16,000 to the most elementary kinds of signals-- 88 00:04:16,000 --> 00:04:22,120 unit sample signal, unit impulse signal-- 89 00:04:22,120 --> 00:04:24,700 but generally speaking, we're interested in much more 90 00:04:24,700 --> 00:04:25,720 complicated signals. 91 00:04:25,720 --> 00:04:27,760 As you've already seen, I've already 92 00:04:27,760 --> 00:04:30,477 asked you to calculate things like unit step responses. 93 00:04:30,477 --> 00:04:32,560 So generally, we're going to be interested in much 94 00:04:32,560 --> 00:04:34,780 more complicated signals-- 95 00:04:34,780 --> 00:04:38,800 the responses of systems to much more complicated signals. 96 00:04:38,800 --> 00:04:40,930 That's not hard. 97 00:04:40,930 --> 00:04:42,760 The reason we skipped over it was 98 00:04:42,760 --> 00:04:46,030 that you can always figure out the response 99 00:04:46,030 --> 00:04:49,210 to a more complicated signal at least 100 00:04:49,210 --> 00:04:51,520 by falling back on some of our more primitive ways 101 00:04:51,520 --> 00:04:52,829 of thinking about systems. 102 00:04:52,829 --> 00:04:55,120 So for example, if we think about a difference equation 103 00:04:55,120 --> 00:04:57,580 or a block diagram, we can think about how 104 00:04:57,580 --> 00:05:03,430 a more complicated signal excites a response by simply 105 00:05:03,430 --> 00:05:05,970 thinking about the system operating on a sample 106 00:05:05,970 --> 00:05:08,200 by sample basis. 107 00:05:08,200 --> 00:05:09,650 So you, of course, all know that, 108 00:05:09,650 --> 00:05:11,150 and to prove that you all know that, 109 00:05:11,150 --> 00:05:13,190 answer the following question. 110 00:05:13,190 --> 00:05:14,215 Here is a system. 111 00:05:16,790 --> 00:05:19,700 Here is a signal that is more complicated than just a unit 112 00:05:19,700 --> 00:05:22,220 sample or unit sample signal. 113 00:05:22,220 --> 00:05:26,780 Figure out what is the third response 114 00:05:26,780 --> 00:05:29,280 of this system to that signal? 115 00:05:37,400 --> 00:05:40,310 You should be absolutely quiet. 116 00:05:40,310 --> 00:05:43,180 That was sarcasm, just so you know. 117 00:06:24,620 --> 00:06:28,950 Lots of self-satisfied looks so I assume everybody's done. 118 00:06:28,950 --> 00:06:29,870 So what's the answer? 119 00:06:29,870 --> 00:06:32,000 Raise the number of fingers that corresponds 120 00:06:32,000 --> 00:06:35,136 to the answer y of 3. 121 00:06:35,136 --> 00:06:36,760 Raise your hand so that I can see them. 122 00:06:39,390 --> 00:06:40,940 Drop the ones with the wrong answers. 123 00:06:40,940 --> 00:06:43,730 I don't want to see those. 124 00:06:43,730 --> 00:06:46,430 OK, about 50%, so maybe take another 10 seconds. 125 00:06:50,580 --> 00:06:53,105 Notice that I'm asking for y of 3. 126 00:07:27,490 --> 00:07:30,970 So what's the answer y of 3, and raise your hand. 127 00:07:30,970 --> 00:07:34,670 Everybody has the right answer, raise your hand. 128 00:07:34,670 --> 00:07:35,170 Much better. 129 00:07:35,170 --> 00:07:38,590 OK, so now the overwhelming majority says the answer is 2. 130 00:07:38,590 --> 00:07:40,930 That's not very hard. 131 00:07:40,930 --> 00:07:43,906 If we think about the block diagram representation, 132 00:07:43,906 --> 00:07:45,280 and if we think about propagating 133 00:07:45,280 --> 00:07:47,740 the more complicated signal represented over here 134 00:07:47,740 --> 00:07:51,280 through that signal, through that system, 135 00:07:51,280 --> 00:07:52,840 we start with the system at rest. 136 00:07:52,840 --> 00:07:58,140 That means there's 0's coming out of all the delays, 137 00:07:58,140 --> 00:08:03,390 at time n equals minus 1, x is 0, 138 00:08:03,390 --> 00:08:07,580 combined with the initial 0's, we get the answer is 0. 139 00:08:10,430 --> 00:08:17,270 And then, at time n equals 0, the input becomes 1, 140 00:08:17,270 --> 00:08:24,290 so now the output goes to 1, at time 1, it goes to 2. 141 00:08:24,290 --> 00:08:26,350 At time 2, it goes to 3. 142 00:08:26,350 --> 00:08:28,770 Times 3, it goes to 2. 143 00:08:28,770 --> 00:08:30,930 This is obvious, right? 144 00:08:30,930 --> 00:08:32,580 Et cetera. 145 00:08:32,580 --> 00:08:36,419 So the answer is 2. 146 00:08:36,419 --> 00:08:41,280 y of 3 is 2, and the point is that it's very trivial 147 00:08:41,280 --> 00:08:44,144 to think about by thinking about the system in a sort of sample 148 00:08:44,144 --> 00:08:47,140 by sample way. 149 00:08:47,140 --> 00:08:49,080 Not surprisingly, the point of today 150 00:08:49,080 --> 00:08:53,940 is to not think of the system sample by sample, 151 00:08:53,940 --> 00:08:58,710 but to elevate the conversation from samples to signals. 152 00:08:58,710 --> 00:09:01,770 The first step in thinking about it as signals 153 00:09:01,770 --> 00:09:04,350 is to realize that you can think about the response 154 00:09:04,350 --> 00:09:08,850 of the system by decomposing the input into additive parts. 155 00:09:08,850 --> 00:09:13,740 I can think about x which is this 3 sample signal. 156 00:09:13,740 --> 00:09:17,730 I can decompose it into single samples, 157 00:09:17,730 --> 00:09:22,130 and then think about the response to each of those, 158 00:09:22,130 --> 00:09:25,440 and it may not surprise you that if the system were linear, 159 00:09:25,440 --> 00:09:27,020 then the response to the sum would 160 00:09:27,020 --> 00:09:29,516 be the sum of the responses. 161 00:09:29,516 --> 00:09:30,890 So you can sort of see that there 162 00:09:30,890 --> 00:09:32,390 would be a way of adding together 163 00:09:32,390 --> 00:09:35,280 these rectangular pulses to get a triangle pulse. 164 00:09:38,180 --> 00:09:42,930 So that works simply because the system is linear. 165 00:09:42,930 --> 00:09:47,540 The system has the property that the output for a sum 166 00:09:47,540 --> 00:09:52,720 is the sum of the outputs for the individual components, 167 00:09:52,720 --> 00:09:56,500 and we can write that this way, so a system is linear. 168 00:09:56,500 --> 00:10:00,130 We can define linear in a more rigorous mathematical sense 169 00:10:00,130 --> 00:10:02,050 by saying that a system is linear 170 00:10:02,050 --> 00:10:04,840 if the response to a weighted sum of inputs 171 00:10:04,840 --> 00:10:08,200 is the similarly weighted sum of outputs. 172 00:10:08,200 --> 00:10:11,650 So imagine that I have a system whose 173 00:10:11,650 --> 00:10:16,680 output when the input is x1 is y1 174 00:10:16,680 --> 00:10:19,970 and whose output when the input is x2 is y2. 175 00:10:19,970 --> 00:10:21,900 We'll say the system is linear if 176 00:10:21,900 --> 00:10:27,480 and only if the weighted sum of inputs alpha x1 plus beta x2 177 00:10:27,480 --> 00:10:31,620 gives the same wakings alpha y1 plus beta y2 178 00:10:31,620 --> 00:10:34,650 for all possible values of alpha and beta. 179 00:10:34,650 --> 00:10:36,960 If that's true, then we'll say the system is linear. 180 00:10:41,810 --> 00:10:44,930 So then if it's linear, then we're 181 00:10:44,930 --> 00:10:46,467 allowed to do this decomposition, 182 00:10:46,467 --> 00:10:48,050 because all we're doing is decomposing 183 00:10:48,050 --> 00:10:49,820 the input into a sum of inputs. 184 00:10:52,370 --> 00:10:56,292 We'll always be able to do that operation of the decomposition 185 00:10:56,292 --> 00:10:58,500 if the system is linear according to the definition I 186 00:10:58,500 --> 00:10:59,480 already showed. 187 00:10:59,480 --> 00:11:01,220 If the system also has the property 188 00:11:01,220 --> 00:11:04,010 that we will call time invariance, 189 00:11:04,010 --> 00:11:05,810 then the response to the parts will be 190 00:11:05,810 --> 00:11:08,330 particularly easy to calculate. 191 00:11:08,330 --> 00:11:12,170 Time invariance has a similar formal definition. 192 00:11:12,170 --> 00:11:16,420 We will call a system time invariant 193 00:11:16,420 --> 00:11:18,670 if given that the input-- 194 00:11:18,670 --> 00:11:23,200 the response to x of n is y. 195 00:11:23,200 --> 00:11:26,800 Given that, we'll say the system is time invariant 196 00:11:26,800 --> 00:11:28,480 if a shifted version of the input 197 00:11:28,480 --> 00:11:32,770 simply shifts the response. 198 00:11:32,770 --> 00:11:35,750 That seems like a kind of gobbledygook sort of thing. 199 00:11:35,750 --> 00:11:39,010 It's a very simple minded notion that you should all 200 00:11:39,010 --> 00:11:40,330 have from common experience. 201 00:11:40,330 --> 00:11:43,990 All it says is that if I do something today, 202 00:11:43,990 --> 00:11:50,430 and I get a response, if I do the same thing tomorrow, 203 00:11:50,430 --> 00:11:54,210 I should get the same response just delayed by a day. 204 00:11:54,210 --> 00:11:55,200 That's all it's saying. 205 00:11:55,200 --> 00:11:58,350 So basically the system behaves sort of the same 206 00:11:58,350 --> 00:12:02,380 now as it did previously, and as it will in the future. 207 00:12:02,380 --> 00:12:05,580 That's what time invariance means. 208 00:12:05,580 --> 00:12:08,560 So if the response is time invariant, 209 00:12:08,560 --> 00:12:13,920 then we can compute the response to a shifted unit sample. 210 00:12:13,920 --> 00:12:16,750 Notice that this is the unit sample response. 211 00:12:16,750 --> 00:12:19,650 This is a shifted unit sample so if the system 212 00:12:19,650 --> 00:12:22,080 is time invariant, then the response to a shifted unit 213 00:12:22,080 --> 00:12:25,290 sample is the shifted unit sample response which 214 00:12:25,290 --> 00:12:27,990 you can see from the picture. 215 00:12:27,990 --> 00:12:30,570 So the idea then is that superposition 216 00:12:30,570 --> 00:12:34,470 is a very easy way to think about the response of a system. 217 00:12:34,470 --> 00:12:37,200 If the system is linear and time invariant, 218 00:12:37,200 --> 00:12:40,290 linearity let's us break it up, and think about the response 219 00:12:40,290 --> 00:12:41,580 to each part. 220 00:12:41,580 --> 00:12:43,200 Shift invariance, or time invariance, 221 00:12:43,200 --> 00:12:45,930 allows us to shift the input, and know automatically 222 00:12:45,930 --> 00:12:48,180 what's the response going to look like. 223 00:12:50,810 --> 00:12:54,360 And there's a formal way we can think about the way you 224 00:12:54,360 --> 00:12:56,970 do that operation. 225 00:12:56,970 --> 00:13:01,470 How do you implement this superposition thing? 226 00:13:01,470 --> 00:13:05,650 You think about a system as having a unit sample response. 227 00:13:05,650 --> 00:13:07,200 This is the unit sample signal. 228 00:13:07,200 --> 00:13:12,430 We'll call the unit sample response h of n, 229 00:13:12,430 --> 00:13:16,090 then a shifted unit sample will give a shifted unit 230 00:13:16,090 --> 00:13:17,130 sample response. 231 00:13:17,130 --> 00:13:18,340 That's time invariance. 232 00:13:20,950 --> 00:13:28,590 Then a weighted shifted impulse gives the same weighted shifted 233 00:13:28,590 --> 00:13:29,400 impulse response. 234 00:13:32,170 --> 00:13:42,260 Then the sum of such things gives the sum of such things. 235 00:13:42,260 --> 00:13:46,100 That's just a formal derivation of a process 236 00:13:46,100 --> 00:13:48,990 that we will call convolution. 237 00:13:48,990 --> 00:13:54,290 So the response to an arbitrary DT signal 238 00:13:54,290 --> 00:13:58,460 that excites a linear time invariant system 239 00:13:58,460 --> 00:14:02,660 can be described by the convolution of the input 240 00:14:02,660 --> 00:14:06,680 with the unit sample response. 241 00:14:06,680 --> 00:14:08,510 We'll call that formula-- 242 00:14:08,510 --> 00:14:12,590 we'll call that operation convolution. 243 00:14:12,590 --> 00:14:16,847 Convolution is completely straightforward. 244 00:14:16,847 --> 00:14:19,430 For that reason, we try to make it a little bit more confusing 245 00:14:19,430 --> 00:14:22,610 by using terribly confusing notation. 246 00:14:22,610 --> 00:14:24,140 That too is sarcastic, I mean. 247 00:14:28,140 --> 00:14:30,440 So the only thing that's at all confusing 248 00:14:30,440 --> 00:14:34,310 about convolution-- convolution is completely trivial. 249 00:14:34,310 --> 00:14:38,270 Here's the way we would write it. x convolves with h. 250 00:14:38,270 --> 00:14:41,995 The signal x convolves with the signal h to give a new signal. 251 00:14:44,560 --> 00:14:50,030 Being a signal, I can ask what's the nth sample look like, 252 00:14:50,030 --> 00:14:54,920 and what that symbol means is this sum. 253 00:14:54,920 --> 00:14:59,630 The confusing thing is that most people in the field 254 00:14:59,630 --> 00:15:03,190 write it this way. 255 00:15:03,190 --> 00:15:07,430 The signal x of n convolves with the signal h of n. 256 00:15:07,430 --> 00:15:11,690 The reason that's confusing, and the thing you will never do, 257 00:15:11,690 --> 00:15:13,592 because you are here. 258 00:15:13,592 --> 00:15:15,050 The thing that you will never do is 259 00:15:15,050 --> 00:15:18,830 confuse the meaning of that statement 260 00:15:18,830 --> 00:15:21,613 with what looks like an operation on samples. 261 00:15:24,650 --> 00:15:29,720 Had I said multiply, you would have said, 262 00:15:29,720 --> 00:15:32,420 if this were a multiply operator instead of a convolution 263 00:15:32,420 --> 00:15:35,270 multiplier operator-- 264 00:15:35,270 --> 00:15:38,000 if that were multiply instead of convolve, 265 00:15:38,000 --> 00:15:41,360 you would have said, oh, that means 266 00:15:41,360 --> 00:15:47,200 the oneth sample multiplied by the oneth sample 267 00:15:47,200 --> 00:15:48,940 is the product of the oneth samples. 268 00:15:52,180 --> 00:15:54,220 This is not true. 269 00:15:54,220 --> 00:15:55,990 This is not generally true. 270 00:15:58,720 --> 00:16:04,270 The convolution operation means take the whole signal x. 271 00:16:04,270 --> 00:16:07,030 That's why we think about it as an operation on signals, 272 00:16:07,030 --> 00:16:10,390 not an operation on samples. 273 00:16:10,390 --> 00:16:13,990 Convolution means take the whole signal x, 274 00:16:13,990 --> 00:16:16,870 and convolve it with the whole signal h 275 00:16:16,870 --> 00:16:21,820 to get a brand new signal x convolved h, 276 00:16:21,820 --> 00:16:25,400 and then take the nth sample. 277 00:16:25,400 --> 00:16:28,720 So the only thing that's at all confusing about convolution 278 00:16:28,720 --> 00:16:32,140 is remembering the convolution is an operation that is applied 279 00:16:32,140 --> 00:16:33,580 to signals, not samples. 280 00:16:40,160 --> 00:16:41,987 So structure of convolution. 281 00:16:41,987 --> 00:16:43,820 So I just showed you a mathematical formula. 282 00:16:43,820 --> 00:16:45,530 What I'd like you to have is a little bit 283 00:16:45,530 --> 00:16:50,270 of an intuition for what happens when you convolve two signals. 284 00:16:50,270 --> 00:16:52,930 So let's think about the structure of this operation. 285 00:16:52,930 --> 00:16:54,740 What are we doing? 286 00:16:54,740 --> 00:16:57,467 Imagine that we're going back to that original problem. 287 00:16:57,467 --> 00:17:00,050 What happens when you take x of n and convolve it with h of n? 288 00:17:03,110 --> 00:17:05,900 All we need to do is this formula, right? 289 00:17:05,900 --> 00:17:08,000 That's all we need to do. 290 00:17:08,000 --> 00:17:09,560 What's that formula say? 291 00:17:09,560 --> 00:17:15,060 Well let's think about how you would compute the 0-th output. 292 00:17:15,060 --> 00:17:20,280 According to that formula all I did was substitute n equal 0 293 00:17:20,280 --> 00:17:23,430 every place there was an n, and what 294 00:17:23,430 --> 00:17:27,240 I see is I have to multiply x of k times h of minus k, 295 00:17:27,240 --> 00:17:29,430 but I've got x of n in h of n. 296 00:17:29,430 --> 00:17:31,980 So the first thing I do is I flip the axises and I make them 297 00:17:31,980 --> 00:17:32,610 k's. 298 00:17:32,610 --> 00:17:34,260 That's not hard. 299 00:17:34,260 --> 00:17:39,570 Then the x looks OK, but the h doesn't. 300 00:17:39,570 --> 00:17:45,800 I need h of minus k, so I have to flip it. 301 00:17:45,800 --> 00:17:49,165 So I'm flipping about the n equals 0 axis. 302 00:17:52,120 --> 00:17:55,870 So that positive n becomes minus n, 303 00:17:55,870 --> 00:17:57,880 positive k becomes minus k, because I want 304 00:17:57,880 --> 00:18:01,200 this to be minus k up here. 305 00:18:01,200 --> 00:18:05,780 Then generally, I have some shift thing here. 306 00:18:05,780 --> 00:18:08,801 This 0, because I was looking for the 0-th sample. 307 00:18:08,801 --> 00:18:10,300 In general, that might be different. 308 00:18:10,300 --> 00:18:14,770 That might be 7 if I wanted to find y of 7. 309 00:18:14,770 --> 00:18:21,530 Then I'd have a 7 over here, and that number represents a shift. 310 00:18:21,530 --> 00:18:23,430 In the case of 0, it's a 0 shift. 311 00:18:26,630 --> 00:18:29,860 Then I have to multiply these two, 312 00:18:29,860 --> 00:18:32,380 so I just place this thing over here so I can multiply. 313 00:18:32,380 --> 00:18:34,510 I multiply down. 314 00:18:34,510 --> 00:18:39,400 You can see that there's only one sample that in the two 315 00:18:39,400 --> 00:18:42,280 is both non-zero. 316 00:18:42,280 --> 00:18:46,750 Therefore, I get a single non-zero answer, 317 00:18:46,750 --> 00:18:48,280 and then according to the formula, 318 00:18:48,280 --> 00:19:00,420 I have to sum so the 0-th answer is flip, shift, multiply, sum, 319 00:19:00,420 --> 00:19:02,800 and you just repeat that for all the different answers. 320 00:19:02,800 --> 00:19:06,390 So at time equals 0, the answer at time 0 321 00:19:06,390 --> 00:19:10,950 is flip, shift by 0, multiply, sum. 322 00:19:10,950 --> 00:19:13,030 The answer is 1. 323 00:19:13,030 --> 00:19:19,410 If I want to find the one answer now the shift is shift by 1. 324 00:19:19,410 --> 00:19:21,960 So now instead of having flip which 325 00:19:21,960 --> 00:19:25,590 would have put the 3 samples here, I shift by 1 326 00:19:25,590 --> 00:19:27,820 so now they're over there. 327 00:19:27,820 --> 00:19:31,500 So now when I do the multiply, I pick up 2 non-zero answers 328 00:19:31,500 --> 00:19:34,300 and the answer is 2. 329 00:19:34,300 --> 00:19:37,080 If I wanted y equals 2, I do the same thing 330 00:19:37,080 --> 00:19:39,840 but now I shift by 2. 331 00:19:39,840 --> 00:19:43,280 Flip, shift, multiply, sum, that's all I do. 332 00:19:43,280 --> 00:19:46,410 It's completely trivial. 333 00:19:46,410 --> 00:19:49,320 If I continue, the shift becomes larger 334 00:19:49,320 --> 00:19:52,950 and now it's falling off the end. 335 00:19:52,950 --> 00:19:57,540 Continue, continue, and I get in general 336 00:19:57,540 --> 00:20:00,460 that's the prescription. 337 00:20:00,460 --> 00:20:02,430 So what I've tried to show is two ways 338 00:20:02,430 --> 00:20:05,370 of thinking about this convolution thing. 339 00:20:05,370 --> 00:20:07,650 The first was by superposition, where I just 340 00:20:07,650 --> 00:20:11,364 think about breaking the input into a bunch of samples, 341 00:20:11,364 --> 00:20:13,530 thinking about the response to each of those samples 342 00:20:13,530 --> 00:20:14,029 and adding. 343 00:20:18,528 --> 00:20:21,570 That's an input centric way of thinking about things, 344 00:20:21,570 --> 00:20:23,820 because I think of the input being broken up 345 00:20:23,820 --> 00:20:25,590 by a bunch of samples. 346 00:20:25,590 --> 00:20:28,380 This convolution formula is an output centric way 347 00:20:28,380 --> 00:20:30,120 of thinking about things I tell you. 348 00:20:30,120 --> 00:20:35,450 I'd like to know the output at time p, 349 00:20:35,450 --> 00:20:37,760 and to compute the output of time p, 350 00:20:37,760 --> 00:20:39,290 you say, well that's easy. 351 00:20:39,290 --> 00:20:44,240 Flip, shift by p, multiply, sum. 352 00:20:44,240 --> 00:20:48,149 So input centric, that's the superposition way 353 00:20:48,149 --> 00:20:49,190 of thinking about things. 354 00:20:49,190 --> 00:20:51,080 Output centric, that's the convolution way 355 00:20:51,080 --> 00:20:53,550 of thinking about things. 356 00:20:53,550 --> 00:20:56,180 So now that you know about convolution, 357 00:20:56,180 --> 00:21:00,920 find which plot below 1, 2, 3, 4, or none of the above 358 00:21:00,920 --> 00:21:03,530 shows the result of convolving the two functions shown above. 359 00:21:26,554 --> 00:21:27,659 You're so quiet. 360 00:21:27,659 --> 00:21:29,700 I assume you're practicing for the exam tomorrow. 361 00:21:36,222 --> 00:21:37,180 You're allowed to talk. 362 00:22:58,510 --> 00:22:59,530 So which one's right? 363 00:22:59,530 --> 00:23:03,377 1, 2, 3, 4, or 5? 364 00:23:03,377 --> 00:23:03,960 See if I know. 365 00:23:08,790 --> 00:23:10,430 It looks good. 366 00:23:10,430 --> 00:23:18,530 About I only see one wrong, two wrong, so 95% or so. 367 00:23:18,530 --> 00:23:21,470 So how do I think about this? 368 00:23:21,470 --> 00:23:25,220 What's the way that I should think about convolving those? 369 00:23:25,220 --> 00:23:27,890 Easiest, most straightforward way, go back to the formula. 370 00:23:30,920 --> 00:23:31,890 That will always work. 371 00:23:34,440 --> 00:23:37,170 Can somebody tell me a more intuitive, insightful way 372 00:23:37,170 --> 00:23:40,350 of thinking about what will be the result of convolving 373 00:23:40,350 --> 00:23:42,060 those top two functions? 374 00:23:42,060 --> 00:23:44,758 Tell me a property of the result of convolving. 375 00:23:51,310 --> 00:23:54,190 Yes, yes, flip, shift, multiply, sum. 376 00:23:57,900 --> 00:23:59,860 What's the answer at n equals 1? 377 00:24:03,640 --> 00:24:04,990 1. 378 00:24:04,990 --> 00:24:09,850 So I've got two things that look kind of like geometrics. 379 00:24:09,850 --> 00:24:11,890 Imagine for the moment-- 380 00:24:11,890 --> 00:24:14,230 that was intended to be a hint-- 381 00:24:14,230 --> 00:24:17,800 imagine for the moment that the sequence looks like 1, 382 00:24:17,800 --> 00:24:25,180 2/3, 4/9, 8/27, blah, blah, blah. 383 00:24:25,180 --> 00:24:28,030 Imagine that it's a geometric sequence 384 00:24:28,030 --> 00:24:29,260 with the base of about 2/3. 385 00:24:32,870 --> 00:24:36,590 How would I compute the answer when I convolve that sequence 386 00:24:36,590 --> 00:24:40,300 with itself at zero? 387 00:24:40,300 --> 00:24:42,970 Flip, shift, multiply, divide, so I started out 388 00:24:42,970 --> 00:24:46,300 with two things that were both starting at zero. 389 00:24:46,300 --> 00:24:48,580 You flip one of them. 390 00:24:48,580 --> 00:24:51,130 How much overlap is there? 391 00:24:51,130 --> 00:24:52,690 Just the 1. 392 00:24:52,690 --> 00:24:55,660 Just the n equals 0, what's the answer at n equals 0? 393 00:24:59,580 --> 00:25:01,750 1, right? 394 00:25:01,750 --> 00:25:05,650 So if I imagine that this is the sequence after I flip it. 395 00:25:05,650 --> 00:25:07,750 There's a 1 under the 1. 396 00:25:07,750 --> 00:25:09,820 The 0's here kill the terms down here. 397 00:25:09,820 --> 00:25:12,010 The 0's here kill the terms up there. 398 00:25:12,010 --> 00:25:14,530 The only thing that lives is 1 times 1 is 1, 399 00:25:14,530 --> 00:25:16,570 so the answer at y equals 0 is 1. 400 00:25:16,570 --> 00:25:17,970 What's the answer at y equals 1? 401 00:25:22,470 --> 00:25:24,220 Flip, shift, multiply, sum. 402 00:25:29,820 --> 00:25:35,180 So I flip, shift. 403 00:25:35,180 --> 00:25:44,010 So when I shift, the new answer looks like 1, 2/3, 4/9, 8/27, 404 00:25:44,010 --> 00:25:46,130 et cetera. 405 00:25:46,130 --> 00:25:48,970 Multiply and sum, what's the answer? 406 00:25:48,970 --> 00:25:49,950 AUDIENCE: [INAUDIBLE] 407 00:25:49,950 --> 00:25:51,970 PROFESSOR: 4/3. 408 00:25:51,970 --> 00:25:57,095 So the only non-zero answers are 1 times 2/3 plus 2/3 times 1-- 409 00:25:57,095 --> 00:25:57,595 4/3. 410 00:26:00,360 --> 00:26:08,320 If I want to compute y of 0 1 2, I shift it further. 411 00:26:08,320 --> 00:26:14,810 So I do 1, 2/3, 4/9, 8/27, blah, blah, blah. 412 00:26:17,710 --> 00:26:20,220 So flip, shift, I shift 1 more, multiply, 413 00:26:20,220 --> 00:26:24,900 sum, multiply 1 times 4/9, and I get 4/9. 414 00:26:24,900 --> 00:26:29,160 Multiply 2/3 times 2/3, I get 4/9. 415 00:26:29,160 --> 00:26:32,920 Multiply 4/9 times 1, I get 4/9. 416 00:26:32,920 --> 00:26:36,670 The answer to the sum of those is 4/3. 417 00:26:36,670 --> 00:26:39,750 So I get one 4/3, 4/3. 418 00:26:39,750 --> 00:26:41,640 So you can see it's tracing out. 419 00:26:41,640 --> 00:26:44,340 This wave form so far, this is the only one that 420 00:26:44,340 --> 00:26:48,810 has up and then flat, and if I continue that process 421 00:26:48,810 --> 00:26:50,010 it will start to fall off. 422 00:26:53,250 --> 00:26:57,980 If you're exclusively mathematically minded, 423 00:26:57,980 --> 00:27:01,100 you can also just do it with math. 424 00:27:01,100 --> 00:27:03,800 All you do is think about a mathematical description 425 00:27:03,800 --> 00:27:06,430 of the left signal. 426 00:27:06,430 --> 00:27:12,440 Say 2/3 of the nu of n and the right signal, and now all 427 00:27:12,440 --> 00:27:15,810 I need to do is think about that formula. 428 00:27:15,810 --> 00:27:19,670 So do a sum, taking this, the function of n 429 00:27:19,670 --> 00:27:24,010 and turning it into a function of k. 430 00:27:24,010 --> 00:27:27,460 The second one, I want to make a function of n minus k. 431 00:27:27,460 --> 00:27:29,740 I have to shift both. 432 00:27:29,740 --> 00:27:34,090 I have to change the exponent as well as the index 433 00:27:34,090 --> 00:27:39,350 into the unit sample signal, same thing over here. 434 00:27:39,350 --> 00:27:42,910 Now when I think about multiplying them, u of k 435 00:27:42,910 --> 00:27:45,760 kills all the terms for k less than 0, 436 00:27:45,760 --> 00:27:50,660 so I can start at 0 instead of minus infinity. 437 00:27:50,660 --> 00:27:57,170 This u kills everything for which n minus k is less than 0. 438 00:27:57,170 --> 00:27:59,870 That means k less than n-- 439 00:27:59,870 --> 00:28:01,430 less than or equal to n. 440 00:28:01,430 --> 00:28:04,180 So I end up with this. 441 00:28:04,180 --> 00:28:06,180 This product is particularly easy 442 00:28:06,180 --> 00:28:10,060 because it's the the k into the minus k, 443 00:28:10,060 --> 00:28:15,590 so the answer is to the n, and now I'm summing over k, 444 00:28:15,590 --> 00:28:20,380 but there are no k's, so that's summing over 1. 445 00:28:20,380 --> 00:28:23,000 And so my answer is just n plus 1 and 2/3 446 00:28:23,000 --> 00:28:26,170 to the nu of n which is the same thing by thinking about it 447 00:28:26,170 --> 00:28:29,650 intuitively. 448 00:28:29,650 --> 00:28:36,060 So the point is that the operation is friendly, 449 00:28:36,060 --> 00:28:40,200 and so the idea then the big picture was convolution 450 00:28:40,200 --> 00:28:45,400 is a different way to represent a system. 451 00:28:45,400 --> 00:28:49,180 Using convolution, we represent an entire system 452 00:28:49,180 --> 00:28:53,120 by a single signal. 453 00:28:53,120 --> 00:28:55,670 That signal, the unit sample response, 454 00:28:55,670 --> 00:28:58,550 is sufficient to characterize the output of the system 455 00:28:58,550 --> 00:28:59,930 for any possible input. 456 00:28:59,930 --> 00:29:03,950 We just saw how the operation is called convolution, 457 00:29:03,950 --> 00:29:06,230 so that enables us the big picture. 458 00:29:06,230 --> 00:29:11,810 We've represented an entire system by a single signal, 459 00:29:11,810 --> 00:29:14,905 in this case h of n. 460 00:29:14,905 --> 00:29:16,030 That's what convolution is. 461 00:29:16,030 --> 00:29:18,710 It's a new representation. 462 00:29:18,710 --> 00:29:25,940 You can do exactly the same thing for a CT system, 463 00:29:25,940 --> 00:29:29,760 and the reason we use delta to represent the unit sample 464 00:29:29,760 --> 00:29:31,950 signal and the unit impulse response. 465 00:29:31,950 --> 00:29:35,590 The unit impulse signal is clear, 466 00:29:35,590 --> 00:29:41,350 because the representation of an arbitrary signal, 467 00:29:41,350 --> 00:29:45,480 in terms of delta functions, looks much the same 468 00:29:45,480 --> 00:29:49,450 in CT and in DT. 469 00:29:49,450 --> 00:29:53,200 You can get there by thinking about the limiting argument 470 00:29:53,200 --> 00:29:56,020 for how to interpret the unit impulse. 471 00:29:56,020 --> 00:29:59,110 The unit impulse function was a function 472 00:29:59,110 --> 00:30:02,030 that is easiest to think about in a limit. 473 00:30:02,030 --> 00:30:05,500 Imagine that I have a signal that 474 00:30:05,500 --> 00:30:10,780 is a square pulse whose area, regardless of width, is 1. 475 00:30:10,780 --> 00:30:15,130 That's what a unit impulse function is. 476 00:30:15,130 --> 00:30:18,560 Imagine how you would construct an arbitrary signal x of t 477 00:30:18,560 --> 00:30:21,160 by having such a signal. 478 00:30:21,160 --> 00:30:25,920 You could take a signal and it's shifted version, and come up 479 00:30:25,920 --> 00:30:29,970 with a weighted sum of impulse functions 480 00:30:29,970 --> 00:30:33,900 or rectangular approximations to impulse functions 481 00:30:33,900 --> 00:30:36,900 to represent an arbitrary signal. 482 00:30:36,900 --> 00:30:40,950 If you did that, you would get an approximation to the signal 483 00:30:40,950 --> 00:30:44,130 x which could be written as a limit. 484 00:30:47,040 --> 00:30:51,080 So if you think about each of these being of width capital 485 00:30:51,080 --> 00:30:54,420 delta, then the height has to be 1 over delta. 486 00:30:54,420 --> 00:30:59,780 So the area remains one. 487 00:30:59,780 --> 00:31:02,690 Then if I want to build an arbitrary function x 488 00:31:02,690 --> 00:31:06,710 out of such signals, I need a sum of them, 489 00:31:06,710 --> 00:31:10,950 and each one of these p's has to be multiplied by delta. 490 00:31:10,950 --> 00:31:15,560 So that when I multiply by the value of x at one 491 00:31:15,560 --> 00:31:17,330 point k delta. 492 00:31:17,330 --> 00:31:22,860 I get the right height independent of what is delta. 493 00:31:22,860 --> 00:31:26,330 So for a given delta, I get a sum that looks like that, 494 00:31:26,330 --> 00:31:28,280 and then in keeping with the idea of thinking 495 00:31:28,280 --> 00:31:30,440 about a unit impulse function as a limit, 496 00:31:30,440 --> 00:31:32,720 I take the limit of that. 497 00:31:32,720 --> 00:31:34,580 The result is a function that looks 498 00:31:34,580 --> 00:31:40,160 very much like the decomposition of a signal in terms 499 00:31:40,160 --> 00:31:43,070 of the unit sample. 500 00:31:43,070 --> 00:31:48,200 In the previous case, we sum together a weighted version 501 00:31:48,200 --> 00:31:51,050 of a unit sample signal. 502 00:31:51,050 --> 00:31:53,630 Here the sum is replaced by an integral, 503 00:31:53,630 --> 00:31:55,425 and is weighted just like it was before. 504 00:31:58,690 --> 00:32:04,180 The point is that the mathematics for CT and DT 505 00:32:04,180 --> 00:32:06,220 look very similar. 506 00:32:06,220 --> 00:32:11,950 I decompose in the case of the CT, and arbitrary signal x of t 507 00:32:11,950 --> 00:32:19,670 into an entire row of weighted unit impulse functions. 508 00:32:19,670 --> 00:32:21,830 Once I have it in that form, the argument's 509 00:32:21,830 --> 00:32:25,820 precisely the same for CT as it was in DT. 510 00:32:25,820 --> 00:32:28,940 Imagine that I have a linear time invariant system. 511 00:32:28,940 --> 00:32:33,530 Linear means that I can compute the response to a sum 512 00:32:33,530 --> 00:32:36,860 as the sum of the responses. 513 00:32:36,860 --> 00:32:39,950 Time invariant means that shifting the input 514 00:32:39,950 --> 00:32:41,680 merely shifts the output. 515 00:32:41,680 --> 00:32:43,820 Doing the experiment tomorrow is the same 516 00:32:43,820 --> 00:32:49,010 as doing the experiment today, except it's now a day later. 517 00:32:49,010 --> 00:32:53,330 So if the response of a system is 518 00:32:53,330 --> 00:32:56,510 h of t when the input is delta of t, 519 00:32:56,510 --> 00:32:59,495 if the system is shift invariant, shifting this by tau 520 00:32:59,495 --> 00:33:01,095 is the same as shifting that by tau. 521 00:33:04,240 --> 00:33:10,970 A weighted sum of such things is a weighted sum of such things, 522 00:33:10,970 --> 00:33:14,780 and a sum of such things is the sum of such things, 523 00:33:14,780 --> 00:33:16,760 so I get an expression which we'll 524 00:33:16,760 --> 00:33:20,570 think of as convolution for CT that looks just the same. 525 00:33:23,290 --> 00:33:28,120 So in DT, we thought about if you convolve x with h, 526 00:33:28,120 --> 00:33:31,030 you take the first index, x of n, 527 00:33:31,030 --> 00:33:34,750 and turn it into x of k, a dummy variable. 528 00:33:34,750 --> 00:33:37,900 You take the second one, and do n minus k. 529 00:33:37,900 --> 00:33:42,540 Here we do the same thing. t goes to tau, 530 00:33:42,540 --> 00:33:46,158 and the second one goes to t minus tau. 531 00:33:46,158 --> 00:33:48,476 The sum up here turns into an integral. 532 00:33:48,476 --> 00:33:50,100 Otherwise, it's exactly the same thing. 533 00:33:52,750 --> 00:33:58,610 So to show your mastery of such things, what signal would 534 00:33:58,610 --> 00:34:02,930 result if you convolved e to the minus tu of t with e 535 00:34:02,930 --> 00:34:04,185 to the minus tu of t? 536 00:34:04,185 --> 00:34:05,840 1, 2, 3, 4, or none? 537 00:35:53,430 --> 00:35:55,260 Well, the place is quiet so I assume 538 00:35:55,260 --> 00:35:56,940 that means you stopped talking, so that 539 00:35:56,940 --> 00:36:01,400 means you've all agreed, yes? 540 00:36:01,400 --> 00:36:03,980 So which wave form best represents 541 00:36:03,980 --> 00:36:08,350 the convolution of the top two signals, 1, 2, 3, or 4? 542 00:36:14,150 --> 00:36:15,650 Almost 100% correct. 543 00:36:15,650 --> 00:36:16,830 Most people say 4. 544 00:36:16,830 --> 00:36:17,720 How do you get 4? 545 00:36:23,900 --> 00:36:25,040 Yeah? 546 00:36:25,040 --> 00:36:28,470 AUDIENCE: Same reason [INAUDIBLE] 547 00:36:28,470 --> 00:36:31,144 PROFESSOR: So what would I do? 548 00:36:31,144 --> 00:36:32,310 What would be my first step? 549 00:36:34,830 --> 00:36:37,530 So I imagine that I want to think 550 00:36:37,530 --> 00:36:44,490 of flip so this gets multiplied by the flip of the other one. 551 00:36:44,490 --> 00:36:48,900 So at time t equals 0, the answer is-- 552 00:36:48,900 --> 00:36:49,729 AUDIENCE: 0 553 00:36:49,729 --> 00:36:51,520 PROFESSOR: --0, because there's no overlap. 554 00:36:54,316 --> 00:36:55,720 AUDIENCE: [INAUDIBLE] 555 00:36:55,720 --> 00:36:57,730 PROFESSOR: OK so that it starts at 0, 556 00:36:57,730 --> 00:37:00,540 so that means that this is out. 557 00:37:00,540 --> 00:37:04,590 OK, OK, OK, fine. 558 00:37:04,590 --> 00:37:05,380 Now shift. 559 00:37:05,380 --> 00:37:06,330 What do I shift? 560 00:37:06,330 --> 00:37:09,484 Which one do I shift which way? 561 00:37:09,484 --> 00:37:13,510 AUDIENCE: Why is there a [INAUDIBLE] flipping is there 562 00:37:13,510 --> 00:37:15,760 a value that equals 0? 563 00:37:15,760 --> 00:37:17,940 PROFESSOR: That's a valid question. 564 00:37:17,940 --> 00:37:22,170 So the question is if I'm integrating 565 00:37:22,170 --> 00:37:26,220 over a function that goes from minus infinity to 0, and from 0 566 00:37:26,220 --> 00:37:27,220 to infinity. 567 00:37:27,220 --> 00:37:30,780 Let's say the answer right at 0 is 1. 568 00:37:30,780 --> 00:37:35,190 You could say that there is a single point 569 00:37:35,190 --> 00:37:38,250 whose value is non-zero. 570 00:37:38,250 --> 00:37:41,940 What would happen if I integrated a function that is 0 571 00:37:41,940 --> 00:37:44,460 everywhere except at a point? 572 00:37:44,460 --> 00:37:49,800 So it's 0 everywhere up to here, then a 0 everywhere after that, 573 00:37:49,800 --> 00:37:51,540 and at zero it's not zero. 574 00:37:51,540 --> 00:37:54,360 What's the integral of a function that differs 575 00:37:54,360 --> 00:37:57,090 from 0 at a single point? 576 00:37:57,090 --> 00:37:58,400 AUDIENCE: [INAUDIBLE] 577 00:37:58,400 --> 00:38:01,380 PROFESSOR: 0, it's a little bit of a trick question, 578 00:38:01,380 --> 00:38:04,290 because we will later have some functions for which that's not 579 00:38:04,290 --> 00:38:06,090 true. 580 00:38:06,090 --> 00:38:09,004 What kind of a function would that not be true for? 581 00:38:09,004 --> 00:38:10,812 AUDIENCE: [INAUDIBLE] 582 00:38:10,812 --> 00:38:12,620 PROFESSOR: Delta. 583 00:38:12,620 --> 00:38:19,870 If I were convolving delta with delta, 584 00:38:19,870 --> 00:38:25,560 then you can integrate over an infinitesimal area region, 585 00:38:25,560 --> 00:38:28,230 and get something that's not 0. 586 00:38:28,230 --> 00:38:30,810 So a little bit of a caveat, so as long 587 00:38:30,810 --> 00:38:33,570 as the function that I'm convolving 588 00:38:33,570 --> 00:38:36,540 doesn't have an impulse in it or worse. 589 00:38:36,540 --> 00:38:38,730 We will talk later in the course about things worse 590 00:38:38,730 --> 00:38:40,650 than impulses. 591 00:38:40,650 --> 00:38:46,110 If there's nothing as high as an impulse or worse, 592 00:38:46,110 --> 00:38:48,770 so that has a step in it. 593 00:38:48,770 --> 00:38:51,090 We would think of a step as a singularity that 594 00:38:51,090 --> 00:38:56,550 is better, less ill behaved than an impulse. 595 00:38:56,550 --> 00:39:02,220 As long as the function does not have an impulse or a worse, 596 00:39:02,220 --> 00:39:07,160 when you flip it, you'll get zero contribution at zero. 597 00:39:07,160 --> 00:39:09,780 That all makes sense? 598 00:39:09,780 --> 00:39:12,860 So this is zero at zero, but the reasoning is a little bit 599 00:39:12,860 --> 00:39:13,820 complicated. 600 00:39:13,820 --> 00:39:18,170 So now what do I get when t gets a little bit bigger than 0? 601 00:39:18,170 --> 00:39:21,680 What's the result of convolving when the time is 602 00:39:21,680 --> 00:39:25,870 slightly bigger than time 0? 603 00:39:25,870 --> 00:39:29,920 All flip, shift, multiply, integrate. 604 00:39:29,920 --> 00:39:32,610 So I have to shift one of those, so now 605 00:39:32,610 --> 00:39:36,570 instead of having this one, I might have shifted a little bit 606 00:39:36,570 --> 00:39:39,370 to the right. 607 00:39:39,370 --> 00:39:41,190 So it might look like that. 608 00:39:41,190 --> 00:39:44,636 So now what happens when I multiply? 609 00:39:44,636 --> 00:39:47,360 Well you don't get 0 anymore. 610 00:39:47,360 --> 00:39:52,280 So as for small t for t on the order of epsilon, 611 00:39:52,280 --> 00:39:57,510 how does the integral grow with t? 612 00:39:57,510 --> 00:40:01,421 So if I want to make a plot of the convolution, 613 00:40:01,421 --> 00:40:03,170 so if I want to think about e to the minus 614 00:40:03,170 --> 00:40:11,990 tu of t convolved with e to the minus tu of t versus t, 615 00:40:11,990 --> 00:40:15,800 I already know that that's like that for t small. 616 00:40:15,800 --> 00:40:16,940 How will the function grow? 617 00:40:20,040 --> 00:40:26,850 Linear so if this is very small then 618 00:40:26,850 --> 00:40:34,050 the deviations from the height which is 1 is very small. 619 00:40:34,050 --> 00:40:36,600 So if this distance is small, the deviation 620 00:40:36,600 --> 00:40:39,870 is that little triangle which goes like t-square. 621 00:40:39,870 --> 00:40:45,015 So for t small t-square is very small compared to t, 622 00:40:45,015 --> 00:40:48,960 I can ignore it, and so the function 623 00:40:48,960 --> 00:40:52,940 is going to start going up like t, 624 00:40:52,940 --> 00:40:56,720 and if you work out the details it will eventually roll off. 625 00:40:56,720 --> 00:41:00,830 Because as you shift it further and further, 626 00:41:00,830 --> 00:41:05,840 the one exponential is in the tail of the other exponential, 627 00:41:05,840 --> 00:41:08,610 so one of the exponentials kills the other one, 628 00:41:08,610 --> 00:41:10,110 and so the response goes to zero. 629 00:41:12,830 --> 00:41:15,066 If you're more mathematically inclined, yes? 630 00:41:15,066 --> 00:41:17,298 AUDIENCE: [INAUDIBLE] you were saying 631 00:41:17,298 --> 00:41:20,026 that if both functions are left sided [INAUDIBLE] 632 00:41:20,026 --> 00:41:25,494 right sided it always starts out t equals 0 is always 0. 633 00:41:25,494 --> 00:41:27,910 PROFESSOR: That's not quite right because right sided just 634 00:41:27,910 --> 00:41:31,520 means that the left is 0. 635 00:41:31,520 --> 00:41:34,940 So if I tell you that a signal is right sided, all I've said 636 00:41:34,940 --> 00:41:38,430 is that all of the non-zero values are on the right, 637 00:41:38,430 --> 00:41:42,330 but I haven't told you whether they're impulses or not. 638 00:41:42,330 --> 00:41:44,310 Right sided says something about the left. 639 00:41:44,310 --> 00:41:46,620 The left is zero. 640 00:41:46,620 --> 00:41:50,025 Kind of weird, so if I'm right handed, 641 00:41:50,025 --> 00:41:51,900 I might as well not have a left hand when I'm 642 00:41:51,900 --> 00:41:54,910 writing so the left is zero. 643 00:41:54,910 --> 00:41:58,200 So right sided signals have zeros. 644 00:41:58,200 --> 00:42:02,430 The signals on the left of right sided signals are zero, 645 00:42:02,430 --> 00:42:04,305 but I haven't told you what was on the right. 646 00:42:04,305 --> 00:42:06,300 The right could be an impulse or worse. 647 00:42:09,580 --> 00:42:11,770 So I just inferred some properties 648 00:42:11,770 --> 00:42:13,750 of what this convolution is going to look like. 649 00:42:13,750 --> 00:42:15,250 If you were mathematically inclined, 650 00:42:15,250 --> 00:42:17,450 you could do it by math. 651 00:42:17,450 --> 00:42:18,950 The math doesn't look very different 652 00:42:18,950 --> 00:42:23,000 from the math for the discrete version. 653 00:42:23,000 --> 00:42:27,950 You simply write this as a function of tau rather than t. 654 00:42:27,950 --> 00:42:31,100 This one as a function of t minus tau rather than t. 655 00:42:34,300 --> 00:42:37,240 Recognize that the u's cut off parts of the integral. 656 00:42:37,240 --> 00:42:42,040 This u is 1 only if t is bigger than 0. 657 00:42:42,040 --> 00:42:45,010 So that lops off the t less than 0 part. 658 00:42:45,010 --> 00:42:49,210 This lopped off the part bigger than capital, bigger than t. 659 00:42:49,210 --> 00:42:53,460 So that leaves the integrals 0 to t. 660 00:42:53,460 --> 00:42:55,680 Putting these two together results 661 00:42:55,680 --> 00:42:58,120 in the tau parts killing each other, 662 00:42:58,120 --> 00:43:01,500 so I'm left with only a t part but the integrals on tau. 663 00:43:01,500 --> 00:43:04,980 So just like the other one, the integral 664 00:43:04,980 --> 00:43:08,990 goes to the integral of 1 over finite limit 665 00:43:08,990 --> 00:43:11,910 0 to t, so the answer is t. 666 00:43:11,910 --> 00:43:15,180 So the overall answer is te to the minus tu of t 667 00:43:15,180 --> 00:43:16,170 which is plotted here. 668 00:43:18,840 --> 00:43:22,320 So the point of today then is that this 669 00:43:22,320 --> 00:43:27,370 is a different representation for the way systems work. 670 00:43:27,370 --> 00:43:29,900 It's often computationally interesting. 671 00:43:29,900 --> 00:43:30,820 This is the way. 672 00:43:30,820 --> 00:43:32,320 This is a perfectly plausible way 673 00:43:32,320 --> 00:43:35,740 of doing discrete time signal processing. 674 00:43:35,740 --> 00:43:40,780 Represent the system by a signal h of n, 675 00:43:40,780 --> 00:43:45,610 and then compute the response to any signal by convolving h of n 676 00:43:45,610 --> 00:43:47,410 with that signal. 677 00:43:47,410 --> 00:43:50,570 Perfectly reasonable way to compute things. 678 00:43:50,570 --> 00:43:52,930 Honestly, we'll find better ways of computing things 679 00:43:52,930 --> 00:43:55,820 by the end of the course. 680 00:43:55,820 --> 00:43:59,710 The real reason for studying convolution is conceptually, 681 00:43:59,710 --> 00:44:01,540 you can think about how a system ought 682 00:44:01,540 --> 00:44:04,349 to work by thinking about convolution, 683 00:44:04,349 --> 00:44:05,890 and I want to show an example of that 684 00:44:05,890 --> 00:44:09,430 by thinking about systems that I'm interested in. 685 00:44:09,430 --> 00:44:10,370 I do work on hearing. 686 00:44:10,370 --> 00:44:14,560 I do work with microscopes, and we can regard a microscope 687 00:44:14,560 --> 00:44:15,550 as an LTI system. 688 00:44:15,550 --> 00:44:18,700 A linear time invariant system, and convolution 689 00:44:18,700 --> 00:44:22,735 is a very good way of thinking about such optical systems. 690 00:44:25,300 --> 00:44:30,040 So the idea is that even the best microscope 691 00:44:30,040 --> 00:44:36,306 gives blurry images, and that's very fundamental physics. 692 00:44:36,306 --> 00:44:37,680 It has to do with the diffraction 693 00:44:37,680 --> 00:44:39,549 limit of optical systems. 694 00:44:39,549 --> 00:44:41,340 If you're interested in that sort of thing, 695 00:44:41,340 --> 00:44:44,970 take an optics course in the physics department 696 00:44:44,970 --> 00:44:48,730 or come to my lab and do a [INAUDIBLE].. 697 00:44:48,730 --> 00:44:51,760 So the idea is that even the best microscopes in the world 698 00:44:51,760 --> 00:44:52,260 are blurred. 699 00:44:52,260 --> 00:44:55,470 We have the best microscopes in the world in my lab, 700 00:44:55,470 --> 00:44:57,240 and they fundamentally blur things, 701 00:44:57,240 --> 00:44:59,760 and we have to worry about that. 702 00:44:59,760 --> 00:45:02,580 The blurring is inversely related 703 00:45:02,580 --> 00:45:05,070 to the numerical aperture which has to do 704 00:45:05,070 --> 00:45:06,870 with the size of the optic. 705 00:45:06,870 --> 00:45:09,890 Big optics are good. 706 00:45:09,890 --> 00:45:14,180 So if you imagine that a target emits 707 00:45:14,180 --> 00:45:16,900 a spherical wave of light. 708 00:45:16,900 --> 00:45:19,870 So every point on the target emits a spherical wave, 709 00:45:19,870 --> 00:45:22,915 then there's some optic that's collecting all of those waves, 710 00:45:22,915 --> 00:45:26,590 , and relaying them back to a different point. 711 00:45:26,590 --> 00:45:28,150 The resolution of the picture goes 712 00:45:28,150 --> 00:45:31,540 with how many of those rays the optic system picked up. 713 00:45:34,210 --> 00:45:37,630 So if you make the optics smaller, 714 00:45:37,630 --> 00:45:40,420 the picture becomes blurrier. 715 00:45:40,420 --> 00:45:42,400 If you make the optic even smaller, 716 00:45:42,400 --> 00:45:44,620 the picture becomes even smaller, 717 00:45:44,620 --> 00:45:49,800 and the way we think about that is by convolution. 718 00:45:49,800 --> 00:45:53,810 We think about the microscope as an LTI system, 719 00:45:53,810 --> 00:45:57,920 so we characterize it by its point spread function. 720 00:45:57,920 --> 00:45:59,484 We don't like to use any words that 721 00:45:59,484 --> 00:46:00,650 come from a different field. 722 00:46:00,650 --> 00:46:04,230 Just like every other field, we like to invent our own. 723 00:46:04,230 --> 00:46:09,050 So in optics, the thing that we will call a impulse response 724 00:46:09,050 --> 00:46:11,450 is called a point spread function. 725 00:46:11,450 --> 00:46:14,570 It just says if you had an ideal point of light, 726 00:46:14,570 --> 00:46:17,750 what would the image look like? 727 00:46:17,750 --> 00:46:19,370 It's exactly the same as convolving, 728 00:46:19,370 --> 00:46:22,670 so you can think of the blurry image as the convolution 729 00:46:22,670 --> 00:46:25,760 of the effect of the microscope, the point spread function, 730 00:46:25,760 --> 00:46:28,655 the impulse response with the ideal target. 731 00:46:31,310 --> 00:46:35,450 So then as you change the size of the optic, 732 00:46:35,450 --> 00:46:38,810 it changes the size of the impulse response, 733 00:46:38,810 --> 00:46:40,490 the point spread function. 734 00:46:40,490 --> 00:46:43,820 Crummy optics, fat point spread motions. 735 00:46:43,820 --> 00:46:48,410 Fat point spread functions, blurry pictures, 736 00:46:48,410 --> 00:46:51,290 and so here's a picture of how our system works. 737 00:46:51,290 --> 00:46:55,610 This is a representation to scale 738 00:46:55,610 --> 00:46:59,910 of a tiny microscopic bead a fraction of a micron. 739 00:46:59,910 --> 00:47:03,660 So it's about six times smaller than the image. 740 00:47:03,660 --> 00:47:06,960 So this is an image taken with our microscope system, 741 00:47:06,960 --> 00:47:08,990 and you can see that most of the energy 742 00:47:08,990 --> 00:47:12,620 fits inside a region about half a micron. 743 00:47:12,620 --> 00:47:15,470 World's best microscope, you can't do better 744 00:47:15,470 --> 00:47:17,780 than this by physics. 745 00:47:17,780 --> 00:47:21,110 This is using 500 nanometer light, and the size of that 746 00:47:21,110 --> 00:47:24,800 has to do with the length scale of the light. 747 00:47:24,800 --> 00:47:27,500 So you end up with this particular microscope 748 00:47:27,500 --> 00:47:33,530 not being able to make images with less blurring than that. 749 00:47:33,530 --> 00:47:35,637 That's the point spread function. 750 00:47:35,637 --> 00:47:37,970 Now of course, the point spread function of a microscope 751 00:47:37,970 --> 00:47:40,370 is three dimensional. 752 00:47:40,370 --> 00:47:44,240 In this class, we're only talking about 1-D time. 753 00:47:44,240 --> 00:47:48,620 In a microscope is 3D, x, y, and z. 754 00:47:48,620 --> 00:47:53,240 So the impulse response has extent in x, y, and z. 755 00:47:53,240 --> 00:47:55,870 So here is a picture taken by Anthony Patire, who 756 00:47:55,870 --> 00:48:01,040 was a student in my lab of that same tiny little dot. 757 00:48:01,040 --> 00:48:04,210 When the in-focus plane shows the dot, 758 00:48:04,210 --> 00:48:06,350 and as you go out of focus, the dot 759 00:48:06,350 --> 00:48:10,060 gets bigger, smearier, and blurrier. 760 00:48:10,060 --> 00:48:12,980 What you can do then is assemble those pictures that 761 00:48:12,980 --> 00:48:16,820 were taken one of the time into a 3-D volume, 762 00:48:16,820 --> 00:48:20,750 and that 3-D volume then represents the point spread 763 00:48:20,750 --> 00:48:22,490 function or the three dimensional impulse 764 00:48:22,490 --> 00:48:26,420 response of the microscope. 765 00:48:26,420 --> 00:48:29,630 So the idea then is that convolution is a very good way 766 00:48:29,630 --> 00:48:31,770 to think about optical systems because they 767 00:48:31,770 --> 00:48:34,820 are very easy to relate to the underlying physics. 768 00:48:34,820 --> 00:48:39,860 The blurring is a direct result of a fundamental property 769 00:48:39,860 --> 00:48:40,580 of light. 770 00:48:40,580 --> 00:48:43,490 The diffraction limit, and you can very accurately 771 00:48:43,490 --> 00:48:45,350 represent the effect of the blurring 772 00:48:45,350 --> 00:48:48,590 as convolving with a point spread function and impulse 773 00:48:48,590 --> 00:48:50,150 response. 774 00:48:50,150 --> 00:48:53,240 Same sort of thing applies to optics at any scale. 775 00:48:53,240 --> 00:48:56,270 So going from the microscopic to the rather macroscopic-- that 776 00:48:56,270 --> 00:48:59,330 is to say the universe and beyond. 777 00:48:59,330 --> 00:49:02,000 We can think about the Hubble Space Telescope. 778 00:49:02,000 --> 00:49:04,030 Same thing. 779 00:49:04,030 --> 00:49:08,030 Light, that's all that matters, and here the issue 780 00:49:08,030 --> 00:49:10,910 is-- the reason they wanted to make a Space Telescope 781 00:49:10,910 --> 00:49:13,520 is that there are two principal sources of blurring 782 00:49:13,520 --> 00:49:16,940 for a ground based telescope. 783 00:49:16,940 --> 00:49:19,340 One is the atmosphere blurring, because we're 784 00:49:19,340 --> 00:49:22,250 looking through an atmosphere and other is blurring because 785 00:49:22,250 --> 00:49:26,600 of the property of the optic elements in the telescope, 786 00:49:26,600 --> 00:49:29,630 and it turns out pretty easy to show 787 00:49:29,630 --> 00:49:33,410 that the combined effect of the atmosphere and the lenses 788 00:49:33,410 --> 00:49:36,650 is the convolution of the individual parts. 789 00:49:36,650 --> 00:49:39,350 You can think about atmospheric blurring 790 00:49:39,350 --> 00:49:41,510 as convolving with the atmosphere's point spread 791 00:49:41,510 --> 00:49:46,350 function, and you can think of the blurring 792 00:49:46,350 --> 00:49:50,690 due to the microscope as convolving with the blurring 793 00:49:50,690 --> 00:49:53,360 function due to optics. 794 00:49:53,360 --> 00:49:56,330 The combined is the convolution of those 795 00:49:56,330 --> 00:49:59,360 which means that if you've got some amount 796 00:49:59,360 --> 00:50:04,820 of atmospheric blurring and a telescope made out 797 00:50:04,820 --> 00:50:11,310 of 12 centimeter optics, then the combined responses showed 798 00:50:11,310 --> 00:50:16,360 here not very different from each individual. 799 00:50:16,360 --> 00:50:19,620 But if you made a big telescope-- 800 00:50:19,620 --> 00:50:21,870 a one meter type telescope-- you might be 801 00:50:21,870 --> 00:50:24,210 expecting this much blurring. 802 00:50:24,210 --> 00:50:27,240 But because of the atmosphere, you get much more blurring. 803 00:50:27,240 --> 00:50:30,600 So the deviation between the small telescope, 804 00:50:30,600 --> 00:50:33,780 and what you actually measure is not very different. 805 00:50:33,780 --> 00:50:35,970 The atmosphere makes an enormous difference 806 00:50:35,970 --> 00:50:40,500 when you start talking about a high resolution telescope. 807 00:50:40,500 --> 00:50:42,250 That's the reason for putting it in space. 808 00:50:42,250 --> 00:50:43,830 You get rid of the atmosphere. 809 00:50:47,340 --> 00:50:50,190 The Hubble Space Telescope was made principally 810 00:50:50,190 --> 00:50:53,160 out of two big mirrors, both were parabolic, 811 00:50:53,160 --> 00:50:56,490 both were highly optimized, both were enormous. 812 00:50:56,490 --> 00:51:00,390 This is the main lens. 813 00:51:00,390 --> 00:51:05,110 The lens is 2.4 meters, about eight feet in diameter, 814 00:51:05,110 --> 00:51:07,930 and it was astonishing the thing. 815 00:51:07,930 --> 00:51:12,470 So in order for a lens to work perfectly like I illustrated, 816 00:51:12,470 --> 00:51:14,400 it's important that every reflection 817 00:51:14,400 --> 00:51:18,610 remain in phase coherence. 818 00:51:18,610 --> 00:51:21,660 So the length that every ray travels 819 00:51:21,660 --> 00:51:25,200 has to be precisely matched to what it's supposed to be. 820 00:51:25,200 --> 00:51:29,910 In the case of the Hubble, they matched the surface. 821 00:51:29,910 --> 00:51:33,250 The surface was controlled to within 10 nanometers. 822 00:51:33,250 --> 00:51:35,280 That's absurd. 823 00:51:35,280 --> 00:51:40,540 So the blurring of my microscope was about half a micron. 824 00:51:40,540 --> 00:51:43,050 So 50 times worse. 825 00:51:43,050 --> 00:51:45,360 The best I could see with my microscope 826 00:51:45,360 --> 00:51:47,820 is 50 times worse than was required 827 00:51:47,820 --> 00:51:49,890 for making this mirror. 828 00:51:49,890 --> 00:51:53,220 It was absolutely astonishing feat to make the mirror 829 00:51:53,220 --> 00:51:56,530 and they made a mistake. 830 00:51:56,530 --> 00:51:58,170 So when they put this in space, they 831 00:51:58,170 --> 00:52:00,165 were expecting to see pictures like this. 832 00:52:00,165 --> 00:52:03,090 This is a picture of a distant star. 833 00:52:03,090 --> 00:52:05,820 They were expecting the distant star would look like this. 834 00:52:05,820 --> 00:52:07,320 This is what they actually measured, 835 00:52:07,320 --> 00:52:10,680 and the reason was that the feedback 836 00:52:10,680 --> 00:52:13,230 system that they used to grind the lens 837 00:52:13,230 --> 00:52:17,980 made a mistake by 2.2 microns. 838 00:52:17,980 --> 00:52:21,420 2.2 microns would have been just barely resolvable 839 00:52:21,420 --> 00:52:25,300 on my microscope, but barely. 840 00:52:25,300 --> 00:52:26,970 So the hair? 841 00:52:26,970 --> 00:52:30,360 That's about 100 microns in diameter. 842 00:52:30,360 --> 00:52:35,310 They were off by 2.2 microns, and because of that, 843 00:52:35,310 --> 00:52:38,560 it was like a complete disaster. 844 00:52:38,560 --> 00:52:43,420 So that small error was enough to make the images terrible. 845 00:52:43,420 --> 00:52:46,770 So the solution-- it wasn't very practical to ship up 846 00:52:46,770 --> 00:52:50,910 a new lens, so they shipped up eyeglasses. 847 00:52:50,910 --> 00:52:53,400 The eyeglasses was another transformation, 848 00:52:53,400 --> 00:52:55,530 just like your eyeglasses. 849 00:52:55,530 --> 00:52:59,190 Your eyeglasses work by changing the point spread function that 850 00:52:59,190 --> 00:53:02,100 is determined by your retina and your lens 851 00:53:02,100 --> 00:53:04,500 into a new point spread function. 852 00:53:04,500 --> 00:53:06,840 They shipped up eyeglasses, and the result 853 00:53:06,840 --> 00:53:09,300 of putting the eyeglasses into Hubble 854 00:53:09,300 --> 00:53:12,090 was to turn this into that, and to give 855 00:53:12,090 --> 00:53:15,820 some of the most dazzling pictures we've ever had. 856 00:53:15,820 --> 00:53:19,710 So the point is that convolution is a complete way 857 00:53:19,710 --> 00:53:21,220 of describing a system. 858 00:53:21,220 --> 00:53:24,690 It's a very intuitive way for certain kinds of systems, 859 00:53:24,690 --> 00:53:26,670 and it's especially useful for systems 860 00:53:26,670 --> 00:53:29,460 like light based systems where blurring 861 00:53:29,460 --> 00:53:33,930 is a natural way of thinking about the way the system works. 862 00:53:33,930 --> 00:53:34,890 Have a good time. 863 00:53:34,890 --> 00:53:37,820 See you tomorrow at 7:30.