1 00:00:00,090 --> 00:00:02,490 The following content is provided under a Creative 2 00:00:02,490 --> 00:00:04,030 Commons license. 3 00:00:04,030 --> 00:00:06,330 Your support will help MIT OpenCourseWare 4 00:00:06,330 --> 00:00:10,720 continue to offer high quality educational resources for free. 5 00:00:10,720 --> 00:00:13,320 To make a donation or view additional materials 6 00:00:13,320 --> 00:00:17,280 from hundreds of MIT courses, visit MIT OpenCourseWare 7 00:00:17,280 --> 00:00:18,450 at ocw.mit.edu. 8 00:00:21,980 --> 00:00:25,022 [MUSIC PLAYING] 9 00:00:45,340 --> 00:00:46,880 ALAN OPPENHEIM: In this lecture, we 10 00:00:46,880 --> 00:00:49,790 will begin the discussion of the computation 11 00:00:49,790 --> 00:00:53,160 of the discrete Fourier transform. 12 00:00:53,160 --> 00:00:55,940 This is an extremely important topic 13 00:00:55,940 --> 00:00:59,490 in this set of lectures for a number of reasons. 14 00:00:59,490 --> 00:01:03,800 One of the reasons is that the principal algorithm that we'll 15 00:01:03,800 --> 00:01:06,890 be discussing, the algorithm, which 16 00:01:06,890 --> 00:01:10,520 we'll refer to as the fast Fourier transform algorithm, 17 00:01:10,520 --> 00:01:12,980 relies, for its derivation, on many 18 00:01:12,980 --> 00:01:16,040 of the properties of the discrete Fourier transform. 19 00:01:16,040 --> 00:01:19,610 And so, in fact, developing this algorithm, 20 00:01:19,610 --> 00:01:23,240 provides us with an opportunity to exercise 21 00:01:23,240 --> 00:01:27,260 some of the intuition and some of the results related 22 00:01:27,260 --> 00:01:30,750 to discrete Fourier transforms. 23 00:01:30,750 --> 00:01:33,080 Of much more practical importance 24 00:01:33,080 --> 00:01:36,600 is the fact that the fast Fourier transform 25 00:01:36,600 --> 00:01:39,180 algorithm, which we'll be presenting, 26 00:01:39,180 --> 00:01:43,350 is a highly efficient algorithm for computing 27 00:01:43,350 --> 00:01:45,660 the discrete Fourier transform. 28 00:01:45,660 --> 00:01:49,470 And in fact, as we'll see, it is orders 29 00:01:49,470 --> 00:01:53,970 of magnitude more efficient than a straightforward direct 30 00:01:53,970 --> 00:01:57,500 computation of the discrete Fourier transform. 31 00:01:57,500 --> 00:02:01,980 Now, We've indicated in previous lectures, 32 00:02:01,980 --> 00:02:05,880 and you can well imagine that this is true, 33 00:02:05,880 --> 00:02:09,120 that there are many contexts in which we 34 00:02:09,120 --> 00:02:13,470 would want to explicitly compute the discrete Fourier transform. 35 00:02:13,470 --> 00:02:16,710 For example, obviously, in spectral analysis 36 00:02:16,710 --> 00:02:19,260 of discrete time data, we would want 37 00:02:19,260 --> 00:02:23,400 to use the computation of the discrete Fourier transform. 38 00:02:23,400 --> 00:02:28,200 And we've seen also that the discrete Fourier transform can 39 00:02:28,200 --> 00:02:32,430 be used for the implementation of finite impulse response 40 00:02:32,430 --> 00:02:35,160 digital filters, or more generally, can be used 41 00:02:35,160 --> 00:02:38,440 to implement a convolution. 42 00:02:38,440 --> 00:02:44,650 It will turn out that because of the efficiency of the algorithm 43 00:02:44,650 --> 00:02:47,870 that we'll be discussing in these lectures, it, 44 00:02:47,870 --> 00:02:54,040 in fact, in many cases is more efficient to implement 45 00:02:54,040 --> 00:02:58,510 the convolution by explicitly computing a discrete Fourier 46 00:02:58,510 --> 00:03:01,870 transform, multiplying the transforms together, 47 00:03:01,870 --> 00:03:05,210 and then Inverse transforming the result. Of course, 48 00:03:05,210 --> 00:03:07,540 keeping track of the fact that that implements 49 00:03:07,540 --> 00:03:10,600 a circular convolution and doing the necessary things that 50 00:03:10,600 --> 00:03:12,700 need to be done if we're interested 51 00:03:12,700 --> 00:03:15,840 in a linear convolution. 52 00:03:15,840 --> 00:03:19,190 Now, I'd like, just briefly, to remark 53 00:03:19,190 --> 00:03:22,490 that this algorithm, this-- actually, it's 54 00:03:22,490 --> 00:03:24,260 a class of algorithms, which I'll 55 00:03:24,260 --> 00:03:28,800 refer to as the fast Fourier transform algorithms, 56 00:03:28,800 --> 00:03:33,150 were a major landmark in the development 57 00:03:33,150 --> 00:03:37,770 of the field of digital signal processing in its present form. 58 00:03:37,770 --> 00:03:41,400 The algorithm, one of the forms of the algorithm, 59 00:03:41,400 --> 00:03:47,230 was disclosed in a paper by Cooley and Tukey in 1965. 60 00:03:47,230 --> 00:03:52,380 And it turns out, though, that the algorithm 61 00:03:52,380 --> 00:03:56,160 has, in a variety of forms, been reinvented several times 62 00:03:56,160 --> 00:03:58,870 over the past 20 or 30 years. 63 00:03:58,870 --> 00:04:01,860 So it is mainly this class of algorithms 64 00:04:01,860 --> 00:04:05,640 that we'll be talking about in the next several lectures. 65 00:04:05,640 --> 00:04:12,700 And I stress again that one of the primary reasons why 66 00:04:12,700 --> 00:04:17,200 this is so important is because of the rather dramatic 67 00:04:17,200 --> 00:04:19,480 improvements in efficiency that it 68 00:04:19,480 --> 00:04:21,850 offers over just a straightforward 69 00:04:21,850 --> 00:04:25,630 computation of the discrete Fourier transform. 70 00:04:25,630 --> 00:04:28,390 Well, let's look specifically at what's 71 00:04:28,390 --> 00:04:34,070 involved in the computation of the discrete Fourier transform. 72 00:04:34,070 --> 00:04:36,880 The discrete Fourier transform, as we know, 73 00:04:36,880 --> 00:04:41,980 is given by the sum of x sub n times W sub n 74 00:04:41,980 --> 00:04:47,470 to the nk, where W sub n is the complex exponential, e 75 00:04:47,470 --> 00:04:51,400 to the minus j, 2 pi over capital n. 76 00:04:51,400 --> 00:04:56,620 So it is this expression that we want to compute. 77 00:04:56,620 --> 00:05:00,340 Or equivalently, there is the inverse discrete Fourier 78 00:05:00,340 --> 00:05:04,390 transform given by 1 over capital N 79 00:05:04,390 --> 00:05:07,930 times the sum of x of k, the DFT, 80 00:05:07,930 --> 00:05:11,320 times W sub N to the minus nk. 81 00:05:11,320 --> 00:05:14,680 Now, hopefully you recall from the discussion 82 00:05:14,680 --> 00:05:16,960 of the discrete Fourier transform, 83 00:05:16,960 --> 00:05:21,100 that the inverse DFT is really the same 84 00:05:21,100 --> 00:05:24,280 as the discrete Fourier transform if we reverse 85 00:05:24,280 --> 00:05:26,200 the result in N, et cetera. 86 00:05:26,200 --> 00:05:31,420 In other words, computation of the inverse DFT 87 00:05:31,420 --> 00:05:38,020 is really essentially the same as the computation of the DFT. 88 00:05:38,020 --> 00:05:40,690 So whereas we'll be referring the discussion 89 00:05:40,690 --> 00:05:44,380 to the computation of the discrete Fourier transform, 90 00:05:44,380 --> 00:05:46,690 essentially everything that we say 91 00:05:46,690 --> 00:05:53,300 applies also to a computation of the inverse DFT. 92 00:05:53,300 --> 00:05:56,360 Well, let's look, first of all, at what 93 00:05:56,360 --> 00:05:59,810 would be involved in just a straightforward 94 00:05:59,810 --> 00:06:03,200 computation of the discrete Fourier transform. 95 00:06:03,200 --> 00:06:08,570 Basically what this expression says is to take x of n, 96 00:06:08,570 --> 00:06:11,810 multiply it by the appropriate powers of W, 97 00:06:11,810 --> 00:06:16,810 and then sum the result from 0 to capital N minus 1. 98 00:06:16,810 --> 00:06:19,010 And so, we basically have that set 99 00:06:19,010 --> 00:06:24,920 of operations to carry out for each value of k for which we're 100 00:06:24,920 --> 00:06:28,190 computing the discrete Fourier transform. 101 00:06:28,190 --> 00:06:31,040 Well, if we think of x of n as being complex, 102 00:06:31,040 --> 00:06:34,190 which in the most general case it will be, 103 00:06:34,190 --> 00:06:42,440 then we have in the direct computation of the DFT, 104 00:06:42,440 --> 00:06:48,440 we have to form the product, x of n times W sub N to the nk. 105 00:06:48,440 --> 00:06:54,860 And what that involves is one complex multiplication. 106 00:06:54,860 --> 00:06:56,190 This is a complex number. 107 00:06:56,190 --> 00:06:57,800 That's a complex number. 108 00:06:57,800 --> 00:07:01,400 Multiplying the two is then one complex multiply, 109 00:07:01,400 --> 00:07:06,980 or equivalently, four real multiplies and two real adds. 110 00:07:09,680 --> 00:07:15,410 OK, well now that is to form one term in the sum. 111 00:07:15,410 --> 00:07:22,680 We have n terms altogether in the sum, or capital N 112 00:07:22,680 --> 00:07:25,800 minus one additions, complex additions, 113 00:07:25,800 --> 00:07:27,990 to implement in the sum. 114 00:07:27,990 --> 00:07:35,680 So to compute the DFT for one value of k 115 00:07:35,680 --> 00:07:41,350 will require capital N complex multiplications and capital 116 00:07:41,350 --> 00:07:45,100 N minus 1 complex additions. 117 00:07:45,100 --> 00:07:48,130 Well to compute the discrete Fourier transform 118 00:07:48,130 --> 00:07:53,990 for all capital N values of k requires then 119 00:07:53,990 --> 00:07:58,810 n squared complex multiplies and n times 120 00:07:58,810 --> 00:08:01,940 n minus 1 complex additions. 121 00:08:01,940 --> 00:08:08,200 Or basically if capital N is any reasonable number, 122 00:08:08,200 --> 00:08:11,380 we can approximate this as N squared. 123 00:08:11,380 --> 00:08:15,730 And so, approximately, the computation involved 124 00:08:15,730 --> 00:08:21,460 is on the order of capital N squared complex multiplies 125 00:08:21,460 --> 00:08:23,680 and adds. 126 00:08:23,680 --> 00:08:25,530 Now, a few remarks here. 127 00:08:25,530 --> 00:08:29,590 One, first of all, is that in counting up 128 00:08:29,590 --> 00:08:32,650 the number of complex multiplies and adds, 129 00:08:32,650 --> 00:08:38,140 we didn't particularly take account of multiplies by unity. 130 00:08:38,140 --> 00:08:42,390 For example, when k is equal to 0, W sub N to the nk 131 00:08:42,390 --> 00:08:44,800 is equal to unity, and we, in fact, 132 00:08:44,800 --> 00:08:48,400 wouldn't have to carry out a multiplication. 133 00:08:48,400 --> 00:08:52,660 But there are a small enough number 134 00:08:52,660 --> 00:08:56,920 of those cases in comparison to all of the multiplies involved 135 00:08:56,920 --> 00:09:00,760 so that this is generally an indication of what 136 00:09:00,760 --> 00:09:04,990 the dependence of the computation is on N. 137 00:09:04,990 --> 00:09:09,610 Second of all, and this will occur throughout this next set 138 00:09:09,610 --> 00:09:13,810 of lectures, we are tying a discussion 139 00:09:13,810 --> 00:09:19,030 of the computational complexity or efficiency to counting 140 00:09:19,030 --> 00:09:22,390 the number of complex multiplies and adds 141 00:09:22,390 --> 00:09:26,020 that are involved in the computation of x of k, 142 00:09:26,020 --> 00:09:29,680 and not particularly focusing on other issues, such as indexing, 143 00:09:29,680 --> 00:09:31,270 bookkeeping, et cetera. 144 00:09:31,270 --> 00:09:34,060 So in discussing the computational complexity 145 00:09:34,060 --> 00:09:37,510 or efficiency, we'll tend to refer 146 00:09:37,510 --> 00:09:41,050 to the number of multiplications and adds 147 00:09:41,050 --> 00:09:45,320 that are involved in implementing the transform. 148 00:09:45,320 --> 00:09:48,980 Well, we have then a direct computation, 149 00:09:48,980 --> 00:09:53,180 which involves on the order of n squared complex multiplies 150 00:09:53,180 --> 00:09:54,590 and adds. 151 00:09:54,590 --> 00:09:59,000 And the important point then is its dependency on N, 152 00:09:59,000 --> 00:10:01,190 namely N squared. 153 00:10:01,190 --> 00:10:03,830 There are a variety of algorithms 154 00:10:03,830 --> 00:10:07,340 that have been known for some time, in fact, 155 00:10:07,340 --> 00:10:10,040 and ones that you can generate yourself 156 00:10:10,040 --> 00:10:13,430 by taking advantage of various symmetry properties 157 00:10:13,430 --> 00:10:16,220 of these complex exponentials. 158 00:10:16,220 --> 00:10:21,200 To reduce the constant of proportionality 159 00:10:21,200 --> 00:10:26,960 on the computation, that is, to make the computation perhaps 160 00:10:26,960 --> 00:10:31,820 not involve N squared multiplies and adds but only 1/2 N squared 161 00:10:31,820 --> 00:10:33,650 or 1/4 N squared. 162 00:10:33,650 --> 00:10:37,940 But still, in cases like that, the algorithms 163 00:10:37,940 --> 00:10:40,460 involved basically fooling around 164 00:10:40,460 --> 00:10:45,530 with a constant of proportionality out in front, 165 00:10:45,530 --> 00:10:50,360 whereas the main dependence on N, the size of the transform, 166 00:10:50,360 --> 00:10:53,430 remains the same. 167 00:10:53,430 --> 00:10:57,180 And you can, I think, convince yourself rather quickly 168 00:10:57,180 --> 00:11:03,000 with some fairly standard computers 169 00:11:03,000 --> 00:11:07,650 that if, in fact, the computation to be done 170 00:11:07,650 --> 00:11:12,690 is proportional to N squared, then very large transforms 171 00:11:12,690 --> 00:11:15,810 certainly aren't possible in very reasonable amounts 172 00:11:15,810 --> 00:11:17,230 of time. 173 00:11:17,230 --> 00:11:20,370 For example, if N is on the order of 1,000, 174 00:11:20,370 --> 00:11:23,340 then we're talking about on the order of a million 175 00:11:23,340 --> 00:11:25,980 complex multiplies and adds. 176 00:11:25,980 --> 00:11:29,310 And if we assume that one of those took, 177 00:11:29,310 --> 00:11:33,120 say, two microseconds or three microseconds to implement, 178 00:11:33,120 --> 00:11:35,820 or even one microsecond, then clearly we're 179 00:11:35,820 --> 00:11:38,790 talking about computation on the order of seconds. 180 00:11:38,790 --> 00:11:41,220 And you can convince yourself rather quickly 181 00:11:41,220 --> 00:11:46,920 that the computation grows very fast as N increases from 1,000 182 00:11:46,920 --> 00:11:50,220 to 2,000 to 4,000, et cetera. 183 00:11:50,220 --> 00:11:52,920 All right, so direct computation, the dependency 184 00:11:52,920 --> 00:11:56,190 is an n squared type of dependency. 185 00:11:56,190 --> 00:12:00,450 And this is in contrast to the primary algorithm 186 00:12:00,450 --> 00:12:03,960 that we want to discuss, which is the fast Fourier transform 187 00:12:03,960 --> 00:12:06,480 algorithm. 188 00:12:06,480 --> 00:12:11,940 Basically, the fast Fourier transform algorithm 189 00:12:11,940 --> 00:12:18,450 is directed at computing the discrete Fourier transform 190 00:12:18,450 --> 00:12:24,630 for a choice of N, the size of the transform, 191 00:12:24,630 --> 00:12:26,100 highly composite. 192 00:12:26,100 --> 00:12:29,940 That is, N is, in general-- 193 00:12:29,940 --> 00:12:32,100 of course, any N can be represented 194 00:12:32,100 --> 00:12:34,110 as a product of primes. 195 00:12:34,110 --> 00:12:38,370 And as we'll see, the larger the number of primes 196 00:12:38,370 --> 00:12:41,430 involved in the decomposition of N, 197 00:12:41,430 --> 00:12:44,940 the greater the efficiency of the fast Fourier 198 00:12:44,940 --> 00:12:48,840 transform as an algorithm for computing the discrete Fourier 199 00:12:48,840 --> 00:12:50,810 transform. 200 00:12:50,810 --> 00:12:56,930 In particular, we'll see that for N, a product of primes, 201 00:12:56,930 --> 00:13:00,860 using the fast Fourier transform algorithm, 202 00:13:00,860 --> 00:13:04,760 the number of complex multiplies and additions 203 00:13:04,760 --> 00:13:10,100 involved will turn out to be proportional not to N 204 00:13:10,100 --> 00:13:13,620 squared as it is in the direct computation, 205 00:13:13,620 --> 00:13:18,500 but proportional to N times a sum of the primes 206 00:13:18,500 --> 00:13:24,530 involved in that decomposition of N. So the greater the number 207 00:13:24,530 --> 00:13:28,640 of primes that we can decompose N in and the smaller each 208 00:13:28,640 --> 00:13:32,930 of those primes, the higher the efficiency of the fast Fourier 209 00:13:32,930 --> 00:13:36,691 transform algorithm will be. 210 00:13:36,691 --> 00:13:37,190 Well, 211 00:13:37,190 --> 00:13:42,350 The most highly composite that we can make N, and at the same 212 00:13:42,350 --> 00:13:46,700 time keeping the primes, p, as small as possible, 213 00:13:46,700 --> 00:13:50,760 is to choose N to be a power of 2. 214 00:13:50,760 --> 00:13:54,550 So that N is of the form 2 the nu, where 215 00:13:54,550 --> 00:13:58,520 nu is the log to the base 2 of N. And it will turn out, 216 00:13:58,520 --> 00:14:02,570 in that case, that the complex multiplies and additions 217 00:14:02,570 --> 00:14:07,280 is proportional to N times log to the base 2 N, 218 00:14:07,280 --> 00:14:09,530 rather than, in the direct computation, 219 00:14:09,530 --> 00:14:11,900 proportional to N squared. 220 00:14:11,900 --> 00:14:14,990 Let me incidentally comment that although I've 221 00:14:14,990 --> 00:14:19,010 referred to these p's as primes, they, in fact, 222 00:14:19,010 --> 00:14:20,750 don't have to be primes. 223 00:14:20,750 --> 00:14:23,280 The important issue, as we'll see, 224 00:14:23,280 --> 00:14:27,050 is that N be representable as a product of numbers. 225 00:14:27,050 --> 00:14:28,550 And it's not particularly important 226 00:14:28,550 --> 00:14:32,600 that those numbers turn out to be primes. 227 00:14:32,600 --> 00:14:39,160 Well, OK, so this then is the kind of efficiency 228 00:14:39,160 --> 00:14:43,420 that we will get out of the fast Fourier transform algorithm. 229 00:14:43,420 --> 00:14:47,530 And let me begin the discussion of the algorithm 230 00:14:47,530 --> 00:14:52,560 by concentrating on the case in which N is a power of 2. 231 00:14:52,560 --> 00:14:59,540 Well, basically, the idea behind the fast Fourier transform 232 00:14:59,540 --> 00:15:04,460 algorithm is to decompose the computation 233 00:15:04,460 --> 00:15:08,200 of the discrete Fourier transform 234 00:15:08,200 --> 00:15:12,740 into a set of subsequences. 235 00:15:12,740 --> 00:15:18,950 And this is-- this generates a class of forms 236 00:15:18,950 --> 00:15:21,110 for the algorithm, which are referred 237 00:15:21,110 --> 00:15:25,280 to as decimation-in-time forms of the algorithm. 238 00:15:25,280 --> 00:15:29,210 In particular, suppose that we consider 239 00:15:29,210 --> 00:15:33,170 decomposing the original sequence x of n 240 00:15:33,170 --> 00:15:35,330 into two subsequences. 241 00:15:35,330 --> 00:15:38,060 One sequence, subsequence, consists 242 00:15:38,060 --> 00:15:41,690 of the even-numbered points of x of n. 243 00:15:41,690 --> 00:15:44,450 And the second subsequence consists 244 00:15:44,450 --> 00:15:47,390 of the odd-numbered points of x of n. 245 00:15:47,390 --> 00:15:51,020 Well, let's just do that and see where it gets us. 246 00:15:51,020 --> 00:15:55,610 We can then rewrite the discrete Fourier transform 247 00:15:55,610 --> 00:16:01,070 as a sum over n even of x of n times W sub N 248 00:16:01,070 --> 00:16:05,950 to the nk plus the sum over n odd of x 249 00:16:05,950 --> 00:16:08,750 of n W sub N to the nk. 250 00:16:08,750 --> 00:16:13,280 Simply pulling out from this the even-numbered values 251 00:16:13,280 --> 00:16:16,976 and the odd-numbered values. 252 00:16:16,976 --> 00:16:19,980 All right, now by a substitution of variables, 253 00:16:19,980 --> 00:16:28,260 we can substitute for n 2 r in the even-numbered terms. 254 00:16:28,260 --> 00:16:31,950 And we can substitute n equals 2 r plus 1 255 00:16:31,950 --> 00:16:35,170 for the odd-number terms, in which case, 256 00:16:35,170 --> 00:16:39,780 when we convert these two sums over r, the sum on r 257 00:16:39,780 --> 00:16:44,700 will now range from 0 to capital N over 2 minus 1. 258 00:16:44,700 --> 00:16:48,300 As r runs from 0 to capital N over 2 minus 1, 259 00:16:48,300 --> 00:16:52,740 2 r ranges over the even-numbered terms, and 2 r 260 00:16:52,740 --> 00:16:57,250 plus 1 ranges over the odd-numbered terms. 261 00:16:57,250 --> 00:17:01,650 All right, so making the substitution of variables then, 262 00:17:01,650 --> 00:17:08,770 this sum is converted into a sum from r equals 0 263 00:17:08,770 --> 00:17:11,186 to N over 2 minus 1. 264 00:17:11,186 --> 00:17:16,400 x of 2r W sub N to the 2rk plus a sum-- 265 00:17:16,400 --> 00:17:18,640 those are the even-numbered terms-- 266 00:17:18,640 --> 00:17:22,510 plus a sum over the odd-numbered terms from 0 267 00:17:22,510 --> 00:17:26,109 to n over 2 minus 1. x of 2r plus 1, 268 00:17:26,109 --> 00:17:33,760 the odd-numbered points, times W sub N to the 2r plus 1 times k. 269 00:17:33,760 --> 00:17:35,820 Well, let's look at this factor. 270 00:17:35,820 --> 00:17:39,220 W sub N to the 2r plus 1 times k, 271 00:17:39,220 --> 00:17:46,170 that's equal to W sub N to the k, because of the 1, 272 00:17:46,170 --> 00:17:50,080 and W sub N to the 2rk. 273 00:17:50,080 --> 00:17:53,530 So we can, first of all, break this term up 274 00:17:53,530 --> 00:17:56,560 into a product of these two terms. 275 00:17:56,560 --> 00:18:01,420 And now, let's look at this one, W sub N to the 2rk. 276 00:18:01,420 --> 00:18:07,055 Well, W sub N squared is equal to e to the minus j 277 00:18:07,055 --> 00:18:10,410 2 pi over capital N squared. 278 00:18:10,410 --> 00:18:12,960 So the factor of 2. 279 00:18:12,960 --> 00:18:14,960 If we take the 2 down in the denominator 280 00:18:14,960 --> 00:18:18,440 and associate it with the N, we can rewrite this as e 281 00:18:18,440 --> 00:18:23,820 to the minus j 2 pi over N/2. 282 00:18:23,820 --> 00:18:31,540 So what W sub N squared is equal to is W sub capital N over 2. 283 00:18:31,540 --> 00:18:34,680 In other words, the W that we would associate with an N/2 284 00:18:34,680 --> 00:18:38,580 point discrete Fourier transform. 285 00:18:38,580 --> 00:18:43,470 Finally, substituting this together 286 00:18:43,470 --> 00:18:47,470 with this statement back into this expression, 287 00:18:47,470 --> 00:18:55,240 we have, for x of k, the sum from 0 to N/2 minus 1, 288 00:18:55,240 --> 00:19:01,125 x of 2r W sub N over 2 to the rk plus W sub 289 00:19:01,125 --> 00:19:07,050 N to the k times the sum of x of 2r plus 1 W sub N over 2 290 00:19:07,050 --> 00:19:09,630 to the rk. 291 00:19:09,630 --> 00:19:14,610 Well, what this looks like is an N/2 point discrete Fourier 292 00:19:14,610 --> 00:19:16,420 transform. 293 00:19:16,420 --> 00:19:18,850 And what this looks like is an N/2 294 00:19:18,850 --> 00:19:20,320 point discrete Fourier transform. 295 00:19:20,320 --> 00:19:22,550 And in fact, that's what they are. 296 00:19:22,550 --> 00:19:27,880 This is an N/2 point DFT of the even-numbered points 297 00:19:27,880 --> 00:19:30,100 in a sequence x of n. 298 00:19:30,100 --> 00:19:35,560 This is an N/2 point of the odd-numbered points 299 00:19:35,560 --> 00:19:37,870 in the sequence x of n. 300 00:19:37,870 --> 00:19:41,110 So this says we can compute x of k 301 00:19:41,110 --> 00:19:47,680 by computing an N/2 point DFT, another N/2 point DFT, 302 00:19:47,680 --> 00:19:51,010 and combining those two together with this factor 303 00:19:51,010 --> 00:19:54,520 out in front, W sub N to the k. 304 00:19:54,520 --> 00:19:56,350 Now, it's not particularly obvious 305 00:19:56,350 --> 00:19:58,450 yet that, in fact, in doing this, 306 00:19:58,450 --> 00:20:00,980 we've saved something in computation. 307 00:20:00,980 --> 00:20:04,600 But let's look at what this implies 308 00:20:04,600 --> 00:20:08,650 in terms of the computational steps in a little more detail. 309 00:20:11,210 --> 00:20:20,567 So here we have, again, X of k as an N/2 point 310 00:20:20,567 --> 00:20:28,020 DFT involving the even-numbered points in x of n 311 00:20:28,020 --> 00:20:33,110 plus an N/2 point DFT involving the odd-numbered points in x 312 00:20:33,110 --> 00:20:34,590 of n. 313 00:20:34,590 --> 00:20:37,620 Those are both N/2 point DFTs. 314 00:20:37,620 --> 00:20:41,250 This one, if we refer to as G of k, 315 00:20:41,250 --> 00:20:44,670 this one, if we refer to as H of k, 316 00:20:44,670 --> 00:20:48,240 then the computation of the DFT, X of k 317 00:20:48,240 --> 00:20:54,790 is G of k plus W sub N to the k times H of k. 318 00:20:54,790 --> 00:20:56,410 All right, let's see what this means 319 00:20:56,410 --> 00:21:01,180 in terms of computation, computational complexity. 320 00:21:01,180 --> 00:21:05,910 We have to compute an N/2 point DFT. 321 00:21:05,910 --> 00:21:09,240 We just argued that an endpoint DFT 322 00:21:09,240 --> 00:21:14,950 involves N squared complex multiplies and adds. 323 00:21:14,950 --> 00:21:18,260 So the computation of G of k involves 324 00:21:18,260 --> 00:21:23,010 N/2 squared complex multiplies and adds. 325 00:21:23,010 --> 00:21:27,900 The computation of H of k involves, likewise, 326 00:21:27,900 --> 00:21:32,010 N/2 squared complex multiplies and adds. 327 00:21:32,010 --> 00:21:36,710 So we have then 2 times N/2 squared multiplies 328 00:21:36,710 --> 00:21:44,400 and adds due to the computation of these two N/2 point DFT. 329 00:21:44,400 --> 00:21:47,970 In addition, we have to combine these together 330 00:21:47,970 --> 00:21:51,810 by multiplying by W sub N to the k. 331 00:21:51,810 --> 00:21:56,010 That involves capital N complex multiplies 332 00:21:56,010 --> 00:22:00,430 and adds, because k ranges from 0 to N minus 1. 333 00:22:00,430 --> 00:22:04,570 In other words, k ranges over N different values. 334 00:22:04,570 --> 00:22:08,580 So there are N complex multiplies involved in this, 335 00:22:08,580 --> 00:22:12,610 and then N complex adds in combining these together. 336 00:22:12,610 --> 00:22:16,200 So in addition to the computation of the two N/2 337 00:22:16,200 --> 00:22:20,430 point DFTs, we have the combination of them. 338 00:22:20,430 --> 00:22:26,880 And the resulting computation involves N plus N squared over 339 00:22:26,880 --> 00:22:30,720 2 complex multiplies and adds. 340 00:22:30,720 --> 00:22:33,540 And you can convince yourself on the back of an envelope 341 00:22:33,540 --> 00:22:37,350 very quickly that this number will always turn out 342 00:22:37,350 --> 00:22:39,810 to be less than N squared, except, I 343 00:22:39,810 --> 00:22:46,410 think, for some trivial cases like N equals 1 or N equals 2. 344 00:22:46,410 --> 00:22:51,490 Now, one minor point to be made-- 345 00:22:51,490 --> 00:22:53,320 in combining these two together, I 346 00:22:53,320 --> 00:22:58,080 commented that k has to run from 0 to capital N minus 1. 347 00:22:58,080 --> 00:23:02,890 That is, there are N different values that k ranges over, 348 00:23:02,890 --> 00:23:08,050 so doesn't that say that in computing G of k, 349 00:23:08,050 --> 00:23:13,120 I can't just compute it from k equals 0 to n over 2 minus 1 350 00:23:13,120 --> 00:23:16,050 as I normally would in computing a DFT? 351 00:23:16,050 --> 00:23:19,870 Don't I have to compute it from 0 all the way up to N 352 00:23:19,870 --> 00:23:22,050 or all the way up to N minus 1? 353 00:23:22,050 --> 00:23:23,620 Well, sure I do. 354 00:23:23,620 --> 00:23:26,440 I do, because those are the values that I 355 00:23:26,440 --> 00:23:29,110 need in here and in here. 356 00:23:29,110 --> 00:23:32,380 But what does that imply in terms of computation? 357 00:23:32,380 --> 00:23:41,550 Well, we know that an N/2 point DFT is periodic in k, 358 00:23:41,550 --> 00:23:47,070 so that if we compute only the first N/2 points, 359 00:23:47,070 --> 00:23:49,230 we know that the next N/2 points are 360 00:23:49,230 --> 00:23:51,000 going to look exactly the same. 361 00:23:51,000 --> 00:23:53,100 And so, there's no point in computing them. 362 00:23:53,100 --> 00:23:57,330 So the point then is that in computing G of k 363 00:23:57,330 --> 00:24:00,450 for this computation, even though we want k 364 00:24:00,450 --> 00:24:04,410 to run from 0 to N minus 1, we only 365 00:24:04,410 --> 00:24:08,610 need to explicitly do this computation for k 366 00:24:08,610 --> 00:24:11,170 from 0 to N over 2 minus 1. 367 00:24:11,170 --> 00:24:14,490 And then we simply use those values over again 368 00:24:14,490 --> 00:24:19,320 for the second range from k from N/2 to N minus 1. 369 00:24:19,320 --> 00:24:23,820 And that, by the way, is the essential key 370 00:24:23,820 --> 00:24:27,300 in the fast Fourier transform algorithm. 371 00:24:27,300 --> 00:24:31,080 It's basically that periodicity in the discrete Fourier 372 00:24:31,080 --> 00:24:33,270 transform that will lead to the kind 373 00:24:33,270 --> 00:24:36,630 of computational efficiency that's about to result. 374 00:24:36,630 --> 00:24:40,260 All right, so let's look then at how we would actually 375 00:24:40,260 --> 00:24:44,970 do the computation of the discrete Fourier transform. 376 00:24:44,970 --> 00:24:50,430 We know then that we will break the computation 377 00:24:50,430 --> 00:24:57,830 into a computation involving the even-numbered points and x 378 00:24:57,830 --> 00:25:02,880 of n and the odd-numbered points and x of n. 379 00:25:02,880 --> 00:25:06,750 Compute an N/2 point discrete Fourier transform 380 00:25:06,750 --> 00:25:10,785 to obtain the sequence G of k, another N/2 381 00:25:10,785 --> 00:25:15,090 point discrete Fourier transform to compute the sequence H of k, 382 00:25:15,090 --> 00:25:19,740 and then combine the results together by multiplying H of k 383 00:25:19,740 --> 00:25:24,510 by W sub N to the k and adding that to G of k. 384 00:25:24,510 --> 00:25:28,140 I'll illustrate the computation, incidentally, 385 00:25:28,140 --> 00:25:31,920 with choosing N equal to 8, simply 386 00:25:31,920 --> 00:25:34,450 because I have to choose N equal to something. 387 00:25:34,450 --> 00:25:36,690 And it turns out, interestingly enough, 388 00:25:36,690 --> 00:25:41,880 that N equals 8 is sufficiently general, 389 00:25:41,880 --> 00:25:45,300 so that you can infer the generality for N 390 00:25:45,300 --> 00:25:48,120 equal to an arbitrary power of 2 just 391 00:25:48,120 --> 00:25:51,420 from the computational issues that we'll 392 00:25:51,420 --> 00:25:53,760 raise for this particular case. 393 00:25:53,760 --> 00:25:56,850 All right, so we have N/2 point DFT 394 00:25:56,850 --> 00:25:59,670 of the even-numbered points, the N/2 395 00:25:59,670 --> 00:26:03,210 point DFT of the odd-numbered points. 396 00:26:03,210 --> 00:26:08,280 And now, to compute the output points, the discrete Fourier 397 00:26:08,280 --> 00:26:18,450 transform, X of k, we then want to take G of 0 398 00:26:18,450 --> 00:26:23,790 and add it to H of 0 multiplied by W sub N to the 0. 399 00:26:23,790 --> 00:26:27,300 So using flow graph notation, we have 400 00:26:27,300 --> 00:26:36,370 G of 0 added to H of 0 multiplied by W sub N so the 0. 401 00:26:36,370 --> 00:26:41,950 For x of 1, we want G of 1 added to H of 1 402 00:26:41,950 --> 00:26:45,830 multiplied by W sub N to the 1. 403 00:26:45,830 --> 00:26:51,050 For x of 2, we want G of 2 plus H of 2 404 00:26:51,050 --> 00:26:53,630 multiplied by W sub N squared. 405 00:26:53,630 --> 00:26:58,010 And for x of 3, likewise, G of 3 added 406 00:26:58,010 --> 00:27:01,970 to H of 3 multiplied by W sub N cubed. 407 00:27:01,970 --> 00:27:09,650 So that computation, then, gives us x of 0 through x of 3. 408 00:27:09,650 --> 00:27:17,420 For x of 4, we want G of 4 plus W sub N to the 4 times H of 4. 409 00:27:17,420 --> 00:27:19,190 But what's G of 4? 410 00:27:19,190 --> 00:27:24,950 G of 4 is equal to G of 0, because of the fact that G of k 411 00:27:24,950 --> 00:27:27,560 is periodic with period 4. 412 00:27:27,560 --> 00:27:30,190 This is a four-point DFT that we're computing. 413 00:27:30,190 --> 00:27:33,350 So G of k is periodic with period 4. 414 00:27:33,350 --> 00:27:40,100 And consequently, H of 4 and G of 4 are equal to H of 0 415 00:27:40,100 --> 00:27:41,870 and G of 0. 416 00:27:41,870 --> 00:27:47,000 So to compute the next four points, 417 00:27:47,000 --> 00:28:05,560 we can likewise use the outputs of these N/2 point DFTs. 418 00:28:05,560 --> 00:28:10,240 To compute x of 4, we want G of 4, which is equal to G of 0, 419 00:28:10,240 --> 00:28:15,520 because of the periodicity, plus W sub N to 4 times H of 4. 420 00:28:15,520 --> 00:28:17,830 H of 4 is equal to H of 0. 421 00:28:17,830 --> 00:28:22,750 So we have then for x of 4, G of 0 plus W sub N 422 00:28:22,750 --> 00:28:25,730 to 4 times H of 0. 423 00:28:25,730 --> 00:28:32,440 Likewise, for x of 5, we have G of 5, or G of 1, 424 00:28:32,440 --> 00:28:37,390 plus W sub N to the 5th times H of 5, or H of 1 425 00:28:37,390 --> 00:28:40,390 because of the periodicity, and likewise, down 426 00:28:40,390 --> 00:28:42,560 through the remaining four points. 427 00:28:42,560 --> 00:28:46,060 So we showed previous-- on the previous view 428 00:28:46,060 --> 00:28:49,310 graph, the computation for the first four points. 429 00:28:49,310 --> 00:28:53,140 And here we have the computation for the last four points. 430 00:28:53,140 --> 00:28:56,920 Putting those two together then, the computation 431 00:28:56,920 --> 00:29:06,040 for the eight-point DFT is, as I've indicated here, 432 00:29:06,040 --> 00:29:08,080 the even-numbered points into an N/2 point 433 00:29:08,080 --> 00:29:13,630 DFT, the odd-numbered points into an N/2 point DFT, 434 00:29:13,630 --> 00:29:16,870 and the results combined together 435 00:29:16,870 --> 00:29:20,600 according to the computation that we've shown here. 436 00:29:20,600 --> 00:29:22,810 Now, a couple of things that I'd like 437 00:29:22,810 --> 00:29:26,070 to draw your attention to about the flow graph as it's 438 00:29:26,070 --> 00:29:27,430 developing. 439 00:29:27,430 --> 00:29:30,790 First of all, note that the flow graph, 440 00:29:30,790 --> 00:29:35,500 as we're looking at it here, involves a computation, which 441 00:29:35,500 --> 00:29:39,790 if we were to just pull out the computation of these two 442 00:29:39,790 --> 00:29:42,550 points, or these two points, et cetera, 443 00:29:42,550 --> 00:29:49,450 it has the structure of a set of butterfly-looking flow graphs. 444 00:29:49,450 --> 00:29:52,390 And, in fact, we'll find it convenient to refer 445 00:29:52,390 --> 00:29:57,680 to these as butterflies in the computation. 446 00:29:57,680 --> 00:30:00,620 The second thing that I'd like to draw your attention to 447 00:30:00,620 --> 00:30:07,110 is that the multiplications by powers of W 448 00:30:07,110 --> 00:30:13,020 always are applied to the branches which come 449 00:30:13,020 --> 00:30:14,970 from this bottom transform. 450 00:30:14,970 --> 00:30:18,390 And that's because we had the expression that X of k 451 00:30:18,390 --> 00:30:23,460 was equal to G of k plus W sub N to the k times H of k, 452 00:30:23,460 --> 00:30:25,560 and this was the H of k. 453 00:30:25,560 --> 00:30:30,390 So it's in the bottom branch of these butterflies 454 00:30:30,390 --> 00:30:34,410 that we see always the multiplication 455 00:30:34,410 --> 00:30:37,440 by these powers of W. And I say that, I 456 00:30:37,440 --> 00:30:41,220 want to point that out hit now, because it's somewhat 457 00:30:41,220 --> 00:30:44,730 difficult in drawing these flow graphs to always get 458 00:30:44,730 --> 00:30:48,610 these powers of W close enough to the arrow 459 00:30:48,610 --> 00:30:50,970 so that it's clear where the computation, where 460 00:30:50,970 --> 00:30:54,110 the multiplication needs to be carried out. 461 00:30:54,110 --> 00:30:56,330 All right, so here's where we're at. 462 00:30:56,330 --> 00:30:59,150 We had counted out previously the number 463 00:30:59,150 --> 00:31:01,850 of complex multiplies and adds involved, 464 00:31:01,850 --> 00:31:07,280 and we found that it was on the order of N plus 2 times 465 00:31:07,280 --> 00:31:12,080 and over 2 squared, which is less than N squared. 466 00:31:12,080 --> 00:31:14,960 And consequently, this is a more efficient way 467 00:31:14,960 --> 00:31:18,380 to compute the discrete Fourier transform than simply 468 00:31:18,380 --> 00:31:20,540 a direct computation. 469 00:31:20,540 --> 00:31:26,610 However, like this assumes that we're 470 00:31:26,610 --> 00:31:33,130 going to compute these N/2 point DFTs as a direct computation. 471 00:31:33,130 --> 00:31:36,990 But in fact, if N/2 is a composite number, 472 00:31:36,990 --> 00:31:39,420 in particular a power of 2, which 473 00:31:39,420 --> 00:31:42,630 it is if N was a power of 2, then we 474 00:31:42,630 --> 00:31:47,580 can apply the same strategy to the computation of these N/2 475 00:31:47,580 --> 00:31:49,530 point DFTs. 476 00:31:49,530 --> 00:31:55,920 So we can break these N/2 point DFT computations into-- 477 00:31:55,920 --> 00:31:59,160 each one of these-- into two N/4 point 478 00:31:59,160 --> 00:32:05,040 DFTs plus some additional arithmetic 479 00:32:05,040 --> 00:32:09,250 in order to form the N/2 point DFTs. 480 00:32:09,250 --> 00:32:13,210 So each one of the N/2 point DFTs 481 00:32:13,210 --> 00:32:22,690 can then be decomposed into an N/4 point DFT computation, 482 00:32:22,690 --> 00:32:26,440 into two N/4 point DFT computations. 483 00:32:26,440 --> 00:32:32,380 In particular, we had, recall, for the top half 484 00:32:32,380 --> 00:32:36,820 of the previous view graph, the even-numbered points 485 00:32:36,820 --> 00:32:38,890 in the original sequence. 486 00:32:38,890 --> 00:32:41,050 All right, now we want to compute the DFT 487 00:32:41,050 --> 00:32:43,100 of those even-numbered points. 488 00:32:43,100 --> 00:32:45,670 So we can go through exactly the same steps. 489 00:32:45,670 --> 00:32:49,360 We can break that into its even-numbered points 490 00:32:49,360 --> 00:32:52,550 and its odd numbered points. 491 00:32:52,550 --> 00:32:56,590 So we have the even numbered of the even-numbered points 492 00:32:56,590 --> 00:33:01,330 into an N/4 point DFT, the odd numbered 493 00:33:01,330 --> 00:33:07,060 of the even-numbered points into a second N/4 point DFT, 494 00:33:07,060 --> 00:33:09,910 and then those results are combined 495 00:33:09,910 --> 00:33:15,400 in exactly the same way as the N/2 point DFTs were combined 496 00:33:15,400 --> 00:33:21,010 in the previous view graph, but now with N/2 497 00:33:21,010 --> 00:33:24,160 as the subscript rather than N as the subscript, 498 00:33:24,160 --> 00:33:27,807 because here what we're computing is an N/2 point DFT. 499 00:33:30,310 --> 00:33:32,020 However, in drawing the flow graph, 500 00:33:32,020 --> 00:33:36,820 we can re-express the powers of W sub N over 2 501 00:33:36,820 --> 00:33:40,960 in terms of powers of W sub N, because of the fact 502 00:33:40,960 --> 00:33:47,120 that W sub N squared is equal to W sub N over 2 or vice versa. 503 00:33:47,120 --> 00:33:49,930 All right, so we have this then apply 504 00:33:49,930 --> 00:33:53,710 to the even-numbered points and to the odd-numbered points. 505 00:33:53,710 --> 00:33:56,290 And when we do that, then the flow 506 00:33:56,290 --> 00:34:04,910 graph that results is as I've indicated here. 507 00:34:04,910 --> 00:34:08,040 We have the even numbered of the even-numbered points 508 00:34:08,040 --> 00:34:10,969 and the odd numbered of the even-numbered points. 509 00:34:10,969 --> 00:34:13,760 We have the even numbered of the odd-numbered points 510 00:34:13,760 --> 00:34:17,480 and the odd numbered of the odd-numbered points. 511 00:34:17,480 --> 00:34:20,570 Those are each going into N/4 point DFTs. 512 00:34:23,429 --> 00:34:26,704 Here is then the sequence G of k. 513 00:34:29,350 --> 00:34:29,850 I'm sorry. 514 00:34:29,850 --> 00:34:33,030 Here is the sequence G of k. 515 00:34:33,030 --> 00:34:35,699 Here is the sequence H of k. 516 00:34:35,699 --> 00:34:40,409 This then implements the computation of the N/2 point 517 00:34:40,409 --> 00:34:43,110 DFTs that we had before. 518 00:34:43,110 --> 00:34:48,900 And then we have the output DFT obtained by combining those. 519 00:34:48,900 --> 00:34:53,260 So this flow graph results from simply carrying 520 00:34:53,260 --> 00:34:58,030 the previous strategy back one step to implementing the N/2 521 00:34:58,030 --> 00:35:00,300 point DFTs. 522 00:35:00,300 --> 00:35:06,040 All right, well, now we have the computation of N/4 point DFTs. 523 00:35:06,040 --> 00:35:13,090 And we can do exactly the same thing with the N/4 point DFTs, 524 00:35:13,090 --> 00:35:17,680 except that for this particular example, N/4-- 525 00:35:17,680 --> 00:35:20,950 if N is equal to 8, N/4 is equal to 2. 526 00:35:20,950 --> 00:35:24,490 The computation of the two-point DFTs 527 00:35:24,490 --> 00:35:28,130 is a relatively straightforward thing. 528 00:35:28,130 --> 00:35:30,940 But in general, if N wasn't equal to 8, 529 00:35:30,940 --> 00:35:33,580 if it was a more general power of 2, 530 00:35:33,580 --> 00:35:35,530 we could likewise apply this strategy 531 00:35:35,530 --> 00:35:40,140 to the computation of the N/4 point DFTs, the N/8 point DFTs, 532 00:35:40,140 --> 00:35:42,760 and over 16 point DFTs, et cetera, 533 00:35:42,760 --> 00:35:48,950 until we got to the computation of two-point DFTs. 534 00:35:48,950 --> 00:35:51,980 All right, so just simply inserting the computation 535 00:35:51,980 --> 00:35:57,290 for these two-point DFTs, which involves simply adding-- 536 00:35:57,290 --> 00:36:01,970 well, adding x of 4 to x of 0, multiplying x of 4 537 00:36:01,970 --> 00:36:05,690 by W sub N to the 4 and adding it to x of 0. 538 00:36:05,690 --> 00:36:12,230 The resulting flow graph for the entire computation 539 00:36:12,230 --> 00:36:20,990 is as I show here, where we have now the input points. 540 00:36:20,990 --> 00:36:23,690 We have the output points. 541 00:36:23,690 --> 00:36:32,410 Here we're computing the two-point DFTs, 542 00:36:32,410 --> 00:36:36,880 combining those together to get the four-point DFTs, 543 00:36:36,880 --> 00:36:41,090 combining those together to get the eight-point DFTs, 544 00:36:41,090 --> 00:36:42,190 et cetera. 545 00:36:42,190 --> 00:36:46,210 And I'll remind you again that the basic computation that's 546 00:36:46,210 --> 00:36:51,970 involved throughout these stages is the butterfly computation. 547 00:36:51,970 --> 00:36:59,560 We see it here, we see it here, and we see it here. 548 00:36:59,560 --> 00:37:03,250 And always, the multiplication by powers of W 549 00:37:03,250 --> 00:37:09,020 is applied to the branches coming from the bottom node 550 00:37:09,020 --> 00:37:10,130 of the butterfly. 551 00:37:10,130 --> 00:37:14,720 And that's because of the fact that X of k 552 00:37:14,720 --> 00:37:19,280 is expressed as G of k plus W sub N to the k times H of k. 553 00:37:19,280 --> 00:37:21,950 And it's always the H of Ks that are 554 00:37:21,950 --> 00:37:24,500 what are being computed in the bottom half 555 00:37:24,500 --> 00:37:26,900 as we've been developing this flow graph. 556 00:37:26,900 --> 00:37:29,849 It will turn out that that's not true for other forms 557 00:37:29,849 --> 00:37:32,390 of the algorithm, but for the form that we're developing here 558 00:37:32,390 --> 00:37:34,140 it is. 559 00:37:34,140 --> 00:37:37,700 All right, well, having carried this 560 00:37:37,700 --> 00:37:42,110 all the way back to the two-point DFTs, 561 00:37:42,110 --> 00:37:46,580 let's see what this has accomplished for us in terms 562 00:37:46,580 --> 00:37:50,620 of computational efficiency. 563 00:37:50,620 --> 00:37:55,410 We began with N equal to a power of 2. 564 00:37:55,410 --> 00:37:59,340 We thought about computing a direct discrete Fourier 565 00:37:59,340 --> 00:38:04,320 transform, but that involved N squared complex multiplies 566 00:38:04,320 --> 00:38:06,720 and adds. 567 00:38:06,720 --> 00:38:09,510 Instead of that, we broke the computation 568 00:38:09,510 --> 00:38:14,970 into two N/2 point DFTs plus some diddling 569 00:38:14,970 --> 00:38:17,460 to fit those together. 570 00:38:17,460 --> 00:38:21,720 The computation of the two N/2 point DFTs 571 00:38:21,720 --> 00:38:27,960 involved complex multiplies and adds equal to 2, 572 00:38:27,960 --> 00:38:31,680 because there were two of them, times N/2 squared 573 00:38:31,680 --> 00:38:34,470 plus N for the diddles-- 574 00:38:34,470 --> 00:38:37,920 the diddles being involved in fitting those two N/2 point 575 00:38:37,920 --> 00:38:40,240 DFTs together. 576 00:38:40,240 --> 00:38:42,820 And this, for N greater than two, 577 00:38:42,820 --> 00:38:46,010 is a bigger number than that. 578 00:38:46,010 --> 00:38:48,510 All right, then we said, well, why stop there? 579 00:38:48,510 --> 00:38:56,670 Let's break these two N/2 point DFTs into four N/4 point DFTs. 580 00:38:56,670 --> 00:39:00,420 That meant then that this N/2 squared, which 581 00:39:00,420 --> 00:39:05,340 came from a direct computation of the DFT, 582 00:39:05,340 --> 00:39:12,330 gets replaced instead by two N/4 point DFTs. 583 00:39:12,330 --> 00:39:19,710 So we have 2 times N/4 squared plus N/2, which 584 00:39:19,710 --> 00:39:24,760 are the diddles involved in, again, fitting those together. 585 00:39:24,760 --> 00:39:29,250 So the N/2 squared, which involved computing one N/2 586 00:39:29,250 --> 00:39:32,520 point DFT gets replaced by this when 587 00:39:32,520 --> 00:39:35,730 we go to the N/4 point DFTs. 588 00:39:35,730 --> 00:39:37,970 And consequently, substituting this in there, 589 00:39:37,970 --> 00:39:40,170 the computation is-- 590 00:39:40,170 --> 00:39:43,200 instead of that, it's N/4 squared 591 00:39:43,200 --> 00:39:49,140 times 4 plus N plus N. This N coming from that 2 times 592 00:39:49,140 --> 00:39:50,580 this N/2. 593 00:39:50,580 --> 00:39:53,670 Well, we can go through this again. 594 00:39:53,670 --> 00:39:59,730 We broke the N/4 point DFTs into N/8 point DFTs. 595 00:39:59,730 --> 00:40:03,420 Therefore, the N/4 squared got replaced by N/8 596 00:40:03,420 --> 00:40:09,290 squared twice, plus N/4 to fit those together. 597 00:40:09,290 --> 00:40:12,350 That gets substituted in for this N/4 squared. 598 00:40:12,350 --> 00:40:15,200 We get 8 times N/8 squared. 599 00:40:15,200 --> 00:40:20,820 We pick up another N, which is this 4 times that N/4. 600 00:40:20,820 --> 00:40:23,900 And you can see that each time we do this, 601 00:40:23,900 --> 00:40:28,850 this number gets smaller, and we keep picking up 602 00:40:28,850 --> 00:40:32,450 another factor of N added in. 603 00:40:32,450 --> 00:40:36,140 Well, if you track this through, what you find is that for N 604 00:40:36,140 --> 00:40:41,180 equal to a power of 2, in fact, the computation then 605 00:40:41,180 --> 00:40:48,860 comes out proportional to N plus N plus N nu times, 606 00:40:48,860 --> 00:40:52,160 where nu is log for the base 2 of N. 607 00:40:52,160 --> 00:40:57,620 And so, the number of complex multiplies and adds involved 608 00:40:57,620 --> 00:41:01,550 in implementing the discrete Fourier transform 609 00:41:01,550 --> 00:41:07,430 using the fast Fourier transform algorithm is proportional to N 610 00:41:07,430 --> 00:41:11,200 times nu rather than proportional to N squared. 611 00:41:11,200 --> 00:41:14,840 A rather dramatic savings. 612 00:41:14,840 --> 00:41:18,000 OK, well there is one-- 613 00:41:18,000 --> 00:41:19,700 there are a number of modifications 614 00:41:19,700 --> 00:41:23,660 that we're going to want to make to this basic structure 615 00:41:23,660 --> 00:41:27,230 for the computation as we go through this lecture 616 00:41:27,230 --> 00:41:29,660 and the next several lectures. 617 00:41:29,660 --> 00:41:36,200 The first is to introduce a fairly straightforward 618 00:41:36,200 --> 00:41:42,290 simplification by, first of all, again, 619 00:41:42,290 --> 00:41:48,220 reminding you that the basic computation involved always, 620 00:41:48,220 --> 00:41:51,760 from one stage to the next, is always 621 00:41:51,760 --> 00:41:57,790 a butterfly type of computation with multiplications 622 00:41:57,790 --> 00:42:03,280 by powers of W applied to the signal from the bottom node. 623 00:42:03,280 --> 00:42:08,260 And you should notice, and it's rather straightforward 624 00:42:08,260 --> 00:42:11,770 to verify why this is true, that in each 625 00:42:11,770 --> 00:42:17,320 of these butterflies, the separation in powers 626 00:42:17,320 --> 00:42:22,780 of W between the factor going to the top node at the end 627 00:42:22,780 --> 00:42:24,550 of the butterfly and the factor going 628 00:42:24,550 --> 00:42:28,090 to the bottom node in the butterfly, 629 00:42:28,090 --> 00:42:31,780 they are always separated by N/2. 630 00:42:31,780 --> 00:42:39,850 So that means that the basic computation in the computation 631 00:42:39,850 --> 00:42:45,250 of the DFT, as I illustrated on the previous flow graph, 632 00:42:45,250 --> 00:42:49,090 is a butterfly, where the characteristic of it 633 00:42:49,090 --> 00:42:53,770 is that it's multiplication by unity from the top node, 634 00:42:53,770 --> 00:42:58,420 and multiplication by some power of W from the bottom 635 00:42:58,420 --> 00:43:04,360 to the top node, and a power of W displaced from this one 636 00:43:04,360 --> 00:43:09,400 by N/2 from the bottom node to the bottom node. 637 00:43:09,400 --> 00:43:13,480 Well, let's decompose this power of W 638 00:43:13,480 --> 00:43:20,520 into W sub N to the r times W sub N to the N/2. 639 00:43:20,520 --> 00:43:21,810 W sub N to the N/2. 640 00:43:21,810 --> 00:43:22,920 Well, what is that? 641 00:43:22,920 --> 00:43:25,590 That's e to the minus j 2 pi over capitalized N-- 642 00:43:25,590 --> 00:43:27,420 that's W sub N-- 643 00:43:27,420 --> 00:43:29,820 raised to the N/2 power. 644 00:43:29,820 --> 00:43:31,440 The capital N's cancel out. 645 00:43:31,440 --> 00:43:32,700 The 2's cancel out. 646 00:43:32,700 --> 00:43:36,820 We're left with e to the minus j pi, which is equal to minus 1. 647 00:43:36,820 --> 00:43:41,730 So then, in fact, this factor, or equivalently, this factor, 648 00:43:41,730 --> 00:43:48,450 is equal to minus 1 or minus W sub N to the r. 649 00:43:48,450 --> 00:43:53,360 Consequently, we could redraw the butterfly 650 00:43:53,360 --> 00:44:00,290 so that rather than implementing that multiplication in both 651 00:44:00,290 --> 00:44:02,660 the branch going to the top node and the branch 652 00:44:02,660 --> 00:44:05,510 going to the bottom node, we simply 653 00:44:05,510 --> 00:44:09,620 multiply the value coming into the butterfly 654 00:44:09,620 --> 00:44:13,880 by W sub N to the r, and then add it 655 00:44:13,880 --> 00:44:18,260 to this value at the top node, and subtract it 656 00:44:18,260 --> 00:44:21,320 from this value at the bottom node. 657 00:44:21,320 --> 00:44:25,960 So we could redraw our original flow graph 658 00:44:25,960 --> 00:44:30,080 by replacing the butterfly computation 659 00:44:30,080 --> 00:44:34,400 in the previous flow graph by this butterfly computation, 660 00:44:34,400 --> 00:44:37,400 where the multiplication by the power of W 661 00:44:37,400 --> 00:44:40,580 is extracted from inside the butterfly, 662 00:44:40,580 --> 00:44:45,230 so that basically we reduce the number of multiplications by W 663 00:44:45,230 --> 00:44:49,430 that we have to implement by a factor of 2. 664 00:44:49,430 --> 00:44:56,500 The flow graph that results when we do that we have here then, 665 00:44:56,500 --> 00:45:02,680 where the powers of W are now slid out of the butterfly 666 00:45:02,680 --> 00:45:06,500 to the entering node to the butterfly. 667 00:45:06,500 --> 00:45:09,550 And we now have, rather than multiplication 668 00:45:09,550 --> 00:45:14,740 by powers of W in the butterfly, each butterfly computation 669 00:45:14,740 --> 00:45:19,670 consists of an addition and a subtraction. 670 00:45:19,670 --> 00:45:22,990 So the basic computation now is a multiplication 671 00:45:22,990 --> 00:45:29,020 by a power of W followed by an addition and a subtraction. 672 00:45:29,020 --> 00:45:31,990 And this follows all the way through a multiplication 673 00:45:31,990 --> 00:45:38,380 by a power of W and then an addition and a subtraction, 674 00:45:38,380 --> 00:45:42,280 this minus 1 being associated with this line. 675 00:45:42,280 --> 00:45:45,340 The minus 1's, of course, always occur 676 00:45:45,340 --> 00:45:50,330 on the branches coming from the bottom node of the butterfly. 677 00:45:50,330 --> 00:45:52,300 That's consistent with everything 678 00:45:52,300 --> 00:45:56,700 that we've been saying throughout this discussion. 679 00:45:56,700 --> 00:46:03,410 All right, so this then is one form of an algorithm, 680 00:46:03,410 --> 00:46:06,620 referred to as the fast Fourier transform algorithm, 681 00:46:06,620 --> 00:46:10,550 for implementing the computation of the discrete Fourier 682 00:46:10,550 --> 00:46:12,110 transform. 683 00:46:12,110 --> 00:46:16,670 As the algorithm developed, it developed somewhat naturally 684 00:46:16,670 --> 00:46:19,490 in a flow graph form. 685 00:46:19,490 --> 00:46:23,360 And in fact, we can interpret this flow graph 686 00:46:23,360 --> 00:46:26,600 as a computational recipe for computing 687 00:46:26,600 --> 00:46:29,900 the discrete Fourier transform. 688 00:46:29,900 --> 00:46:33,650 There are a few things that I would 689 00:46:33,650 --> 00:46:35,450 like to draw your attention to. 690 00:46:35,450 --> 00:46:39,170 One primary one in particular at this point, that 691 00:46:39,170 --> 00:46:45,500 is that we notice that the input sequence values, because 692 00:46:45,500 --> 00:46:49,340 of the way that we derive this algorithm, the input sequence 693 00:46:49,340 --> 00:46:54,830 values got rearranged in a somewhat funny order 694 00:46:54,830 --> 00:46:57,410 at the input to the flow graph. 695 00:46:57,410 --> 00:46:59,120 Although, they come out of the flow 696 00:46:59,120 --> 00:47:04,850 graph in a sequential order, 0, 1, 2, 3, et cetera. 697 00:47:04,850 --> 00:47:10,745 Over here, it's an order of 0, 4, 2, 6 1, 5, 3, 7. 698 00:47:10,745 --> 00:47:15,410 It will turn out that that order has a certain amount 699 00:47:15,410 --> 00:47:16,310 of significance. 700 00:47:16,310 --> 00:47:20,180 In fact, as we'll see, as we continue 701 00:47:20,180 --> 00:47:25,550 this discussion in the next lecture, that this is in fact 702 00:47:25,550 --> 00:47:27,260 a bit reversed order. 703 00:47:27,260 --> 00:47:31,100 It corresponds to rearranging the order of the bits 704 00:47:31,100 --> 00:47:36,320 in the binary representation of the indices. 705 00:47:36,320 --> 00:47:40,130 And furthermore, it's possible to interpret 706 00:47:40,130 --> 00:47:43,580 this flow graph in terms, not just 707 00:47:43,580 --> 00:47:49,860 of the computation of the discrete Fourier transform, 708 00:47:49,860 --> 00:47:54,410 but it also suggests how to arrange 709 00:47:54,410 --> 00:48:00,020 the memory, that is, the storage of the input sequence 710 00:48:00,020 --> 00:48:03,690 and the resulting storage of the output sequence. 711 00:48:03,690 --> 00:48:07,200 There are some efficiencies that result when we do that. 712 00:48:07,200 --> 00:48:10,010 And that's one of the first topics 713 00:48:10,010 --> 00:48:12,590 that we'll begin with next time. 714 00:48:12,590 --> 00:48:17,210 So in the next lecture then, we will 715 00:48:17,210 --> 00:48:22,220 continue on the development of the fast Fourier transform 716 00:48:22,220 --> 00:48:25,550 algorithm, picking up from the point 717 00:48:25,550 --> 00:48:29,150 that we've reached so far, which is-- 718 00:48:29,150 --> 00:48:31,970 the point that we've reached is to have 719 00:48:31,970 --> 00:48:37,010 exposed one form of the fast Fourier transform algorithm. 720 00:48:37,010 --> 00:48:40,250 There are some important points to be made about that form. 721 00:48:40,250 --> 00:48:44,930 And then a variety of modifications of that form 722 00:48:44,930 --> 00:48:47,480 that have some advantages, and, as it will turn out, 723 00:48:47,480 --> 00:48:49,510 also some disadvantages. 724 00:48:49,510 --> 00:48:52,350 Thank you. 725 00:48:52,350 --> 00:48:55,824 [MUSIC PLAYING]