1 00:00:15,584 --> 00:00:18,820 PATRICK WINSTON: I have extremely bad news. 2 00:00:18,820 --> 00:00:21,690 Halloween falls this year on a Sunday. 3 00:00:21,690 --> 00:00:25,880 But we in 6.034 refuse to suffer the slings and arrows 4 00:00:25,880 --> 00:00:27,890 of outrageous fortune. 5 00:00:27,890 --> 00:00:30,980 So we've decided that Halloween is today, as far 6 00:00:30,980 --> 00:00:33,190 6.034 is concerned. 7 00:00:33,190 --> 00:00:34,695 Kenny, could you give me a hand, please? 8 00:00:43,910 --> 00:00:47,305 If you could take that and put it over there. 9 00:00:47,305 --> 00:00:49,260 STUDENT: [INAUDIBLE]? 10 00:00:49,260 --> 00:00:49,960 PATRICK WINSTON: Mm hm. 11 00:00:49,960 --> 00:00:51,210 Just give it to them. 12 00:00:54,120 --> 00:00:56,990 You can take as much of this as you like. 13 00:00:56,990 --> 00:00:59,655 The rest will be given to that herd of stampeding freshman 14 00:00:59,655 --> 00:01:00,905 that comes in after. 15 00:01:16,039 --> 00:01:21,000 It's a cornucopia of legal drugs. 16 00:01:21,000 --> 00:01:25,330 Chocolate does produce a kind of mild high, and I recommend 17 00:01:25,330 --> 00:01:30,890 it before quizzes and giving lectures. 18 00:01:30,890 --> 00:01:33,370 I have a friend of mine, one of the Nobel laureates in 19 00:01:33,370 --> 00:01:36,060 biology, always eats chocolate before he lectures. 20 00:01:36,060 --> 00:01:38,750 Gives him a little edge. 21 00:01:38,750 --> 00:01:41,930 Otherwise, he'll be flat. 22 00:01:41,930 --> 00:01:47,265 So I recommend it. 23 00:01:54,200 --> 00:01:56,470 It will take, I suppose, a little while to digest that 24 00:01:56,470 --> 00:01:59,020 neural net stuff. 25 00:01:59,020 --> 00:02:03,030 A little richer than usual in mathematics. 26 00:02:03,030 --> 00:02:05,190 So today, we're going to talk about another effort at 27 00:02:05,190 --> 00:02:06,440 mimicking biology. 28 00:02:09,217 --> 00:02:11,550 This is easy stuff. 29 00:02:11,550 --> 00:02:14,460 It's just conceptual. 30 00:02:14,460 --> 00:02:15,890 And you won't see this on the next quiz. 31 00:02:15,890 --> 00:02:17,040 But you will see it on the final. 32 00:02:17,040 --> 00:02:20,650 It's one of those quiz five type problems, where I ask you 33 00:02:20,650 --> 00:02:23,690 questions to see if you were here and awake. 34 00:02:23,690 --> 00:02:28,140 So a typical question might be, Professor Winston is a 35 00:02:28,140 --> 00:02:31,540 creationist, or something like that. 36 00:02:31,540 --> 00:02:34,290 Not too hard to answer. 37 00:02:34,290 --> 00:02:40,970 In any event, if it's been hard to develop a real 38 00:02:40,970 --> 00:02:43,210 understanding of intelligence, occasionally the hope is that 39 00:02:43,210 --> 00:02:48,470 by mimicking biology or mimicking evolution, you can 40 00:02:48,470 --> 00:02:53,640 circumnavigate all the problems. 41 00:02:53,640 --> 00:02:55,550 And one of those kinds of efforts is 42 00:02:55,550 --> 00:02:59,740 ever to imitate evolution. 43 00:02:59,740 --> 00:03:02,290 So we're going to talk today about so-called genetic 44 00:03:02,290 --> 00:03:06,840 algorithms, which are naive attempts 45 00:03:06,840 --> 00:03:09,890 to mimic naive evolution. 46 00:03:09,890 --> 00:03:15,860 Now, I realize that most MIT students have a basic grasp of 47 00:03:15,860 --> 00:03:17,910 sexual reproduction. 48 00:03:17,910 --> 00:03:21,170 But I've found in talking with students that many times, 49 00:03:21,170 --> 00:03:24,270 they're a little fuzzy on some of the details. 50 00:03:24,270 --> 00:03:26,540 So let's start off by reflecting a little bit about 51 00:03:26,540 --> 00:03:29,320 how that works. 52 00:03:29,320 --> 00:03:34,630 So let's see, we need pink and blue. 53 00:03:34,630 --> 00:03:40,990 And here's our cell, and here is its nucleus, and here are 54 00:03:40,990 --> 00:03:42,590 mommy and daddy's chromosomes. 55 00:03:42,590 --> 00:03:45,770 We'll just pretend there's one pair. 56 00:03:45,770 --> 00:03:48,870 Now ordinarily, in ordinary cell 57 00:03:48,870 --> 00:03:53,440 division, you get two cells. 58 00:03:53,440 --> 00:03:57,660 Both have a nucleus, and the process of producing them 59 00:03:57,660 --> 00:04:01,310 involves the duplication of those chromosomes. 60 00:04:01,310 --> 00:04:07,360 And then one pair ends up in each of the child cells, and 61 00:04:07,360 --> 00:04:08,780 that's all there is to that. 62 00:04:08,780 --> 00:04:10,030 That's mitosis. 63 00:04:16,360 --> 00:04:20,850 But then, when we talk about reproduction, it's more 64 00:04:20,850 --> 00:04:25,540 complicated because those chromosomes get all twisted 65 00:04:25,540 --> 00:04:28,520 up, and they break, and they recombine. 66 00:04:28,520 --> 00:04:32,330 So when we talk about the cells that split off from one 67 00:04:32,330 --> 00:04:39,440 of these germ cells, it's no longer appropriate to talk 68 00:04:39,440 --> 00:04:42,850 about the pink one and the blue one, because the pink one 69 00:04:42,850 --> 00:04:45,280 and the blue one are all mixed up. 70 00:04:45,280 --> 00:04:49,640 So you get two chromosomes here and two here. 71 00:04:49,640 --> 00:04:52,920 But through some miracle of nature, which has always 72 00:04:52,920 --> 00:04:58,850 amazed me, these two cells split, in turn, into four 73 00:04:58,850 --> 00:05:00,100 cells altogether. 74 00:05:05,340 --> 00:05:10,180 And each of those four cells at the bottom gets one of the 75 00:05:10,180 --> 00:05:13,400 chromosomes that was produced by the twisting up of the rope 76 00:05:13,400 --> 00:05:16,250 and recombination. 77 00:05:16,250 --> 00:05:23,090 Then, along comes a special occasion. 78 00:05:23,090 --> 00:05:29,390 And now we can think of this as being a blue one and this 79 00:05:29,390 --> 00:05:31,480 as being a pink one. 80 00:05:31,480 --> 00:05:38,440 And they come together, and you get a new person, like so. 81 00:05:38,440 --> 00:05:42,060 Note that your mother and father's chromosomes are 82 00:05:42,060 --> 00:05:43,335 never, never recombined. 83 00:05:43,335 --> 00:05:47,700 It's your grandparents chromosomes that recombine. 84 00:05:47,700 --> 00:05:49,040 So that's what it's like. 85 00:05:49,040 --> 00:05:50,730 And the main thing to note about this is-- 86 00:05:50,730 --> 00:05:51,570 well, a couple things. 87 00:05:51,570 --> 00:05:57,260 If you happen to be female, this part of the process-- 88 00:05:57,260 --> 00:05:59,170 this part over here-- 89 00:05:59,170 --> 00:06:01,700 took place before you were born. 90 00:06:01,700 --> 00:06:05,130 If you happen to be male, it's going on right now as we 91 00:06:05,130 --> 00:06:07,400 speak, which probably explains something. 92 00:06:07,400 --> 00:06:10,930 But in any event, it's going on right now. 93 00:06:10,930 --> 00:06:16,330 But whenever it goes on, there are lots of opportunities for 94 00:06:16,330 --> 00:06:18,200 throwing dice. 95 00:06:18,200 --> 00:06:21,890 So God threw all the dice before you were born, if you 96 00:06:21,890 --> 00:06:26,190 happen to be a female, in this part of the process over here. 97 00:06:26,190 --> 00:06:29,100 Then, of course, more dice got thrown here when the decision 98 00:06:29,100 --> 00:06:33,090 was made about which particular cells got to fuse 99 00:06:33,090 --> 00:06:35,280 to form a new individual. 100 00:06:35,280 --> 00:06:38,890 So I want to beat on this idea of lots of choice. 101 00:06:38,890 --> 00:06:42,409 When we talk about genetic algorithms, and we talk about 102 00:06:42,409 --> 00:06:44,409 nature, there are lots of choices in there. 103 00:06:44,409 --> 00:06:47,550 And that means there are lots of choices to intervene, to 104 00:06:47,550 --> 00:06:52,159 screw around, to make things work out the way you want. 105 00:06:52,159 --> 00:06:54,120 But in any event, there we are. 106 00:06:54,120 --> 00:06:58,040 That's the basic idea, and it all starts with chromosomes. 107 00:06:58,040 --> 00:07:00,380 So we could think of implementing something that 108 00:07:00,380 --> 00:07:05,735 imitates that with the ACTG stuff that you learned all 109 00:07:05,735 --> 00:07:07,030 about in Seminar 1. 110 00:07:07,030 --> 00:07:08,480 But we're computer scientists. 111 00:07:08,480 --> 00:07:10,700 We don't like Base 4. 112 00:07:10,700 --> 00:07:11,240 We like Base 2. 113 00:07:11,240 --> 00:07:14,320 So I'm going to just suggest that our chromosomes are 114 00:07:14,320 --> 00:07:17,870 binary in this system that we're going to build. 115 00:07:21,230 --> 00:07:23,590 So that might be a chromosome. 116 00:07:23,590 --> 00:07:26,280 And it doesn't have to be binary. 117 00:07:26,280 --> 00:07:27,890 It can be symbolic for fall I care. 118 00:07:27,890 --> 00:07:32,170 But it's just some string of things that determine how the 119 00:07:32,170 --> 00:07:35,390 ultimate system behaves. 120 00:07:35,390 --> 00:07:38,230 So it all starts out, then, with some of these 121 00:07:38,230 --> 00:07:46,840 chromosomes, some of these simulated chromosomes, 122 00:07:46,840 --> 00:07:51,070 simulated, simplified, and naive. 123 00:07:51,070 --> 00:07:55,570 And there's a population of chromosomes. 124 00:07:55,570 --> 00:07:57,780 The population of chromosomes-- it might be 125 00:07:57,780 --> 00:08:00,010 subject to a little bit of mutation. 126 00:08:00,010 --> 00:08:03,100 That is to say, a zero becomes a one, or a 127 00:08:03,100 --> 00:08:05,430 one becomes a zero. 128 00:08:05,430 --> 00:08:06,170 That happens a lot. 129 00:08:06,170 --> 00:08:08,340 That mutation stuff happens over here when things get 130 00:08:08,340 --> 00:08:09,940 twisted up and recombined. 131 00:08:09,940 --> 00:08:12,020 There are copying errors and stuff. 132 00:08:12,020 --> 00:08:13,600 Cosmic rays hit it all the time. 133 00:08:13,600 --> 00:08:16,060 All sorts of reasons why there might be a 134 00:08:16,060 --> 00:08:18,090 single point of change. 135 00:08:18,090 --> 00:08:21,970 That produces the mutation effect. 136 00:08:21,970 --> 00:08:32,360 So here, we have a population that starts off over here. 137 00:08:32,360 --> 00:08:34,929 And some of those things are subject to mutation. 138 00:08:37,976 --> 00:08:41,429 And you'll note, here's a whole bunch of choice already. 139 00:08:41,429 --> 00:08:44,250 How many of these mutations do you allow per 140 00:08:44,250 --> 00:08:48,330 chromosome, for example? 141 00:08:48,330 --> 00:08:50,110 How many of the chromosomes just slip 142 00:08:50,110 --> 00:08:51,280 through without any mutation? 143 00:08:51,280 --> 00:08:53,640 Those are choices you can make. 144 00:08:53,640 --> 00:08:55,820 Once you've made those choices, then we have the 145 00:08:55,820 --> 00:08:57,070 crossover phenomenon. 146 00:09:01,990 --> 00:09:05,600 Let's identify one of these guys as the pink one and one 147 00:09:05,600 --> 00:09:08,820 of these guys as the blue one. 148 00:09:08,820 --> 00:09:13,450 And so now we have the pink one cruised along as well as 149 00:09:13,450 --> 00:09:14,730 the blue one. 150 00:09:14,730 --> 00:09:17,620 The pink one and the blue one cross and produce a new 151 00:09:17,620 --> 00:09:20,820 chromosome, just like in nature. 152 00:09:20,820 --> 00:09:23,720 So we take the front part of one, back part of the other, 153 00:09:23,720 --> 00:09:26,760 and we fuse them together. 154 00:09:26,760 --> 00:09:32,470 And some may slip by without any of that, like so. 155 00:09:32,470 --> 00:09:35,935 Well, these things are meant to be combined in pairs, like 156 00:09:35,935 --> 00:09:39,060 so, but they may not have any crossover in them. 157 00:09:39,060 --> 00:09:40,400 So you have another set of choices. 158 00:09:40,400 --> 00:09:44,790 How many crossovers do you allow per recombination? 159 00:09:44,790 --> 00:09:47,190 You get another set of choices. 160 00:09:47,190 --> 00:09:53,280 So now we've got a population of modified chromosomes 161 00:09:53,280 --> 00:09:54,630 through mutation and crossover. 162 00:09:54,630 --> 00:09:59,040 So the next thing to do is we have the genotype to phenotype 163 00:09:59,040 --> 00:10:00,520 transition. 164 00:10:00,520 --> 00:10:03,660 That is to say the chromosome determines the individual. 165 00:10:03,660 --> 00:10:05,250 It may be a person. 166 00:10:05,250 --> 00:10:06,030 It may be a cow. 167 00:10:06,030 --> 00:10:07,290 It may be a computer program. 168 00:10:07,290 --> 00:10:08,830 I don't care what. 169 00:10:08,830 --> 00:10:12,150 But that code down there has to be 170 00:10:12,150 --> 00:10:15,670 interpreted to be a something. 171 00:10:15,670 --> 00:10:19,030 So it is the genotype, and it has to be interpreted to be 172 00:10:19,030 --> 00:10:23,870 something which is the phenotype, the thing that the 173 00:10:23,870 --> 00:10:26,810 stuff down there is encoding for. 174 00:10:26,810 --> 00:10:29,955 So here, we have a bunch of individuals. 175 00:10:32,840 --> 00:10:35,030 Now, each of those individuals, because they have 176 00:10:35,030 --> 00:10:38,230 varying chromosomal composition, will have a 177 00:10:38,230 --> 00:10:39,480 different fitness. 178 00:10:42,380 --> 00:10:44,890 So these fitnesses might be-- 179 00:10:44,890 --> 00:10:46,990 well, who knows how they might be scored. 180 00:10:46,990 --> 00:10:48,480 But we're computer scientists. 181 00:10:48,480 --> 00:10:49,570 We might as well use numbers. 182 00:10:49,570 --> 00:10:52,970 So maybe this guy's fitness is 88, and this guy's fitness is 183 00:10:52,970 --> 00:10:57,400 77, and so on. 184 00:10:57,400 --> 00:10:58,650 So now that we've got fitness-- 185 00:11:01,290 --> 00:11:03,280 by the way, notice all the choices involved there-- 186 00:11:03,280 --> 00:11:06,410 choice of how you interpret the genotype, choice about how 187 00:11:06,410 --> 00:11:09,610 the phenotype produces the fitness. 188 00:11:09,610 --> 00:11:13,480 And now we have a choice about how the fitness produces a 189 00:11:13,480 --> 00:11:21,840 probability, like 0.8 and 0.1, or something like that-- 190 00:11:21,840 --> 00:11:25,500 probability of survival into the next generation. 191 00:11:25,500 --> 00:11:27,590 So now, once we've got those probabilities, we actually 192 00:11:27,590 --> 00:11:28,840 have selection. 193 00:11:32,250 --> 00:11:36,620 And those phenotypes out there produce genotypes, a new set 194 00:11:36,620 --> 00:11:42,890 of chromosomes, and that completes our loop that goes 195 00:11:42,890 --> 00:11:43,830 back in there. 196 00:11:43,830 --> 00:11:45,630 And so that's the new generation. 197 00:11:51,650 --> 00:11:54,440 Sounds simple. 198 00:11:54,440 --> 00:11:56,960 So if you're going to make this work, of course, you have 199 00:11:56,960 --> 00:11:59,450 a million choices, as I'm going to emphasize over and 200 00:11:59,450 --> 00:11:59,950 over again. 201 00:11:59,950 --> 00:12:05,220 And one of your choices is, for example, how do you 202 00:12:05,220 --> 00:12:07,380 compute the probability of survival to the next 203 00:12:07,380 --> 00:12:10,440 generation given the fitness? 204 00:12:10,440 --> 00:12:13,740 So we have to go, somehow, from numbers like these to 205 00:12:13,740 --> 00:12:16,830 probabilities like those. 206 00:12:16,830 --> 00:12:20,270 So I'm going to talk about several ways of doing it. 207 00:12:20,270 --> 00:12:21,920 None of them are magic. 208 00:12:21,920 --> 00:12:24,420 None of them was specified and stipulated by God 209 00:12:24,420 --> 00:12:26,110 as the right way. 210 00:12:26,110 --> 00:12:28,630 But they have increasingly good properties with respect 211 00:12:28,630 --> 00:12:31,950 to this kind of processing. 212 00:12:31,950 --> 00:12:34,540 So the simplest thing you can do-- 213 00:12:34,540 --> 00:12:38,340 idea number one for computing the-- 214 00:12:38,340 --> 00:12:42,200 see, what you do is you get this whole bag of individuals, 215 00:12:42,200 --> 00:12:44,370 and you have to decide who's going to survive to the next 216 00:12:44,370 --> 00:12:45,800 generation. 217 00:12:45,800 --> 00:12:49,420 So at each step, everything in the bank has a probability of 218 00:12:49,420 --> 00:12:52,370 being the one you pick out and put in the next generation. 219 00:12:52,370 --> 00:12:55,480 So at any step, the sum of the probabilities for each of 220 00:12:55,480 --> 00:12:58,330 those guys is 1, because that's how 221 00:12:58,330 --> 00:12:59,330 high probability works. 222 00:12:59,330 --> 00:13:03,140 The probability of a complete set, added all up, is 223 00:13:03,140 --> 00:13:05,700 probability of 1. 224 00:13:05,700 --> 00:13:09,370 So one thing you can do is you can say that the probability 225 00:13:09,370 --> 00:13:14,590 that you're going to draw individual i is equal to, or 226 00:13:14,590 --> 00:13:18,710 maybe is proportional to, the fitness of that individual. 227 00:13:25,090 --> 00:13:26,600 I haven't completed the expression, so it's not a 228 00:13:26,600 --> 00:13:31,010 probability yet, because some piece of it won't add up to 1. 229 00:13:31,010 --> 00:13:34,370 How can I ensure that it will add up to 1? 230 00:13:34,370 --> 00:13:34,940 That's easy. 231 00:13:34,940 --> 00:13:36,280 Right. 232 00:13:36,280 --> 00:13:40,140 All I have to do is divide by the sum of the 233 00:13:40,140 --> 00:13:43,180 fitnesses over i. 234 00:13:43,180 --> 00:13:46,260 So there's a probability measure that's produced from 235 00:13:46,260 --> 00:13:46,690 the fitnesses. 236 00:13:46,690 --> 00:13:47,602 Yeah. 237 00:13:47,602 --> 00:13:48,970 STUDENT: You need to make sure that the 238 00:13:48,970 --> 00:13:50,338 fitnesses aren't negative. 239 00:13:50,338 --> 00:13:51,780 PATRICK WINSTON: Have to make sure the fitnesses are what? 240 00:13:51,780 --> 00:13:52,712 STUDENT: Aren't negative. 241 00:13:52,712 --> 00:13:54,600 PATRICK WINSTON: He says I have to make sure the 242 00:13:54,600 --> 00:13:55,350 fitnesses aren't negative. 243 00:13:55,350 --> 00:13:59,020 Yeah, it would be embarrassing if they were. 244 00:13:59,020 --> 00:14:01,465 So we'll just [? strike ?] anything like that as 0. 245 00:14:01,465 --> 00:14:04,730 You've got a lot of choice how you can calculate the fitness. 246 00:14:04,730 --> 00:14:06,330 And maybe you will produce negative numbers, in which 247 00:14:06,330 --> 00:14:08,964 case you have to think a little bit more about it. 248 00:14:11,570 --> 00:14:13,410 So now, what about an example? 249 00:14:13,410 --> 00:14:16,880 Well, I'm going to show you an example. 250 00:14:16,880 --> 00:14:19,450 Why don't I show you the example. 251 00:14:19,450 --> 00:14:21,840 What we're going to do is we're going to have a genetic 252 00:14:21,840 --> 00:14:24,615 algorithm that looks for an optimal value in a space. 253 00:14:27,870 --> 00:14:29,836 And there's the space. 254 00:14:29,836 --> 00:14:32,856 Now, you'll notice it's a bunch of contour lines, a 255 00:14:32,856 --> 00:14:34,840 bunch of hills in that space. 256 00:14:34,840 --> 00:14:38,170 Let me show you how that space was produced. 257 00:14:38,170 --> 00:14:44,250 The fitness is a function of x and y, and it's equal to the 258 00:14:44,250 --> 00:14:50,890 sine of some constant times x, quantity squared, times the 259 00:14:50,890 --> 00:14:57,540 sine of some constant y, quantity squared, e to the 260 00:14:57,540 --> 00:15:01,910 plus x plus y divided by some constant. 261 00:15:01,910 --> 00:15:05,830 So sigma and omega there are just in there so that it kind 262 00:15:05,830 --> 00:15:09,890 of makes a nice picture for demonstration. 263 00:15:09,890 --> 00:15:10,790 So there's a space. 264 00:15:10,790 --> 00:15:12,930 And clearly, where you want to be in this space is in the 265 00:15:12,930 --> 00:15:14,180 upper right-hand corner. 266 00:15:16,670 --> 00:15:18,160 That's the optimal value. 267 00:15:18,160 --> 00:15:19,570 But we have a genetic algorithm that 268 00:15:19,570 --> 00:15:21,300 doesn't know anything. 269 00:15:21,300 --> 00:15:26,950 All it knows how to do is mutate and cross over. 270 00:15:26,950 --> 00:15:30,130 So it's going to start off with a population of 1. 271 00:15:30,130 --> 00:15:33,130 It's a little red dot down in the lower left. 272 00:15:33,130 --> 00:15:36,080 So here's how it's going to evolve. 273 00:15:36,080 --> 00:15:39,580 There's going to be s chromosome consisting of two 274 00:15:39,580 --> 00:15:44,380 numbers, an x number and a y number, like, 275 00:15:44,380 --> 00:15:47,450 say, 0.3 and 0.7. 276 00:15:47,450 --> 00:15:54,080 Here's another one, which might be 0.6 and 0.2. 277 00:15:54,080 --> 00:15:56,160 So the mutation operator is going to take one of those 278 00:15:56,160 --> 00:15:58,720 values and change it a little bit. 279 00:15:58,720 --> 00:16:03,980 So it might say, well, we'll take 3, and we'll make it 0.2. 280 00:16:03,980 --> 00:16:07,090 And the crossover operation is going to exchange the x and y 281 00:16:07,090 --> 00:16:08,850 values of pairs. 282 00:16:08,850 --> 00:16:12,180 So if we have a crossover here, then what we're going to 283 00:16:12,180 --> 00:16:16,060 get out from this one-- 284 00:16:16,060 --> 00:16:18,390 well, we're going to get out a combination of these two. 285 00:16:22,970 --> 00:16:31,300 And it's going to look like this. 286 00:16:31,300 --> 00:16:33,740 Because what we're going to do is we're going to take the x 287 00:16:33,740 --> 00:16:37,120 value of 1 and combine it with the y value of the other one. 288 00:16:37,120 --> 00:16:38,570 So this is going to be 0.2-- 289 00:16:38,570 --> 00:16:42,050 my mutated value and 0.2-- 290 00:16:42,050 --> 00:16:46,420 and this is going to be 0.6 and 0.7. 291 00:16:46,420 --> 00:16:49,300 So that's how my little genetic algorithm heck is 292 00:16:49,300 --> 00:16:51,860 going to work. 293 00:16:51,860 --> 00:16:56,090 So having coded this up, we can now see how it flows. 294 00:17:01,000 --> 00:17:04,460 Let's run it 10 generations. 295 00:17:04,460 --> 00:17:07,800 So the population is rapidly expanded to some fixed limit. 296 00:17:07,800 --> 00:17:08,440 I forgot what it is-- 297 00:17:08,440 --> 00:17:10,440 30 or so. 298 00:17:10,440 --> 00:17:15,260 And we can run that 100 generations. 299 00:17:15,260 --> 00:17:18,540 And so this seems to be getting stuck, kind of, right? 300 00:17:18,540 --> 00:17:20,319 So what's the problem? 301 00:17:20,319 --> 00:17:21,858 The problem is local maxima. 302 00:17:21,858 --> 00:17:30,950 This is fundamentally a hill-climbing mechanism. 303 00:17:30,950 --> 00:17:33,350 Note that I have not included any crossover so far. 304 00:17:36,210 --> 00:17:39,345 So if I do have crossover, then if I've got a good x 305 00:17:39,345 --> 00:17:44,050 value and a good y value, I can cross them over and get 306 00:17:44,050 --> 00:17:48,880 them both in the same situation. 307 00:17:48,880 --> 00:17:50,470 But nevertheless, this thing doesn't seem to be 308 00:17:50,470 --> 00:17:51,783 working very well. 309 00:17:51,783 --> 00:17:53,635 STUDENT: Professor, I have a question. 310 00:17:53,635 --> 00:17:54,561 PATRICK WINSTON: Yeah. 311 00:17:54,561 --> 00:17:56,185 STUDENT: That picture is just the contour 312 00:17:56,185 --> 00:17:56,660 lines of that function. 313 00:17:56,660 --> 00:17:58,570 PATRICK WINSTON: The contour lines of that function. 314 00:17:58,570 --> 00:18:00,710 So the reason you see a lot of contour lines in the upper 315 00:18:00,710 --> 00:18:02,680 right is because it gets much higher because there's that 316 00:18:02,680 --> 00:18:04,290 exponential term that increases as you 317 00:18:04,290 --> 00:18:06,430 go up to the right. 318 00:18:06,430 --> 00:18:09,270 So I don't know, it looks-- 319 00:18:09,270 --> 00:18:12,500 let's put some crossover in and repeat the experience. 320 00:18:15,960 --> 00:18:24,100 We'll run 100 generations. 321 00:18:24,100 --> 00:18:24,560 I don't know. 322 00:18:24,560 --> 00:18:27,440 It just doesn't seem to be going anywhere. 323 00:18:27,440 --> 00:18:31,760 Sometimes, it'll go right to the global maximum. 324 00:18:31,760 --> 00:18:33,120 Sometimes it takes a long time. 325 00:18:33,120 --> 00:18:35,320 It's got a random number generator in there, so I have 326 00:18:35,320 --> 00:18:37,610 no control over it. 327 00:18:37,610 --> 00:18:39,520 So it's going to get there. 328 00:18:39,520 --> 00:18:41,330 I couldn't tell whether the crossover was 329 00:18:41,330 --> 00:18:43,236 doing any good or not. 330 00:18:43,236 --> 00:18:47,030 Oh, well, here's one. 331 00:18:47,030 --> 00:18:49,970 Let's make this a little bit more complicated. 332 00:18:49,970 --> 00:18:51,220 Suppose that's the space. 333 00:19:02,780 --> 00:19:04,890 Now it's going to be in real trouble, because it'll never 334 00:19:04,890 --> 00:19:06,140 get across that moat. 335 00:19:09,280 --> 00:19:14,540 You know, you would think that it would climb up to the x 336 00:19:14,540 --> 00:19:16,880 maximum or to the y maximum, but it's not 337 00:19:16,880 --> 00:19:19,990 going to do very well. 338 00:19:19,990 --> 00:19:22,100 Even with crossover, it's just not going to do very well, 339 00:19:22,100 --> 00:19:24,130 because it's climbing up those local hills. 340 00:19:24,130 --> 00:19:27,220 Anybody got an idea about one simple thing we could do to 341 00:19:27,220 --> 00:19:30,100 make it work better? 342 00:19:30,100 --> 00:19:34,860 Yeah, you could increase step size, right? 343 00:19:34,860 --> 00:19:36,110 Let me you see if that will help. 344 00:19:49,360 --> 00:19:51,750 You know, even that doesn't seem to help. 345 00:19:51,750 --> 00:19:54,695 So we have to conclude-- do we conclude that 346 00:19:54,695 --> 00:19:55,520 this is a bad idea? 347 00:19:55,520 --> 00:19:58,170 Well, we don't have to conclude it's a bad idea yet, 348 00:19:58,170 --> 00:20:04,420 because we may just look at it and ask why five times. 349 00:20:04,420 --> 00:20:09,050 And we might ask, well, maybe we can get a better mechanism 350 00:20:09,050 --> 00:20:11,600 in there to translate fitness into probability of survival. 351 00:20:14,540 --> 00:20:17,480 Using this formula is kind of strange, anyway, because 352 00:20:17,480 --> 00:20:20,910 suppose temperature is one of your fitness characteristics. 353 00:20:20,910 --> 00:20:22,520 The hotter, the better. 354 00:20:22,520 --> 00:20:25,310 Then the ratio of the probability that you'll 355 00:20:25,310 --> 00:20:30,730 survive versus the person next to you, that ratio will depend 356 00:20:30,730 --> 00:20:32,460 on whether you're measuring the temperature in Celsius or 357 00:20:32,460 --> 00:20:35,680 Fahrenheit, right? 358 00:20:35,680 --> 00:20:38,820 Because you've shifted the origin that shifts the ratio 359 00:20:38,820 --> 00:20:41,910 that shifts the probability of success. 360 00:20:41,910 --> 00:20:44,230 So it seems kind of strange to just take these things right 361 00:20:44,230 --> 00:20:46,550 straight into probabilities. 362 00:20:46,550 --> 00:20:47,820 So a better idea-- 363 00:20:47,820 --> 00:20:49,430 idea number two-- 364 00:20:49,430 --> 00:20:53,150 is to say, well, shoot, maybe we don't care about what the 365 00:20:53,150 --> 00:20:55,730 actual fitnesses are. 366 00:20:55,730 --> 00:20:59,160 All we really care about is the rank order of all the 367 00:20:59,160 --> 00:21:00,520 candidates. 368 00:21:00,520 --> 00:21:02,870 So the candidate with the most fitness will have the most 369 00:21:02,870 --> 00:21:05,770 probability of getting into the next generation. 370 00:21:05,770 --> 00:21:08,460 The candidate with the second-most fitness will have 371 00:21:08,460 --> 00:21:11,880 the second-highest probability, and so on. 372 00:21:11,880 --> 00:21:14,450 But we're not going to use the actual fitnesses themselves to 373 00:21:14,450 --> 00:21:16,530 make the determination. 374 00:21:16,530 --> 00:21:17,750 Instead, what we're going to do with this 375 00:21:17,750 --> 00:21:18,980 mechanism number two-- 376 00:21:18,980 --> 00:21:20,390 this is the rank space method-- 377 00:21:28,670 --> 00:21:29,370 is this. 378 00:21:29,370 --> 00:21:31,690 We're going to say that the probability of the 379 00:21:31,690 --> 00:21:35,500 highest-ranking individual of getting into the next 380 00:21:35,500 --> 00:21:39,465 generation is some constant P sub c, which, of course, you 381 00:21:39,465 --> 00:21:39,720 can select. 382 00:21:39,720 --> 00:21:42,350 You have another choice. 383 00:21:42,350 --> 00:21:46,820 Then, if that guy doesn't get selected, the probability of 384 00:21:46,820 --> 00:21:49,490 the second-highest-ranking individual getting in the next 385 00:21:49,490 --> 00:21:51,730 generation is going to be the probability that that guy 386 00:21:51,730 --> 00:21:53,620 didn't get in there. 387 00:21:53,620 --> 00:21:59,870 That's 1 minus P sub c times the same probability constant. 388 00:21:59,870 --> 00:22:02,360 And so you can see how this is going. 389 00:22:02,360 --> 00:22:10,220 P3 will be equal to 1 minus P sub c squared terms P sub c. 390 00:22:10,220 --> 00:22:18,130 P sub n minus 1 will be equal to 1 minus that probability 391 00:22:18,130 --> 00:22:22,340 constant to the n minus-- 392 00:22:22,340 --> 00:22:27,370 n minus 2 times P sub c. 393 00:22:27,370 --> 00:22:31,000 And then there's only one individual left. 394 00:22:31,000 --> 00:22:33,030 And then if you got through all these guys and haven't got 395 00:22:33,030 --> 00:22:37,260 anybody selected, then you've got to select the last guy. 396 00:22:37,260 --> 00:22:40,640 And so the probability you're going to select the last guy 397 00:22:40,640 --> 00:22:47,160 is going to be 1 minus P sub c to the n minus 1. 398 00:22:47,160 --> 00:22:51,140 So it's a probability you've missed all those guys in the 399 00:22:51,140 --> 00:22:53,300 first n minus 1 choices. 400 00:22:53,300 --> 00:22:55,010 Yeah, it is, honest to God. 401 00:22:55,010 --> 00:22:58,970 See, this is the probability that this last guys is going 402 00:22:58,970 --> 00:22:59,580 to get selected. 403 00:22:59,580 --> 00:23:02,160 It's not the probability that it's the last guy getting 404 00:23:02,160 --> 00:23:05,970 selected, given that the others haven't been select. 405 00:23:05,970 --> 00:23:08,650 Trust me, it's right. 406 00:23:08,650 --> 00:23:09,720 Are you thinking it ought to be 1? 407 00:23:09,720 --> 00:23:10,592 STUDENT: What? 408 00:23:10,592 --> 00:23:13,080 PATRICK WINSTON: Were you thinking it ought to be 1? 409 00:23:13,080 --> 00:23:17,326 STUDENT: No, I was thinking that I was wondering why you 410 00:23:17,326 --> 00:23:19,460 were re-rolling the dice, so to speak. 411 00:23:19,460 --> 00:23:20,700 PATRICK WINSTON: You are re-rolling the dice. 412 00:23:20,700 --> 00:23:22,930 You've got a probability each time, except for the last 413 00:23:22,930 --> 00:23:24,630 time, when, of course, you have to take it. 414 00:23:24,630 --> 00:23:25,200 There's nothing left. 415 00:23:25,200 --> 00:23:26,050 There's no other choice. 416 00:23:26,050 --> 00:23:27,130 STUDENT: I have a question. 417 00:23:27,130 --> 00:23:28,424 PATRICK WINSTON: Yeah, [INAUDIBLE]. 418 00:23:28,424 --> 00:23:33,244 STUDENT: So when you jump from [INAUDIBLE], that makes sense. 419 00:23:33,244 --> 00:23:35,670 [INAUDIBLE] you're saying [INAUDIBLE]. 420 00:23:35,670 --> 00:23:37,255 PATRICK WINSTON: It's the probability [INAUDIBLE] the 421 00:23:37,255 --> 00:23:38,421 first two choices. 422 00:23:38,421 --> 00:23:39,837 STUDENT: Yeah, but the second choice had probability one 423 00:23:39,837 --> 00:23:41,570 minus P sub c times P sub c, not-- 424 00:23:41,570 --> 00:23:43,410 PATRICK WINSTON: Think about it this way. 425 00:23:43,410 --> 00:23:46,030 It's the probability you didn't choose the first two. 426 00:23:46,030 --> 00:23:47,820 So the probability you didn't choose the first one is one 427 00:23:47,820 --> 00:23:49,130 minus P sub c. 428 00:23:49,130 --> 00:23:51,460 The probability you didn't choose the next one, as well, 429 00:23:51,460 --> 00:23:53,596 because you're choosing that next one with probability P 430 00:23:53,596 --> 00:23:56,385 sub c, it's the square of it. 431 00:23:59,180 --> 00:24:00,760 So that might work better. 432 00:24:00,760 --> 00:24:03,220 Let's give it a shot. 433 00:24:03,220 --> 00:24:11,170 Let's go back to our original space choice, and we set and 434 00:24:11,170 --> 00:24:14,800 switch to the rank fitness method. 435 00:24:14,800 --> 00:24:19,200 And we'll run out 100 generations. 436 00:24:19,200 --> 00:24:21,680 Whoa! 437 00:24:21,680 --> 00:24:22,270 What happened there? 438 00:24:22,270 --> 00:24:23,360 That was pretty fast. 439 00:24:23,360 --> 00:24:24,850 Maybe I used a big step size. 440 00:24:28,714 --> 00:24:30,150 Yeah, that's a little bit more reasonable. 441 00:24:30,150 --> 00:24:30,890 Oops-- 442 00:24:30,890 --> 00:24:33,150 what happened? 443 00:24:33,150 --> 00:24:34,910 It's really getting stuck on a local maximum. 444 00:24:39,560 --> 00:24:44,370 So evidently, I've choosed a constant P sub c such that it 445 00:24:44,370 --> 00:24:48,340 just drove it right up the nearest hill. 446 00:24:48,340 --> 00:24:50,960 On the other hand, if I change the step size a little bit, 447 00:24:50,960 --> 00:24:53,690 maybe I can get it to spread out. 448 00:24:53,690 --> 00:24:55,170 I sure did. 449 00:24:55,170 --> 00:24:58,800 And now that it's managed to evolve over there to find the 450 00:24:58,800 --> 00:25:05,020 maximum value, now I can clamp down on the step size again. 451 00:25:05,020 --> 00:25:06,730 And now it shows no more diversity. 452 00:25:06,730 --> 00:25:10,190 It's just locked on to that global maximum. 453 00:25:10,190 --> 00:25:13,450 So this is not unlike what evolution sometimes does. 454 00:25:16,190 --> 00:25:21,070 Sometimes, species collapse into a state where they don't 455 00:25:21,070 --> 00:25:23,420 change for 500 million or 600 million years, like sharks, 456 00:25:23,420 --> 00:25:24,670 for example. 457 00:25:28,120 --> 00:25:30,780 Sometimes, they only survive if they've got a lot of 458 00:25:30,780 --> 00:25:34,780 diversity built into their way of life so that they can 459 00:25:34,780 --> 00:25:37,750 adjust to habitat changes. 460 00:25:37,750 --> 00:25:40,250 Now, when you increase the step size, because you're 461 00:25:40,250 --> 00:25:44,245 stuck on a local maximum, it's like heating up a metal. 462 00:25:44,245 --> 00:25:46,610 You make everything kind of vibrate 463 00:25:46,610 --> 00:25:49,230 more, make bigger steps. 464 00:25:49,230 --> 00:25:53,100 So this kind of process, where you may start with a big step 465 00:25:53,100 --> 00:25:57,150 size and then gradually reduce the step size, is called 466 00:25:57,150 --> 00:26:00,480 simulated annealing, because it's like letting 467 00:26:00,480 --> 00:26:02,000 a metal cool down. 468 00:26:02,000 --> 00:26:04,810 So you start off with a big temperature-- big step size-- 469 00:26:04,810 --> 00:26:06,940 that covers the space. 470 00:26:06,940 --> 00:26:10,400 And then you slowly reduce the step size, so you actually 471 00:26:10,400 --> 00:26:15,840 crawl up to the local maxima that are available. 472 00:26:15,840 --> 00:26:16,960 So that seemed to work pretty well. 473 00:26:16,960 --> 00:26:19,880 Let's see if we can get it to work on the harder 474 00:26:19,880 --> 00:26:21,130 problem of the moat. 475 00:26:32,890 --> 00:26:34,820 So it's not doing very well. 476 00:26:34,820 --> 00:26:36,270 Better increase the step size. 477 00:26:41,260 --> 00:26:42,510 No, it's still kind of stuck. 478 00:26:45,030 --> 00:26:47,620 Even though it's got the capacity to cross over, it's 479 00:26:47,620 --> 00:26:50,010 so stuck on that lower right-hand corner, it can't 480 00:26:50,010 --> 00:26:53,090 get up that vertical branch to get to a point where a 481 00:26:53,090 --> 00:26:56,440 crossover will produce a value up there in the upper 482 00:26:56,440 --> 00:26:57,040 right-hand corner. 483 00:26:57,040 --> 00:26:58,445 So we're still not home yet. 484 00:27:03,570 --> 00:27:05,350 So what's the trouble? 485 00:27:05,350 --> 00:27:10,390 The trouble is that the fitness mechanism is just 486 00:27:10,390 --> 00:27:14,090 driving things up to the local maximum. 487 00:27:14,090 --> 00:27:16,025 It's just terribly unfortunate. 488 00:27:20,840 --> 00:27:21,510 What to do? 489 00:27:21,510 --> 00:27:22,690 Well, here's something you could do. 490 00:27:22,690 --> 00:27:26,330 You can say, well, if the problem is we've lost the 491 00:27:26,330 --> 00:27:29,960 diversity in our population, then we can measure the 492 00:27:29,960 --> 00:27:31,060 diversity-- 493 00:27:31,060 --> 00:27:33,550 not only the fitness of the set of individuals we're 494 00:27:33,550 --> 00:27:37,510 selecting from, but we can measure how different they are 495 00:27:37,510 --> 00:27:39,600 on the individuals we've already selected for the next 496 00:27:39,600 --> 00:27:41,750 population. 497 00:27:41,750 --> 00:27:44,890 In other words, we can get a diverse population as well as 498 00:27:44,890 --> 00:27:47,530 a fit population if, when we make our selection, we 499 00:27:47,530 --> 00:27:50,490 consider not only their fitness but how different they 500 00:27:50,490 --> 00:27:54,290 are from the individuals that have already been selected. 501 00:27:54,290 --> 00:27:56,250 So that's going to be mechanism number three. 502 00:28:07,410 --> 00:28:13,060 So now we have a space, and we can measure 503 00:28:13,060 --> 00:28:14,620 fitness along one axis-- 504 00:28:14,620 --> 00:28:17,620 ordinary fitness-- 505 00:28:17,620 --> 00:28:19,380 and this is rank space fitness, so that's 506 00:28:19,380 --> 00:28:21,026 going to be P sub c. 507 00:28:21,026 --> 00:28:24,150 There will always be some individual 508 00:28:24,150 --> 00:28:25,520 with the highest fitness. 509 00:28:31,680 --> 00:28:32,600 And over here-- 510 00:28:32,600 --> 00:28:36,050 that might not be P sub c, actually. 511 00:28:36,050 --> 00:28:37,900 But there'll be some individual with a maximal 512 00:28:37,900 --> 00:28:41,820 fitness, and at any given step in the selection of the next 513 00:28:41,820 --> 00:28:44,750 population, there'll be some individual that's maximally 514 00:28:44,750 --> 00:28:50,590 diverse from all of the individuals that have been 515 00:28:50,590 --> 00:28:55,300 selected for the next generation so far. 516 00:28:55,300 --> 00:28:58,120 So what kind of individual would you like to pick for the 517 00:28:58,120 --> 00:29:00,470 next generation? 518 00:29:00,470 --> 00:29:04,620 Well, the one with the highest fitness rank and the one with 519 00:29:04,620 --> 00:29:07,250 the highest diversity rank. 520 00:29:07,250 --> 00:29:11,340 So what you'd really like is you'd like to have somebody 521 00:29:11,340 --> 00:29:12,590 right there. 522 00:29:14,500 --> 00:29:16,830 And if you can't have somebody right there, if there's nobody 523 00:29:16,830 --> 00:29:21,200 right there with a maximum fitness, a maximum diversity 524 00:29:21,200 --> 00:29:26,090 at the same time, then maybe you can draw in iso-goodness 525 00:29:26,090 --> 00:29:29,220 lines, like so, which are just how far you 526 00:29:29,220 --> 00:29:30,470 are from that ideal. 527 00:29:32,510 --> 00:29:34,630 So let's summarize. 528 00:29:34,630 --> 00:29:36,140 You've got to pick some individuals for the next 529 00:29:36,140 --> 00:29:37,165 population. 530 00:29:37,165 --> 00:29:40,540 When we pick the first individual, all we've got to 531 00:29:40,540 --> 00:29:43,560 go on is how fit the individual is, because there's 532 00:29:43,560 --> 00:29:45,800 nobody else in that next generation. 533 00:29:45,800 --> 00:29:50,620 After the first individual is selected, then we can look at 534 00:29:50,620 --> 00:29:53,420 our set of candidates, and we can say which candidate would 535 00:29:53,420 --> 00:29:55,670 be more different from the set of things we've already 536 00:29:55,670 --> 00:29:56,850 selected than all the others. 537 00:29:56,850 --> 00:30:00,180 That would get the highest diversity rank and so on down 538 00:30:00,180 --> 00:30:02,150 the candidate list. 539 00:30:02,150 --> 00:30:03,630 So let's see how that might work. 540 00:30:11,330 --> 00:30:14,380 So we're going to use a combination of fitness rank 541 00:30:14,380 --> 00:30:18,990 and diversity rank. 542 00:30:18,990 --> 00:30:21,880 And we'll just use the simple one so far. 543 00:30:21,880 --> 00:30:25,070 We'll use a small step size, and we'll let this run 100 544 00:30:25,070 --> 00:30:26,530 generations to see what happens. 545 00:30:31,570 --> 00:30:33,120 Bingo. 546 00:30:33,120 --> 00:30:35,100 It crawls right up there, because it's trying to keep 547 00:30:35,100 --> 00:30:36,520 itself spread out. 548 00:30:36,520 --> 00:30:39,372 It uses that diversity measurement to do that. 549 00:30:39,372 --> 00:30:43,640 And at the same time, it's seeking high fitness, so 550 00:30:43,640 --> 00:30:46,250 that's why it's crawling up to the upper right-hand corner. 551 00:30:46,250 --> 00:30:50,170 But in the end, that diversity piece of it is keeping the 552 00:30:50,170 --> 00:30:52,140 things spread out. 553 00:30:52,140 --> 00:30:53,650 So suppose you're a shark or something. 554 00:30:53,650 --> 00:30:56,020 You don't care about diversity anymore, And we could just 555 00:30:56,020 --> 00:30:57,440 turn that off. 556 00:30:57,440 --> 00:30:58,700 Is that thing still running? 557 00:31:03,460 --> 00:31:04,980 Go back to fitness rank-- 558 00:31:04,980 --> 00:31:06,500 bingo. 559 00:31:06,500 --> 00:31:12,270 So there you are-- you're stuck for 600 million years. 560 00:31:12,270 --> 00:31:14,000 So let's see if this will handle the moat problem. 561 00:31:24,420 --> 00:31:27,870 See, our step size is still small. 562 00:31:27,870 --> 00:31:31,110 We'll just let this run. 563 00:31:31,110 --> 00:31:34,837 So the diversity of P sub is keeping it spread out, pretty 564 00:31:34,837 --> 00:31:37,020 soon, bingo, it's right in there. 565 00:31:37,020 --> 00:31:39,350 It's across that big moat, because it's got the crossover 566 00:31:39,350 --> 00:31:41,740 mechanism that combines the best of the x's and the best 567 00:31:41,740 --> 00:31:43,570 of the y's. 568 00:31:43,570 --> 00:31:47,830 So that seems to work pretty well. 569 00:31:47,830 --> 00:31:51,400 OK, so see, these are some of the things that you can think 570 00:31:51,400 --> 00:31:54,570 about when you're thinking-- oh, and of course, we're a 571 00:31:54,570 --> 00:31:56,830 shark, we're going to forget about diversity. 572 00:31:56,830 --> 00:31:59,950 We'll change the selection method from fitness and 573 00:31:59,950 --> 00:32:01,245 diversity rank to just diversity. 574 00:32:01,245 --> 00:32:04,060 It collapses down on to the highest hill. 575 00:32:07,650 --> 00:32:09,225 Yeah, [INAUDIBLE], what? 576 00:32:09,225 --> 00:32:12,865 STUDENT: How does step size translate into mutations? 577 00:32:12,865 --> 00:32:15,090 PATRICK WINSTON: Oh, just the-- 578 00:32:15,090 --> 00:32:19,250 question is, how does step size translate into mutation? 579 00:32:19,250 --> 00:32:22,820 Instead of allowing myself to take steps as big as 1/10, I 580 00:32:22,820 --> 00:32:27,680 might allow myself to take steps as big as 3/10, 581 00:32:27,680 --> 00:32:29,190 according to some distribution. 582 00:32:32,430 --> 00:32:33,680 So what to say about all this? 583 00:32:36,630 --> 00:32:40,000 It's very seductive, because it's nature, right? 584 00:32:40,000 --> 00:32:42,610 The trouble is, it's naive nature. 585 00:32:42,610 --> 00:32:49,280 And as evolutionary theories go, this is horrible. 586 00:32:49,280 --> 00:32:51,380 This is naive. 587 00:32:51,380 --> 00:32:53,350 So we'd like to use real evolutionary theory, except we 588 00:32:53,350 --> 00:32:55,360 don't have real evolutionary theory. 589 00:32:55,360 --> 00:32:57,780 Evolution is still a mystery. 590 00:32:57,780 --> 00:33:00,170 Some things are pretty obvious. 591 00:33:00,170 --> 00:33:02,490 You can breed fast race horses. 592 00:33:02,490 --> 00:33:04,450 That works just like so. 593 00:33:04,450 --> 00:33:07,200 The trouble is, we don't have any real good idea about how 594 00:33:07,200 --> 00:33:12,150 speciation takes place and how a lot of evolution works, 595 00:33:12,150 --> 00:33:15,480 because all these chromosomes are connected to their 596 00:33:15,480 --> 00:33:19,280 phenotype consequences in very complicated ways that nobody 597 00:33:19,280 --> 00:33:21,110 fully understands. 598 00:33:21,110 --> 00:33:23,610 So there's a great deal of magic in that genotype to 599 00:33:23,610 --> 00:33:26,580 phenotype transition that nobody really 600 00:33:26,580 --> 00:33:28,950 understands very well. 601 00:33:28,950 --> 00:33:34,680 So when people write these programs that are in the style 602 00:33:34,680 --> 00:33:40,270 of so-called genetic algorithm, they're taking a 603 00:33:40,270 --> 00:33:44,720 photograph of high school biology, and they're spending 604 00:33:44,720 --> 00:33:48,920 a long time building programs based on that naive idea. 605 00:33:48,920 --> 00:33:54,430 But that naive idea has lots of places for intervention, 606 00:33:54,430 --> 00:33:58,620 because look at all the things you can screw around with in 607 00:33:58,620 --> 00:34:02,100 that process of going from one generation to the next. 608 00:34:04,671 --> 00:34:07,110 By the way, what does mutation do? 609 00:34:07,110 --> 00:34:09,230 It's basically hill climbing, right? 610 00:34:09,230 --> 00:34:11,310 It's producing a little spread out, and you're using the 611 00:34:11,310 --> 00:34:14,520 fitness thing to climb the hill. 612 00:34:14,520 --> 00:34:16,380 So you get a lot of choices about how you handle that. 613 00:34:16,380 --> 00:34:17,630 Then you get a lot of choices about much 614 00:34:17,630 --> 00:34:18,420 crossover you're doing. 615 00:34:18,420 --> 00:34:20,275 What does crossover do? 616 00:34:20,275 --> 00:34:25,690 It kind of combines strong features of multiple 617 00:34:25,690 --> 00:34:27,469 individuals into one individual, maybe. 618 00:34:30,310 --> 00:34:32,139 So you've got all kinds of choices there. 619 00:34:32,139 --> 00:34:34,120 And then your genotype to phenotype translation-- 620 00:34:34,120 --> 00:34:38,040 how do you interpret something like those zeroes and ones as 621 00:34:38,040 --> 00:34:40,480 an if-then rule, for example, as something that's in the 622 00:34:40,480 --> 00:34:42,969 hands of the designer? 623 00:34:42,969 --> 00:34:46,719 Then you've got all the rest of those things, all which are 624 00:34:46,719 --> 00:34:48,940 left up to the designer. 625 00:34:48,940 --> 00:34:51,170 So in the end, you really have to ask-- 626 00:34:51,170 --> 00:34:52,780 when you see an impressive demonstration, you have to 627 00:34:52,780 --> 00:34:55,179 say, where does the credit lie? 628 00:34:55,179 --> 00:34:58,690 And I mean that pun intentionally, because usually 629 00:34:58,690 --> 00:35:00,810 the people who are claiming the credit are lying about 630 00:35:00,810 --> 00:35:03,380 where it's coming from. 631 00:35:03,380 --> 00:35:05,300 But nevertheless, let me give you a couple of examples of 632 00:35:05,300 --> 00:35:11,730 where this has found actual, bona fide practical 633 00:35:11,730 --> 00:35:13,890 application. 634 00:35:13,890 --> 00:35:16,080 So when you look for practical application, you might say, 635 00:35:16,080 --> 00:35:20,630 well, in what kind of problem does a good front piece 636 00:35:20,630 --> 00:35:22,580 combine with a good back piece to produce 637 00:35:22,580 --> 00:35:25,640 a good thing overall? 638 00:35:25,640 --> 00:35:28,860 And the answer is, when you're making a plan. 639 00:35:28,860 --> 00:35:33,000 So you might have a problem in planning that requires you to 640 00:35:33,000 --> 00:35:35,045 take a series of steps. 641 00:35:39,860 --> 00:35:43,380 And you might have two plans, each of which 642 00:35:43,380 --> 00:35:45,500 is a series of steps. 643 00:35:45,500 --> 00:35:51,470 And you might combine these to produce something new that's 644 00:35:51,470 --> 00:35:56,710 the front half of one and the back half of another. 645 00:35:56,710 --> 00:35:58,680 So that's practical application number one. 646 00:36:01,750 --> 00:36:05,310 And that requires you to interpret your chromosome as 647 00:36:05,310 --> 00:36:09,250 an indicator of the steps in the plan. 648 00:36:09,250 --> 00:36:16,380 Another example is drawn from a [? UROP ?] 649 00:36:16,380 --> 00:36:18,445 project a student did for me some years ago. 650 00:36:21,170 --> 00:36:21,670 He was a freshman. 651 00:36:21,670 --> 00:36:24,235 He came to me and said, I want to do a [? UROP ?] project. 652 00:36:24,235 --> 00:36:26,320 And I said, have you taken 6.034? 653 00:36:26,320 --> 00:36:27,290 And he said no. 654 00:36:27,290 --> 00:36:28,570 And I said, go away. 655 00:36:28,570 --> 00:36:29,900 And he said, I don't want to go away. 656 00:36:29,900 --> 00:36:31,720 I want to do a [? UROP ?] project. 657 00:36:31,720 --> 00:36:34,000 So I said, have you read my book? 658 00:36:34,000 --> 00:36:34,660 He said no. 659 00:36:34,660 --> 00:36:35,800 I said, well, go away, then. 660 00:36:35,800 --> 00:36:37,150 And he said, I don't want to go away. 661 00:36:37,150 --> 00:36:39,030 I want to do a UROP project. 662 00:36:39,030 --> 00:36:40,860 So I said, I don't have any [? UROP ?] projects. 663 00:36:40,860 --> 00:36:41,560 He said, that's OK. 664 00:36:41,560 --> 00:36:42,810 I've got my own. 665 00:36:46,610 --> 00:36:50,310 He's a finance-type guy, so he was interested in whether he 666 00:36:50,310 --> 00:36:54,180 could build a rule-based expert system that could 667 00:36:54,180 --> 00:36:57,680 predict the winners at horse races. 668 00:36:57,680 --> 00:37:02,000 So his rule-based expert system consisted 669 00:37:02,000 --> 00:37:04,750 of rules like this. 670 00:37:04,750 --> 00:37:11,650 If x and y, then some conclusion. 671 00:37:11,650 --> 00:37:19,560 If l and m, then some kind of conclusion. 672 00:37:19,560 --> 00:37:24,770 And from these, he would produce rules like if x 673 00:37:24,770 --> 00:37:29,160 prime-- that's a slightly mutated version of the x 674 00:37:29,160 --> 00:37:31,070 antecedent-- 675 00:37:31,070 --> 00:37:38,080 and m, then some conclusion. 676 00:37:38,080 --> 00:37:39,630 So it's mutation and crossover. 677 00:37:39,630 --> 00:37:43,165 And he was able to produce a system that seemed to work 678 00:37:43,165 --> 00:37:46,580 about as well as the handicappers in the newspaper. 679 00:37:46,580 --> 00:37:51,090 So he started losing money at a less fast rate. 680 00:37:51,090 --> 00:37:55,660 He is now doing something in the stock market, they say. 681 00:37:55,660 --> 00:37:59,164 Doesn't talk so much about it, though. 682 00:37:59,164 --> 00:38:01,380 But an interesting application. 683 00:38:01,380 --> 00:38:05,050 He came up with rules like, if the sum of the jockey's weight 684 00:38:05,050 --> 00:38:09,820 on the post position is low, that's good. 685 00:38:09,820 --> 00:38:12,070 Well, that makes sense in the end, because the jockey's 686 00:38:12,070 --> 00:38:15,330 weight is always between 100 and 110 pounds, and the post 687 00:38:15,330 --> 00:38:17,930 position is always between 1 and 10 or something, so they 688 00:38:17,930 --> 00:38:20,000 were commensurate values. 689 00:38:20,000 --> 00:38:21,996 And a low one is good, in fact. 690 00:38:21,996 --> 00:38:24,430 Not bad. 691 00:38:24,430 --> 00:38:26,100 But neither of those-- 692 00:38:26,100 --> 00:38:28,590 I mean, this is real stuff. 693 00:38:28,590 --> 00:38:31,610 My company uses this sort of stuff to do 694 00:38:31,610 --> 00:38:33,760 some planning work. 695 00:38:33,760 --> 00:38:37,590 But neither of those is as impressive as the 696 00:38:37,590 --> 00:38:41,680 demonstration I'm about to show you that involves the 697 00:38:41,680 --> 00:38:44,710 evolution of creatures. 698 00:38:44,710 --> 00:38:50,580 And these creatures consist of block-like objects, like so. 699 00:38:50,580 --> 00:38:54,390 And they combine like this, and so on. 700 00:38:54,390 --> 00:38:57,950 And so how can you make a feature like that from a 701 00:38:57,950 --> 00:38:59,050 [? 0-1 ?] chromosome? 702 00:38:59,050 --> 00:39:03,600 Well, some of the bits in the chromosome are interpreted as 703 00:39:03,600 --> 00:39:05,660 the number of objects. 704 00:39:05,660 --> 00:39:08,920 Others are interpreted as the sizes of the objects. 705 00:39:08,920 --> 00:39:11,575 Others are interpreted as the structure of how the objects 706 00:39:11,575 --> 00:39:14,080 are articulated. 707 00:39:14,080 --> 00:39:18,410 And still others are interpreted as fixing the 708 00:39:18,410 --> 00:39:23,380 control algorithm by which the creature operates. 709 00:39:23,380 --> 00:39:26,010 So you see how that roughly goes? 710 00:39:26,010 --> 00:39:29,060 Would you like to see a film of that in action? 711 00:39:29,060 --> 00:39:30,030 Yes. 712 00:39:30,030 --> 00:39:30,430 OK. 713 00:39:30,430 --> 00:39:30,780 [INAUDIBLE] 714 00:39:30,780 --> 00:39:32,110 always likes to see the films. 715 00:39:42,890 --> 00:39:46,791 STUDENT: How would you measure diversity in that graph? 716 00:39:46,791 --> 00:39:48,810 PATRICK WINSTON: The question is, how do I measure the 717 00:39:48,810 --> 00:39:50,050 diversity of the graph? 718 00:39:50,050 --> 00:39:52,610 I did it the same way I measured the fitness. 719 00:39:52,610 --> 00:39:56,560 That is to say, I calculated the distance-- 720 00:39:56,560 --> 00:39:58,490 the actual metric distance-- 721 00:39:58,490 --> 00:40:01,500 of all the candidates for the next generation from all of 722 00:40:01,500 --> 00:40:03,610 the candidates that had already been selected. 723 00:40:03,610 --> 00:40:05,602 I summed that up. 724 00:40:05,602 --> 00:40:08,010 And from that sum, I could rank them according to how 725 00:40:08,010 --> 00:40:10,370 different they were from the individuals that were already 726 00:40:10,370 --> 00:40:11,760 in the next generation. 727 00:40:11,760 --> 00:40:14,470 It's like giving a rank, and then from the rank, I use that 728 00:40:14,470 --> 00:40:18,550 kind of calculation to determine a fitness, ie, a 729 00:40:18,550 --> 00:40:20,510 probability of survival, and then I just combine the two 730 00:40:20,510 --> 00:40:21,466 kinds of probabilities. 731 00:40:21,466 --> 00:40:23,856 STUDENT: So you always kept-- every time that you started 732 00:40:23,856 --> 00:40:25,290 something, [? you cached ?] those. 733 00:40:25,290 --> 00:40:27,680 And you kept everything that you've ever [INAUDIBLE]. 734 00:40:27,680 --> 00:40:30,475 PATRICK WINSTON: I'm always using the individuals that 735 00:40:30,475 --> 00:40:33,135 have already been selected at every step, so every step is a 736 00:40:33,135 --> 00:40:34,860 little different because it's working with a new set of 737 00:40:34,860 --> 00:40:36,770 individuals that have already been selected for the next 738 00:40:36,770 --> 00:40:37,065 generation. 739 00:40:37,065 --> 00:40:38,990 OK? 740 00:40:38,990 --> 00:40:40,240 So let's see how this works. 741 00:40:49,080 --> 00:40:50,940 So this is showing the evolution of 742 00:40:50,940 --> 00:40:53,680 some swimming creatures. 743 00:40:53,680 --> 00:40:55,910 And they're evolved according to how well they can swim, how 744 00:40:55,910 --> 00:40:58,040 fast they can go. 745 00:40:58,040 --> 00:41:00,390 Some of them have quite exotic mechanisms, and some of them 746 00:41:00,390 --> 00:41:01,220 quite natural. 747 00:41:01,220 --> 00:41:04,770 That looked like a sperm cell floating away there. 748 00:41:09,440 --> 00:41:11,490 Once you have these things evolving, then of course, you 749 00:41:11,490 --> 00:41:13,820 can get groups of them to evolve together. 750 00:41:19,640 --> 00:41:24,160 So you saw already some that were evolving to swim. 751 00:41:24,160 --> 00:41:28,270 These are evolving to move around on the land. 752 00:41:28,270 --> 00:41:32,580 It's interesting-- this was done by Karl Sims, who at the 753 00:41:32,580 --> 00:41:35,500 time was at a then-thriving company, Thinking Machines, a 754 00:41:35,500 --> 00:41:37,320 fresh spinoff from MIT. 755 00:41:37,320 --> 00:41:41,200 So he was using a vastly parallel computer, super 756 00:41:41,200 --> 00:41:42,760 powerful for its day, thousands of 757 00:41:42,760 --> 00:41:44,710 processors, to do this. 758 00:41:44,710 --> 00:41:46,170 And it was a demonstration of what you could do 759 00:41:46,170 --> 00:41:48,330 with lots of computing. 760 00:41:48,330 --> 00:41:52,340 In the early stages of the experimentation, though, its 761 00:41:52,340 --> 00:41:55,260 notion of physics wasn't quite complete, so some of the 762 00:41:55,260 --> 00:41:58,050 creatures evolved to move by hitting themselves in the 763 00:41:58,050 --> 00:42:03,000 chest and not knowing about the conservation of momentum. 764 00:42:03,000 --> 00:42:04,645 I thought that was just great. 765 00:42:09,150 --> 00:42:10,880 So here they are, out doing some further-- 766 00:42:27,190 --> 00:42:29,700 So you look at these, and you say, wow, there must be 767 00:42:29,700 --> 00:42:30,630 something to this. 768 00:42:30,630 --> 00:42:32,120 This is interesting. 769 00:42:32,120 --> 00:42:33,370 These are complicated. 770 00:42:40,650 --> 00:42:42,930 I think this is one of the ones that was trained, 771 00:42:42,930 --> 00:42:45,440 initially, to swim and then to do land locomotion. 772 00:42:50,760 --> 00:42:56,670 So eventually, Karl got around to thinking about how to make 773 00:42:56,670 --> 00:42:59,890 these things evolve so that they would compete for food. 774 00:43:02,960 --> 00:43:05,980 That's the fastest, I think, by the way, of the land 775 00:43:05,980 --> 00:43:08,630 locomotors. 776 00:43:08,630 --> 00:43:11,990 So that was training them to-- evolving them to jump. 777 00:43:11,990 --> 00:43:17,680 This is evolving them to follow a little red dot. 778 00:43:17,680 --> 00:43:21,910 Some of them have stumbled upon quite exotic methods, as 779 00:43:21,910 --> 00:43:23,160 you can see. 780 00:43:26,380 --> 00:43:29,880 Seem to be flailing around, but somehow manage to-- 781 00:43:29,880 --> 00:43:32,590 sort of like watching people take a quiz. 782 00:43:36,680 --> 00:43:37,930 Making progress on it. 783 00:43:42,590 --> 00:43:44,135 But now we're on to the food competition. 784 00:44:05,590 --> 00:44:08,260 So some of them go for the food, and some of them go to 785 00:44:08,260 --> 00:44:12,240 excluding their opponent from the food, not actually caring 786 00:44:12,240 --> 00:44:13,670 too much about whether they get it. 787 00:44:13,670 --> 00:44:15,850 That's sort of what children do. 788 00:44:32,110 --> 00:44:35,070 There's a kind of hockey player-- 789 00:44:35,070 --> 00:44:36,270 now, here's two hockey players. 790 00:44:36,270 --> 00:44:37,944 Watch this. 791 00:44:37,944 --> 00:44:39,194 They're kind of-- 792 00:44:44,940 --> 00:44:45,990 one succeeds-- 793 00:44:45,990 --> 00:44:48,270 it reminds me a little bit of hockey, rugby, 794 00:44:48,270 --> 00:44:49,970 something like that. 795 00:44:49,970 --> 00:44:54,400 Sometimes, they just get kind of confused, go right after 796 00:44:54,400 --> 00:44:55,901 their opponent, forgetting about the food. 797 00:45:11,800 --> 00:45:13,050 Gives up. 798 00:45:15,260 --> 00:45:18,835 I think these are the overall winners in 799 00:45:18,835 --> 00:45:21,740 this elimination contest. 800 00:45:21,740 --> 00:45:24,170 I can't quite get there. 801 00:45:24,170 --> 00:45:26,840 OK, so you look at that, and you say, wow, that's cool. 802 00:45:26,840 --> 00:45:30,322 Genetic algorithms must be the way to go. 803 00:45:30,322 --> 00:45:32,790 I remember the first time I saw this film. 804 00:45:32,790 --> 00:45:33,790 It was over in Kresge. 805 00:45:33,790 --> 00:45:37,930 I was walking out of the auditorium with Toma Poggio 806 00:45:37,930 --> 00:45:41,500 And we looked at each other, and we said the same thing 807 00:45:41,500 --> 00:45:44,180 simultaneously. 808 00:45:44,180 --> 00:45:47,500 We didn't say that genetic algorithms were the way to go. 809 00:45:47,500 --> 00:45:52,520 What we said was, wow, that space is rich in solutions. 810 00:45:52,520 --> 00:45:55,620 What we were amazed by was not that simple-minded genetic 811 00:45:55,620 --> 00:45:59,070 algorithms produced solutions but that the space was so rich 812 00:45:59,070 --> 00:46:01,800 with solutions that almost any mechanism that was looking 813 00:46:01,800 --> 00:46:04,100 around in that space would find them. 814 00:46:04,100 --> 00:46:06,190 But there's yet another way of thinking about it, and that is 815 00:46:06,190 --> 00:46:10,270 you could say, wow, look at how smart Karl Sims is, 816 00:46:10,270 --> 00:46:12,890 because Karl Sims is the one who had his hands on all the 817 00:46:12,890 --> 00:46:15,050 levers, all those choices. 818 00:46:15,050 --> 00:46:18,950 And I kept emphasizing, all those choices that enabled him 819 00:46:18,950 --> 00:46:23,240 to trick this thing, in some sense, into stumbling across 820 00:46:23,240 --> 00:46:25,510 the solutions in a space that was guaranteed to 821 00:46:25,510 --> 00:46:27,750 be rich with solutions. 822 00:46:27,750 --> 00:46:29,430 So you have to ask-- 823 00:46:29,430 --> 00:46:31,270 so first of all, diversity is good. 824 00:46:31,270 --> 00:46:33,690 We noticed when we put diversity into the genetic 825 00:46:33,690 --> 00:46:35,540 algorithm calculations, we were much 826 00:46:35,540 --> 00:46:36,750 better at finding solutions. 827 00:46:36,750 --> 00:46:39,845 But the next gold star idea that I'd really like to have 828 00:46:39,845 --> 00:46:42,670 you go away with is the idea that you have to ask where the 829 00:46:42,670 --> 00:46:43,500 credit lies. 830 00:46:43,500 --> 00:46:46,930 Does it lie with the ingenuity of the programmer or with the 831 00:46:46,930 --> 00:46:49,230 value of the algorithm itself? 832 00:46:49,230 --> 00:46:52,880 In this case, impressive as it is, the credit lies in the 833 00:46:52,880 --> 00:46:55,250 richness of the space and in the intelligence of the 834 00:46:55,250 --> 00:46:58,000 programmer, not necessarily in the idea of genetic 835 00:46:58,000 --> 00:46:59,250 algorithms.