1 00:00:00,070 --> 00:00:02,500 The following content is provided under a Creative 2 00:00:02,500 --> 00:00:04,019 Commons license. 3 00:00:04,019 --> 00:00:06,360 Your support will help MIT OpenCourseWare 4 00:00:06,360 --> 00:00:10,730 continue to offer high quality educational resources for free. 5 00:00:10,730 --> 00:00:13,330 To make a donation or view additional materials 6 00:00:13,330 --> 00:00:17,217 from hundreds of MIT courses, visit MIT OpenCourseWare 7 00:00:17,217 --> 00:00:17,842 at ocw.mit.edu. 8 00:00:21,002 --> 00:00:23,140 SRINIVAS DEVADAS: Good morning, everyone. 9 00:00:23,140 --> 00:00:27,930 So we have a singleton lecture today on linear programming, 10 00:00:27,930 --> 00:00:32,000 which is general purpose optimization technique that you 11 00:00:32,000 --> 00:00:34,830 can use to solve a whole bunch of problems, 12 00:00:34,830 --> 00:00:41,350 including ones that we've seen in 6.046 and previously in 006. 13 00:00:41,350 --> 00:00:44,780 And most recently, we looked at max flow. 14 00:00:44,780 --> 00:00:46,520 We wouldn't have had to go through all 15 00:00:46,520 --> 00:00:51,920 of that pain we went through to derive a max flow algorithm 16 00:00:51,920 --> 00:00:55,690 if you had a linear programming package handy 17 00:00:55,690 --> 00:00:59,330 and all you wanted to do was find the optimum solution. 18 00:00:59,330 --> 00:01:02,550 You could have just run the linear program 19 00:01:02,550 --> 00:01:06,270 with an appropriate input, of course, that is 20 00:01:06,270 --> 00:01:08,780 derived from the flow network. 21 00:01:08,780 --> 00:01:11,030 And you'd get your optimal solution. 22 00:01:11,030 --> 00:01:13,840 And we'll spend a couple of minutes on that 23 00:01:13,840 --> 00:01:17,380 as we look at the power of linear programming in today's 24 00:01:17,380 --> 00:01:18,390 lecture. 25 00:01:18,390 --> 00:01:19,770 But it's not just max flow. 26 00:01:19,770 --> 00:01:21,470 You could do shortest paths. 27 00:01:21,470 --> 00:01:23,180 You could do multi commodity max flow, 28 00:01:23,180 --> 00:01:26,250 which is more complicated than max flow and a variety 29 00:01:26,250 --> 00:01:28,410 of other problems. 30 00:01:28,410 --> 00:01:30,380 So that's that good news. 31 00:01:30,380 --> 00:01:32,110 The bad news is that the algorithms 32 00:01:32,110 --> 00:01:35,080 for linear programming are a heck of a lot more complicated 33 00:01:35,080 --> 00:01:36,709 than max flow. 34 00:01:36,709 --> 00:01:39,250 And you can imagine that that would be the case, because it's 35 00:01:39,250 --> 00:01:43,450 a more general purpose and more powerful technique. 36 00:01:43,450 --> 00:01:47,630 The history really is that it was an open problem. 37 00:01:47,630 --> 00:01:52,650 Up until 1979, people did not know if linear programming was 38 00:01:52,650 --> 00:01:56,060 polynomial-time solvable until Khachiyan came up 39 00:01:56,060 --> 00:01:59,800 with this ellipsoid method, and then there's 40 00:01:59,800 --> 00:02:01,390 been progress sense. 41 00:02:01,390 --> 00:02:06,220 But the algorithm we're going to describe today and execute 42 00:02:06,220 --> 00:02:09,539 on a example input is a simplex algorithm-- 43 00:02:09,539 --> 00:02:13,490 the simplex algorithm-- that runs in worst case 44 00:02:13,490 --> 00:02:15,450 exponential time. 45 00:02:15,450 --> 00:02:17,410 But it's a very efficient in practice. 46 00:02:17,410 --> 00:02:22,220 And it's held its ground, even with the advent 47 00:02:22,220 --> 00:02:26,800 of more efficient, from a theoretical sense, 48 00:02:26,800 --> 00:02:32,520 polynomial-time algorithms, namely the ellipsoid method, 49 00:02:32,520 --> 00:02:35,480 which actually is not that efficient in practice 50 00:02:35,480 --> 00:02:38,130 and new interior point methods. 51 00:02:38,130 --> 00:02:40,950 So a little bit of context, let's 52 00:02:40,950 --> 00:02:45,950 just dive into an example of optimization 53 00:02:45,950 --> 00:02:52,620 in the context of politics and see 54 00:02:52,620 --> 00:02:56,210 how you could formulate this particular problem 55 00:02:56,210 --> 00:02:58,470 as a linear program. 56 00:02:58,470 --> 00:03:00,950 So how does politics work? 57 00:03:00,950 --> 00:03:03,510 You buy elections, right? 58 00:03:03,510 --> 00:03:07,120 So you don't want to spend a lot of money, 59 00:03:07,120 --> 00:03:10,050 so you want to minimize the amount of money that 60 00:03:10,050 --> 00:03:12,870 is required to buy an election. 61 00:03:12,870 --> 00:03:17,920 And the way you buy an election is, well, campaign, 62 00:03:17,920 --> 00:03:19,820 but you advertise. 63 00:03:19,820 --> 00:03:21,990 And it doesn't matter facts that are relevant. 64 00:03:21,990 --> 00:03:26,450 As long as you get to the right demographic with the right 65 00:03:26,450 --> 00:03:28,280 message, let's assume that you're 66 00:03:28,280 --> 00:03:29,570 going to win the election. 67 00:03:29,570 --> 00:03:33,360 So that's our mathematical abstraction pf campaigning 68 00:03:33,360 --> 00:03:37,850 and politics all in 30 seconds. 69 00:03:37,850 --> 00:03:47,880 So how to campaign to win an election. 70 00:03:47,880 --> 00:03:51,370 And as I said, we're going to advertise. 71 00:03:51,370 --> 00:03:54,390 But you do have a little bit of work to do here. 72 00:03:54,390 --> 00:03:56,580 That's why you need your campaign manager. 73 00:03:56,580 --> 00:04:08,920 And this manager is going to estimate votes 74 00:04:08,920 --> 00:04:13,735 obtained per dollar spent. 75 00:04:19,160 --> 00:04:29,120 But that dollar is spent advertising in support 76 00:04:29,120 --> 00:04:34,410 of a particular case, or a particular issue. 77 00:04:37,360 --> 00:04:40,730 And contradictions are allowed. 78 00:04:40,730 --> 00:04:44,000 As long as you're sending different messages 79 00:04:44,000 --> 00:04:47,340 to different demographics, you're all good, right? 80 00:04:50,126 --> 00:04:51,500 You're assuming that people don't 81 00:04:51,500 --> 00:04:54,180 watch more than one channel. 82 00:04:54,180 --> 00:04:58,000 So you're a Fox News guy or you're a MSNB guy. 83 00:04:58,000 --> 00:04:58,870 You don't do both. 84 00:05:02,660 --> 00:05:05,560 So now, we get at this estimate. 85 00:05:05,560 --> 00:05:08,690 And it turns into a table. 86 00:05:08,690 --> 00:05:12,955 And so you have your policy, and you've got your demographic. 87 00:05:19,640 --> 00:05:30,670 So you got urban, now think Detroit, suburban, 88 00:05:30,670 --> 00:05:33,530 I guess, you could Lexington where you live, 89 00:05:33,530 --> 00:05:39,540 and rural-- I really don't have any idea what that means. 90 00:05:39,540 --> 00:05:42,880 But presumably, there's places. 91 00:05:42,880 --> 00:05:45,370 And here's our policy. 92 00:05:45,370 --> 00:05:49,210 You want to build roads-- kind of boring, 93 00:05:49,210 --> 00:05:55,750 but some people are interested in roads-- gun control, 94 00:05:55,750 --> 00:06:01,730 of very sensitive, farm subsidies-- 95 00:06:01,730 --> 00:06:06,690 you know who's interested in that-- and gasoline tax-- 96 00:06:06,690 --> 00:06:08,870 kind of more, this hits your pocket, 97 00:06:08,870 --> 00:06:13,380 so more or less everybody's interested in that. 98 00:06:13,380 --> 00:06:16,970 So you tell the urban guys you want to build roads, 99 00:06:16,970 --> 00:06:19,060 and they don't like you. 100 00:06:19,060 --> 00:06:20,240 So you get a minus 2 there. 101 00:06:20,240 --> 00:06:22,720 So this can go, you advertise, and it hurts you. 102 00:06:22,720 --> 00:06:25,730 You lose votes. 103 00:06:25,730 --> 00:06:29,180 Tell the suburban people-- well, typically, 104 00:06:29,180 --> 00:06:34,300 it's a situation where you have these nice cars, 105 00:06:34,300 --> 00:06:37,080 and you don't like potholes, so you like roads 106 00:06:37,080 --> 00:06:39,070 if you live in suburbia. 107 00:06:39,070 --> 00:06:40,720 Rural people have 4-by-4's. 108 00:06:40,720 --> 00:06:42,040 They don't particularly care. 109 00:06:42,040 --> 00:06:44,700 They don't care as much. 110 00:06:44,700 --> 00:06:47,670 Gun control-- well, you can imagine that urban people 111 00:06:47,670 --> 00:06:48,940 like that. 112 00:06:48,940 --> 00:06:55,040 Suburban people are, hm, OK, meh. 113 00:06:55,040 --> 00:06:58,950 And the rural people hate it, right? 114 00:06:58,950 --> 00:07:03,630 You do not want to advertise on gun control in the rural areas. 115 00:07:03,630 --> 00:07:07,350 Farm subsidies-- so like, don't want to deal with that. 116 00:07:07,350 --> 00:07:08,015 What is a farm? 117 00:07:12,290 --> 00:07:16,150 And the rural people love it, right? 118 00:07:16,150 --> 00:07:19,690 And then gasoline tax, well, urban people are commuting. 119 00:07:24,700 --> 00:07:27,770 And, well, they typically don't have a lot of money. 120 00:07:27,770 --> 00:07:30,270 So there you go. 121 00:07:30,270 --> 00:07:33,660 And those are the numbers. 122 00:07:33,660 --> 00:07:37,420 I'm not going to justify every number here. 123 00:07:37,420 --> 00:07:39,610 But you could put whatever numbers you what, I mean. 124 00:07:39,610 --> 00:07:41,230 So let's move on. 125 00:07:41,230 --> 00:07:42,700 This is just a table. 126 00:07:42,700 --> 00:07:44,020 It could have positive numbers. 127 00:07:44,020 --> 00:07:45,750 It could have negative numbers. 128 00:07:45,750 --> 00:07:48,360 And you still want to win this election. 129 00:07:48,360 --> 00:07:50,360 Regardless of how crazy the demographics 130 00:07:50,360 --> 00:07:53,042 are, how crazy your electorate is, 131 00:07:53,042 --> 00:07:54,250 you want to win the election. 132 00:07:54,250 --> 00:07:57,400 So as long as you have a great campaign 133 00:07:57,400 --> 00:08:00,880 manager, who can get you this table, 134 00:08:00,880 --> 00:08:04,250 it's all mathematical here on out. 135 00:08:04,250 --> 00:08:06,570 You've just got to figure out how 136 00:08:06,570 --> 00:08:09,220 you're going to win a majority. 137 00:08:12,700 --> 00:08:19,542 And you could argue that all you want is to win the election. 138 00:08:19,542 --> 00:08:21,750 We're going to do something slightly different, which 139 00:08:21,750 --> 00:08:28,470 is something that's obviously going to guarantee victory. 140 00:08:28,470 --> 00:08:33,429 But you want to win a majority in every demographic. 141 00:08:33,429 --> 00:08:39,070 As the tables may be off by little, you want to be careful. 142 00:08:39,070 --> 00:08:42,090 So the last thing, of course, in order 143 00:08:42,090 --> 00:08:48,030 to estimate how much money you need is the population here. 144 00:08:50,830 --> 00:08:58,640 So that's votes obtained per dollar spent. 145 00:08:58,640 --> 00:09:03,030 So you're getting-- it's $10 a vote, it's $5 a vote, 146 00:09:03,030 --> 00:09:04,110 et cetera. 147 00:09:04,110 --> 00:09:06,690 And so we need to translate that to votes, 148 00:09:06,690 --> 00:09:09,170 because that's the dollars. 149 00:09:09,170 --> 00:09:11,510 And you got your population here corresponding 150 00:09:11,510 --> 00:09:13,445 to each of these areas. 151 00:09:16,920 --> 00:09:19,050 And that's what you got here. 152 00:09:19,050 --> 00:09:25,960 Majority, we'll assume you win in the case of ties, 153 00:09:25,960 --> 00:09:30,300 just to keep these numbers easy, so that's just divided by 2. 154 00:09:34,630 --> 00:09:37,050 So that's what you got so far. 155 00:09:37,050 --> 00:09:44,675 And you want to win by spending the minimum amount of money. 156 00:09:52,740 --> 00:09:55,490 So that's our optimization problem. 157 00:09:55,490 --> 00:09:58,700 So we can take this, and we can convert it 158 00:09:58,700 --> 00:10:00,392 to a set of linear equations. 159 00:10:00,392 --> 00:10:02,350 And that's going to create our linear program-- 160 00:10:02,350 --> 00:10:03,480 our first linear program. 161 00:10:07,110 --> 00:10:09,540 And this is our algebraic setup. 162 00:10:16,062 --> 00:10:18,870 And so let-- we need some variables here, 163 00:10:18,870 --> 00:10:29,170 so let x1, x2, x3, x4 denote the dollars spent per issue. 164 00:10:31,920 --> 00:10:34,199 So you got those four issues up there. 165 00:10:34,199 --> 00:10:35,240 So let me write that out. 166 00:10:35,240 --> 00:10:39,630 It's important to make sure you know 167 00:10:39,630 --> 00:10:43,240 what I'm talking about with respect to a particular issue. 168 00:10:43,240 --> 00:10:45,090 So those are our four issues. 169 00:10:45,090 --> 00:10:47,110 And those are our four variables. 170 00:10:47,110 --> 00:10:49,870 So this linear program has four variables. 171 00:10:49,870 --> 00:10:52,490 You're trying to discover the values of these variables 172 00:10:52,490 --> 00:10:56,950 to optimize, minimize your cost function. 173 00:10:56,950 --> 00:10:58,980 The second thing that a linear program has-- 174 00:10:58,980 --> 00:11:02,060 and pretty much the only other thing a linear program has-- 175 00:11:02,060 --> 00:11:03,340 are constraints. 176 00:11:03,340 --> 00:11:06,700 And these constraints are also linear. 177 00:11:06,700 --> 00:11:11,140 It gets much more complicated if you have quadratic constraints, 178 00:11:11,140 --> 00:11:13,050 and we won't go there. 179 00:11:13,050 --> 00:11:16,020 These constraints that I'm going to write correspond 180 00:11:16,020 --> 00:11:20,530 to this statement here that says you want to majority 181 00:11:20,530 --> 00:11:22,860 in each demographic. 182 00:11:22,860 --> 00:11:25,620 So you can imagine that because you have three demographics, 183 00:11:25,620 --> 00:11:28,244 you're going to have three constraints. 184 00:11:28,244 --> 00:11:29,910 You could have written this differently. 185 00:11:29,910 --> 00:11:33,580 There's just any number of variants here. 186 00:11:33,580 --> 00:11:37,120 And you'll get the sense of that as we go to other examples. 187 00:11:37,120 --> 00:11:39,700 But we'll just stick to one variant here. 188 00:11:39,700 --> 00:11:41,615 So now, I want to translate everything 189 00:11:41,615 --> 00:11:46,860 that I've written in English over there into algebra. 190 00:11:46,860 --> 00:11:55,870 And so I got my minimization criterion-- minimize x1 plus x2 191 00:11:55,870 --> 00:12:09,690 plus x3 plus x4-- subject to minus 2x1 plus 8x2 plus 0x3 192 00:12:09,690 --> 00:12:14,890 plus 10x4 greater than or equal to 50,000. 193 00:12:14,890 --> 00:12:18,960 And this represents the requirement 194 00:12:18,960 --> 00:12:22,960 that I want a majority in the first demographic, namely 195 00:12:22,960 --> 00:12:24,130 the urban demographic. 196 00:12:24,130 --> 00:12:26,560 And so I want at least 50,000 votes there. 197 00:12:26,560 --> 00:12:28,650 And I need to spend the money corresponding 198 00:12:28,650 --> 00:12:31,520 to the values of x1 through x4 in such a way 199 00:12:31,520 --> 00:12:33,946 that I get those 50,000 votes. 200 00:12:33,946 --> 00:12:35,570 And that represents that, and it's just 201 00:12:35,570 --> 00:12:42,210 reading off the minus 2, 8, 0, and 10 from the urban column. 202 00:12:42,210 --> 00:12:45,210 So those numbers that you see here correspond to the column, 203 00:12:45,210 --> 00:12:47,350 because I'm talking about the urban demographic. 204 00:12:47,350 --> 00:12:49,550 And you can imagine that the next constraint 205 00:12:49,550 --> 00:12:52,610 is going to correspond to the middle column, and the third 206 00:12:52,610 --> 00:12:54,180 to the third. 207 00:12:54,180 --> 00:12:55,870 So I'll just write that out. 208 00:12:55,870 --> 00:12:58,120 I will call this constraint the constraint number 209 00:12:58,120 --> 00:13:08,162 1-- I might refer to it later-- 5x1, plus 2x2 plus 0x3, 210 00:13:08,162 --> 00:13:13,340 plus 0x4 greater than or equal to 100,000. 211 00:13:13,340 --> 00:13:15,610 Call this number two. 212 00:13:15,610 --> 00:13:22,620 And then finally, 3x1 minus 5x2 plus 10x3. 213 00:13:31,030 --> 00:13:35,310 And that's our set of constraints, 214 00:13:35,310 --> 00:13:38,290 but there's one more little issue 215 00:13:38,290 --> 00:13:42,700 that we have to be careful about, if you're being precise, 216 00:13:42,700 --> 00:13:50,890 and that is that there's no notion of un-advertising. 217 00:13:50,890 --> 00:13:56,940 And so you're going to spend positive dollars. 218 00:13:56,940 --> 00:14:01,260 And so x1 through x4 is greater than or equal to 0. 219 00:14:01,260 --> 00:14:03,720 So that's our first linear program. 220 00:14:03,720 --> 00:14:06,220 And it came from this particular problem. 221 00:14:06,220 --> 00:14:08,210 It'd be wonderful-- and that's exactly what 222 00:14:08,210 --> 00:14:10,860 we're going to do for the rest of the lecture-- 223 00:14:10,860 --> 00:14:14,710 if we could solve this linear program 224 00:14:14,710 --> 00:14:20,010 and any possible linear program in an efficient way. 225 00:14:20,010 --> 00:14:22,550 And so the number of variables is small n. 226 00:14:29,270 --> 00:14:31,710 And you can imagine that the number of constraints here, 227 00:14:31,710 --> 00:14:35,586 just talking about these constraints, are m constraints. 228 00:14:40,030 --> 00:14:47,440 And you certainly want at run time that is polynomial in n. 229 00:14:47,440 --> 00:14:48,740 That's our goal here. 230 00:14:48,740 --> 00:14:53,170 And as I mentioned early on, it was unclear 231 00:14:53,170 --> 00:14:55,889 for the longest time-- well, at least not until 1979, 232 00:14:55,889 --> 00:14:57,430 but people had been thinking about it 233 00:14:57,430 --> 00:15:00,320 for a long time before that-- as to whether there 234 00:15:00,320 --> 00:15:04,320 was a general algorithm that would solve any linear program 235 00:15:04,320 --> 00:15:06,266 in time-polynomial in n. 236 00:15:06,266 --> 00:15:09,820 And that was resolved in '79 by Khachiyan. 237 00:15:09,820 --> 00:15:14,760 We'll look at a algorithm as simplex that in the worst case 238 00:15:14,760 --> 00:15:18,300 runs exponentially in n but is simpler to describe 239 00:15:18,300 --> 00:15:21,470 and is very efficient in practice. 240 00:15:21,470 --> 00:15:29,616 So in our particular problem, this one, it turns out you-- 241 00:15:29,616 --> 00:15:31,990 and I'm actually going to come back to this in a second-- 242 00:15:31,990 --> 00:15:39,420 but I will just tell you that the optimum 243 00:15:39,420 --> 00:15:42,950 for this particular linear program 244 00:15:42,950 --> 00:15:49,170 with these particular numbers correspond to these numbers 245 00:15:49,170 --> 00:15:49,840 here. 246 00:15:49,840 --> 00:15:56,602 So you want to spend something of the order of $20,000 247 00:15:56,602 --> 00:16:01,780 for-- so there's 100 here, so take away the two 248 00:16:01,780 --> 00:16:06,530 0's-- so spend something of the order of $20,000 for the first 249 00:16:06,530 --> 00:16:11,040 issue, building roads, spend a bit of money for the second 250 00:16:11,040 --> 00:16:15,020 issue, ignore the third corresponding to farm 251 00:16:15,020 --> 00:16:23,500 subsidies, and spend a bit of money for the gasoline tax 252 00:16:23,500 --> 00:16:24,700 issue. 253 00:16:24,700 --> 00:16:27,870 And these numbers aren't important, other than the fact 254 00:16:27,870 --> 00:16:30,190 that they happen to be optimum. 255 00:16:30,190 --> 00:16:40,070 So if you add up these numbers, then x1 plus x2 plus x3 plus x4 256 00:16:40,070 --> 00:16:45,110 is something of the order of $21,000-- 257 00:16:45,110 --> 00:16:47,310 $27,000, excuse me, though I'm writing it out 258 00:16:47,310 --> 00:16:50,890 as this fraction. 259 00:16:50,890 --> 00:16:57,580 So important consideration here is that these values xi 260 00:16:57,580 --> 00:16:58,330 are real numbers. 261 00:17:01,910 --> 00:17:03,640 That's it. 262 00:17:03,640 --> 00:17:06,224 It's not that they have to be integral. 263 00:17:06,224 --> 00:17:07,640 Clearly, there were fractions here 264 00:17:07,640 --> 00:17:10,710 for the optimum, some of them anyway. 265 00:17:10,710 --> 00:17:12,640 But in general, linear programming 266 00:17:12,640 --> 00:17:17,440 says the variable values are real. 267 00:17:17,440 --> 00:17:19,609 There's also integer linear programming, 268 00:17:19,609 --> 00:17:23,829 which is NP complete, which adds the additional constraint 269 00:17:23,829 --> 00:17:26,180 that the xi values are integral. 270 00:17:26,180 --> 00:17:28,620 So it turns into a harder problem. 271 00:17:28,620 --> 00:17:31,700 You got polynomial-time solvable if the xi are real. 272 00:17:31,700 --> 00:17:34,520 You got NP complete, which Eric is 273 00:17:34,520 --> 00:17:40,570 going to talk about on Thursday, if the values are 274 00:17:40,570 --> 00:17:41,690 forced to be integral. 275 00:17:41,690 --> 00:17:45,560 So this extra constraint makes things worse from a complexity 276 00:17:45,560 --> 00:17:46,700 standpoint. 277 00:17:46,700 --> 00:17:50,730 We won't talk about ILP anymore for the rest of this lecture. 278 00:17:50,730 --> 00:17:52,430 So I will come back to this. 279 00:17:52,430 --> 00:17:56,160 And I'll talk about how we can show 280 00:17:56,160 --> 00:17:58,460 that this is optimum without actually going 281 00:17:58,460 --> 00:18:01,370 into a deep algorithmic dive. 282 00:18:01,370 --> 00:18:04,090 But what I want to do just before that 283 00:18:04,090 --> 00:18:07,500 is to give you the general formulation 284 00:18:07,500 --> 00:18:10,130 of a linear program. 285 00:18:10,130 --> 00:18:16,290 It's called the standard form in CLRS, also called 286 00:18:16,290 --> 00:18:18,220 the general form. 287 00:18:18,220 --> 00:18:30,380 In some cases, we'll look at the standard form for LP. 288 00:18:30,380 --> 00:18:35,770 And I want to pop up a level about this example 289 00:18:35,770 --> 00:18:37,850 and give you the general setting. 290 00:18:37,850 --> 00:18:41,580 And we'll focus in on the general setting for the most 291 00:18:41,580 --> 00:18:42,930 part. 292 00:18:42,930 --> 00:18:47,160 But what I have here is I can either minimize 293 00:18:47,160 --> 00:18:49,620 or maximize-- we had a minimization 294 00:18:49,620 --> 00:18:56,480 problem-- for the political problem, 295 00:18:56,480 --> 00:19:01,390 minimize the linear objective function 296 00:19:01,390 --> 00:19:12,395 subject to linear inequalities or equations. 297 00:19:17,790 --> 00:19:24,260 And the variables, think of x as a vector, 298 00:19:24,260 --> 00:19:30,400 it's a column vector, or x1, x2, all the way to xn. 299 00:19:39,030 --> 00:19:55,980 And the objective function is c times x, so that's c1x1 dot, 300 00:19:55,980 --> 00:19:59,760 dot, dot, cnxn. 301 00:19:59,760 --> 00:20:01,860 And we just had all the coefficients being a 1 302 00:20:01,860 --> 00:20:02,950 over there. 303 00:20:02,950 --> 00:20:12,260 And the inequalities, they're the fun part, 304 00:20:12,260 --> 00:20:15,580 you can represent them as a matrix A, 305 00:20:15,580 --> 00:20:20,090 so A times x less than or equal to b. 306 00:20:20,090 --> 00:20:22,590 And notice that this is the standard form 307 00:20:22,590 --> 00:20:23,660 that I'm talking about. 308 00:20:23,660 --> 00:20:26,805 And now, I have diverged from what 309 00:20:26,805 --> 00:20:28,180 I had here, because I had greater 310 00:20:28,180 --> 00:20:30,140 than or equal to over here. 311 00:20:30,140 --> 00:20:33,520 So it turns out, you'll see linear programs 312 00:20:33,520 --> 00:20:34,880 in different settings. 313 00:20:34,880 --> 00:20:36,380 Sometimes, you'll have minimization. 314 00:20:36,380 --> 00:20:38,340 Sometimes, you'll have maximization. 315 00:20:38,340 --> 00:20:40,364 Sometimes, you'll have less than constraints, 316 00:20:40,364 --> 00:20:41,780 less than or equal to constraints. 317 00:20:41,780 --> 00:20:43,970 Sometimes, you'll have greater than or equal to constraints. 318 00:20:43,970 --> 00:20:45,940 Sometimes, you'll have equality constraints. 319 00:20:45,940 --> 00:20:47,730 We'll spend a little bit of time talking 320 00:20:47,730 --> 00:20:52,000 about how you can transform any given linear program 321 00:20:52,000 --> 00:20:53,860 into a standard form. 322 00:20:53,860 --> 00:21:01,040 So our standard form is going to be something that maximizes 323 00:21:01,040 --> 00:21:03,100 the objective function. 324 00:21:03,100 --> 00:21:05,530 So these are our inequalities, and they're 325 00:21:05,530 --> 00:21:07,900 represented as less than or equal to. 326 00:21:07,900 --> 00:21:09,880 That's the standard form. 327 00:21:09,880 --> 00:21:19,330 And you want to maximize c times x-- again, 328 00:21:19,330 --> 00:21:25,960 max for standard-- such that these set of inequalities 329 00:21:25,960 --> 00:21:33,160 told Ax less than or equal to b and x greater than 330 00:21:33,160 --> 00:21:35,510 or equal to 0. 331 00:21:35,510 --> 00:21:40,320 So for each of the values that correspond to the variables, 332 00:21:40,320 --> 00:21:42,160 you want these variables to be non-negative 333 00:21:42,160 --> 00:21:43,900 in the standard form. 334 00:21:43,900 --> 00:21:46,410 And you want less than or equal to corresponding 335 00:21:46,410 --> 00:21:49,280 to each of the inequalities-- not equal to, 336 00:21:49,280 --> 00:21:52,700 not greater than or equal to, but less than or equal to. 337 00:21:52,700 --> 00:21:54,690 And you have this linear cost function, 338 00:21:54,690 --> 00:21:57,070 where you could have arbitrary coefficients, 339 00:21:57,070 --> 00:21:59,950 but you're maximizing it. 340 00:21:59,950 --> 00:22:01,030 So that's it. 341 00:22:01,030 --> 00:22:04,136 So it's all about polarities, not much more than that. 342 00:22:04,136 --> 00:22:05,260 It's just about polarities. 343 00:22:05,260 --> 00:22:11,030 And if you get a linear program, a specific linear program that 344 00:22:11,030 --> 00:22:15,120 doesn't conform to this-- we'll spend a few minutes talking 345 00:22:15,120 --> 00:22:16,730 about conversions-- and it's going 346 00:22:16,730 --> 00:22:18,780 to be fairly straightforward. 347 00:22:18,780 --> 00:22:23,530 May not be immediately obvious, but we'll get to that. 348 00:22:23,530 --> 00:22:24,420 Any questions so far? 349 00:22:28,810 --> 00:22:34,460 So I want to go back to this claim 350 00:22:34,460 --> 00:22:38,380 here, where I said this is optimum. 351 00:22:38,380 --> 00:22:44,520 Now without actually describing an algorithm to you, 352 00:22:44,520 --> 00:22:46,850 I'm going to be able to show you, convince 353 00:22:46,850 --> 00:22:50,690 you, that this value here corresponding 354 00:22:50,690 --> 00:22:54,510 to whatever it is, 3 million, 3.1 million, 355 00:22:54,510 --> 00:22:56,470 is in fact optimum. 356 00:22:56,470 --> 00:22:59,840 And this is something I could do, 357 00:22:59,840 --> 00:23:06,780 because linear programming has this powerful notion 358 00:23:06,780 --> 00:23:09,080 of duality. 359 00:23:09,080 --> 00:23:11,450 So what is that? 360 00:23:11,450 --> 00:23:16,010 Well, let's just first look at our specific example here. 361 00:23:16,010 --> 00:23:20,370 And I'll give you a very specific observation. 362 00:23:20,370 --> 00:23:22,170 I'm going to give you what you can 363 00:23:22,170 --> 00:23:26,555 think of as a certificate of optimality. 364 00:23:30,429 --> 00:23:32,470 I'm going to give you a certificate of optimality 365 00:23:32,470 --> 00:23:34,840 for that set of numbers. 366 00:23:34,840 --> 00:23:37,860 And here's how I'm going to do it. 367 00:23:37,860 --> 00:23:42,800 So is there a short certificate? 368 00:23:45,550 --> 00:23:47,570 I can imagine giving you a long proof 369 00:23:47,570 --> 00:23:51,660 that a particular linear programming algorithm always 370 00:23:51,660 --> 00:23:55,370 gives you the minimum answer, the optimum answer, 371 00:23:55,370 --> 00:23:59,020 walk through that, executive that algorithm 372 00:23:59,020 --> 00:24:03,210 on this particular example, and then you're 373 00:24:03,210 --> 00:24:07,480 convinced, of course, that the solution is 374 00:24:07,480 --> 00:24:08,590 going to be optimum. 375 00:24:08,590 --> 00:24:12,110 But for this specific example, I want to give you a certificate. 376 00:24:12,110 --> 00:24:15,450 This certificate isn't going to work for other examples. 377 00:24:15,450 --> 00:24:18,520 It's going to short, because it only works with this example. 378 00:24:18,520 --> 00:24:21,010 But it won't work for other ones. 379 00:24:21,010 --> 00:24:22,231 And so how do I do that? 380 00:24:25,390 --> 00:24:27,250 So the answer is, in fact, there is 381 00:24:27,250 --> 00:24:33,990 a certificate that shows that the LP solution is optimal. 382 00:24:33,990 --> 00:24:49,230 And consider that I compute this particular algebraic quantity, 383 00:24:49,230 --> 00:24:59,960 where all I've done is I've taken these three equations 384 00:24:59,960 --> 00:25:06,620 and I've multiplied them by these magical constants. 385 00:25:06,620 --> 00:25:08,970 And so I'm not going to tell you how we get 386 00:25:08,970 --> 00:25:10,940 this certificate of optimality. 387 00:25:10,940 --> 00:25:12,900 But I'm going to give you the certificate. 388 00:25:12,900 --> 00:25:14,400 And it's going to be clear that it's 389 00:25:14,400 --> 00:25:16,150 a certificate of optimality. 390 00:25:16,150 --> 00:25:20,030 And if I take these three equations here, 1, 2, and 3-- 391 00:25:20,030 --> 00:25:25,880 actually, I refer to 1, 2, and 3, they refer to the equations. 392 00:25:25,880 --> 00:25:29,850 These are equations or constraints. 393 00:25:29,850 --> 00:25:30,890 And so I take that. 394 00:25:30,890 --> 00:25:33,600 And obviously, if I have a bunch of equations 395 00:25:33,600 --> 00:25:36,430 and I multiply them out, I can certainly add them up, 396 00:25:36,430 --> 00:25:39,040 and I get another equation at the end of it. 397 00:25:39,040 --> 00:25:40,600 And it's all going to be linear. 398 00:25:40,600 --> 00:25:46,220 And if I do that, I get x1 plus x2 399 00:25:46,220 --> 00:25:53,110 plus 140 divided by 222 times x3 plus x4. 400 00:25:53,110 --> 00:25:55,550 So that's what happens to the left hand side. 401 00:25:55,550 --> 00:25:59,830 And the right hand side is-- you'll 402 00:25:59,830 --> 00:26:07,760 recognize this quantity-- five 0's divided by 111. 403 00:26:07,760 --> 00:26:10,010 That's what you get. 404 00:26:10,010 --> 00:26:13,590 So now, can someone tell me why in the last step, why 405 00:26:13,590 --> 00:26:15,940 this is a certificate of optimality-- 406 00:26:15,940 --> 00:26:19,690 the fact that obviously, this is all algebra, once I've 407 00:26:19,690 --> 00:26:23,330 discovered the coefficients-- so now that I've done this, 408 00:26:23,330 --> 00:26:29,690 why have I just shown you that 3.1 million divided by 111 409 00:26:29,690 --> 00:26:30,869 is, in fact, optimum? 410 00:26:30,869 --> 00:26:31,910 Can someone tell me this? 411 00:26:35,007 --> 00:26:36,840 Look at what you have on the left hand side. 412 00:26:40,110 --> 00:26:42,340 No? 413 00:26:42,340 --> 00:26:42,840 Yeah. 414 00:26:42,840 --> 00:26:45,090 AUDIENCE: Any other solution would cost more 415 00:26:45,090 --> 00:26:47,840 than the amount put in spent. 416 00:26:47,840 --> 00:26:49,480 SRINIVAS DEVADAS: Any other solution, 417 00:26:49,480 --> 00:26:53,500 but I want you to relate that to-- what am I spending? 418 00:26:53,500 --> 00:26:56,860 AUDIENCE: You're spending 3,100,000-- 419 00:26:56,860 --> 00:26:59,270 like, it's the same thing. 420 00:26:59,270 --> 00:27:02,140 SRINIVAS DEVADAS: Yeah, but I mean, this was a claim. 421 00:27:02,140 --> 00:27:06,500 This was a claim-- and at this point, an unproven claim. 422 00:27:10,550 --> 00:27:12,025 It was an unproven claim. 423 00:27:12,025 --> 00:27:12,650 Yeah, go ahead. 424 00:27:12,650 --> 00:27:15,191 AUDIENCE: You know that the left hand side of that inequality 425 00:27:15,191 --> 00:27:18,227 is less than or equal to x1 plus x2 plus x3 plus x4. 426 00:27:18,227 --> 00:27:19,310 SRINIVAS DEVADAS: Correct. 427 00:27:19,310 --> 00:27:22,160 And what is x1 plus x2 plus x3 plus x4, to be clear? 428 00:27:22,160 --> 00:27:24,770 AUDIENCE: The thing you're trying to minimize? 429 00:27:24,770 --> 00:27:25,650 SRINIVAS DEVADAS: Yeah, exactly, the thing 430 00:27:25,650 --> 00:27:26,900 you're trying to minimize. 431 00:27:26,900 --> 00:27:27,630 Exactly. 432 00:27:27,630 --> 00:27:29,530 You're almost there. 433 00:27:29,530 --> 00:27:37,550 but the key observation here is that x1 plus x2 plus x3 plus x4 434 00:27:37,550 --> 00:27:41,460 is greater than or equal to x1 plus x2-- because all of these 435 00:27:41,460 --> 00:27:45,140 are positive quantities, remember-- 140 divided by 222, 436 00:27:45,140 --> 00:27:50,940 that's less than 1, x3 plus x4, correct? 437 00:27:50,940 --> 00:27:56,770 So given that, I can say that this is greater than our equal 438 00:27:56,770 --> 00:28:02,550 to 3,100,000 111. 439 00:28:02,550 --> 00:28:06,820 It's because of this observation that it's 440 00:28:06,820 --> 00:28:08,760 a certificate of optimality. 441 00:28:11,770 --> 00:28:14,955 She has her head down, OK. 442 00:28:14,955 --> 00:28:16,600 Great. 443 00:28:16,600 --> 00:28:22,420 So that's pretty cool. 444 00:28:22,420 --> 00:28:26,070 Just cooked up these coefficients from somewhere, 445 00:28:26,070 --> 00:28:28,110 pulled them out of a hat, you're all 446 00:28:28,110 --> 00:28:31,920 convinced now that the value we got was optimum. 447 00:28:31,920 --> 00:28:33,090 Did not run an algorithm. 448 00:28:33,090 --> 00:28:34,800 Maybe I ran an algorithm-- of course, 449 00:28:34,800 --> 00:28:37,260 you ran an algorithm to get those coefficients, right? 450 00:28:37,260 --> 00:28:39,340 Well, how did those coefficients appear? 451 00:28:39,340 --> 00:28:43,700 So we're not going to spend a whole lot of time on this. 452 00:28:43,700 --> 00:28:51,340 You'll see this likely in a problem 453 00:28:51,340 --> 00:28:53,160 set or perhaps in section. 454 00:28:53,160 --> 00:28:59,050 But in general, I won't worry too much about duality, 455 00:28:59,050 --> 00:29:01,980 other than knowing the concept. 456 00:29:01,980 --> 00:29:04,640 And this notion of LP duality essentially 457 00:29:04,640 --> 00:29:08,970 says that what just happened wasn't a coincidence. 458 00:29:08,970 --> 00:29:11,130 You can do this all the time. 459 00:29:11,130 --> 00:29:14,060 There will always be, for a linear program, 460 00:29:14,060 --> 00:29:16,500 a short certificate of optimality 461 00:29:16,500 --> 00:29:20,280 that corresponds to some set of coefficients 462 00:29:20,280 --> 00:29:24,160 that you can do this particular math with by taking 463 00:29:24,160 --> 00:29:27,150 these linear constraints, multiplying them out, adding 464 00:29:27,150 --> 00:29:32,030 them up together, and showing that you have a lower bound 465 00:29:32,030 --> 00:29:39,410 on-- in the case of this problem-- you can't get lower 466 00:29:39,410 --> 00:29:40,470 than this. 467 00:29:40,470 --> 00:29:42,740 And therefore, for a minimization problem, 468 00:29:42,740 --> 00:29:46,580 when you reach that, you clearly found the optimum. 469 00:29:46,580 --> 00:29:49,280 And that's the notion of LP duality. 470 00:29:49,280 --> 00:29:54,800 And the basic theorem-- and this is really more as an FYI, 471 00:29:54,800 --> 00:29:57,570 we won't prove this theorem-- is that if you 472 00:29:57,570 --> 00:30:01,160 had the standard form for the LP, which I'm just writing down 473 00:30:01,160 --> 00:30:09,090 again here, where you had Ax less than b, x vector 474 00:30:09,090 --> 00:30:10,250 greater than or equal to 0. 475 00:30:10,250 --> 00:30:14,710 So that's identically what I had up here, or done here, 476 00:30:14,710 --> 00:30:18,310 corresponding to the standard LP form. 477 00:30:18,310 --> 00:30:22,110 Well, there's a dual-- this is what's called the primal form. 478 00:30:22,110 --> 00:30:28,280 Usually, if you don't say it, you 479 00:30:28,280 --> 00:30:29,610 think of it as the primal form. 480 00:30:29,610 --> 00:30:32,050 And if it's dual, you call it a dual. 481 00:30:32,050 --> 00:30:34,210 And this is primal form of LP. 482 00:30:34,210 --> 00:30:37,890 This is a dual form of LP, or dual LP. 483 00:30:37,890 --> 00:30:41,920 And the dual LP flips everything. 484 00:30:41,920 --> 00:30:45,000 And it's not just negation, but transposed, 485 00:30:45,000 --> 00:30:50,230 and the actual variables also swap functionalities. 486 00:30:50,230 --> 00:30:51,680 So it's really pretty cool. 487 00:30:51,680 --> 00:30:53,970 So your max turns into a min. 488 00:30:53,970 --> 00:30:56,650 The c gets replaced by the b, which is on the right hand 489 00:30:56,650 --> 00:30:58,400 side of the inequalities. 490 00:30:58,400 --> 00:31:03,870 And your constraints are A transpose, 491 00:31:03,870 --> 00:31:07,290 y greater than or equal to c. 492 00:31:07,290 --> 00:31:09,210 So there's a flip there as well. 493 00:31:09,210 --> 00:31:13,290 And y is greater than or equal to 0. 494 00:31:13,290 --> 00:31:16,790 So there's a bunch of things that's going on here. 495 00:31:16,790 --> 00:31:23,700 And these two problems end up being equivalent-- 496 00:31:23,700 --> 00:31:26,990 the primal and the dual, you can always do this. 497 00:31:26,990 --> 00:31:31,910 And essentially, what is happening here 498 00:31:31,910 --> 00:31:39,422 is that you're solving these two problems simultaneously. 499 00:31:39,422 --> 00:31:40,880 And there's lots of algorithms that 500 00:31:40,880 --> 00:31:44,520 keep flipping between these two forms for efficiency. 501 00:31:44,520 --> 00:31:46,890 But ultimately, what ends up happening is 502 00:31:46,890 --> 00:31:52,190 you see that the actual constraints that you had here 503 00:31:52,190 --> 00:31:56,610 corresponding to the b constraints turn into-- the b 504 00:31:56,610 --> 00:31:59,400 ends up in the cost function here. 505 00:31:59,400 --> 00:32:01,400 And that's essentially what's happening out here 506 00:32:01,400 --> 00:32:04,460 with respect to multiplying these equations out 507 00:32:04,460 --> 00:32:06,800 with particular coefficients. 508 00:32:06,800 --> 00:32:11,830 So as I said, this is really more as an FYI. 509 00:32:11,830 --> 00:32:16,220 This is an obviously interesting and an interesting proof 510 00:32:16,220 --> 00:32:18,950 of optimality, which is a different kind of proof 511 00:32:18,950 --> 00:32:22,260 from proving an algorithm correct 512 00:32:22,260 --> 00:32:25,649 and applying that proof to a particular instance. 513 00:32:25,649 --> 00:32:27,190 That's the kind of thing that happens 514 00:32:27,190 --> 00:32:31,880 in LP, especially when you flip from primal and dual forms. 515 00:32:31,880 --> 00:32:35,490 So I'll leave it at that. 516 00:32:35,490 --> 00:32:38,230 What I'd like to do is give you a sense 517 00:32:38,230 --> 00:32:44,470 of how we can convert to standard form, 518 00:32:44,470 --> 00:32:50,760 so you can apply an algorithm that-- for example, you 519 00:32:50,760 --> 00:32:53,060 have a program and it only requires standard form. 520 00:32:53,060 --> 00:32:54,767 It runs on standard form. 521 00:32:54,767 --> 00:32:56,100 Let's go over it really quickly. 522 00:32:56,100 --> 00:33:00,160 This is not going to take very long-- a translation 523 00:33:00,160 --> 00:33:04,450 from different kinds of LPs-- and we had a slightly different 524 00:33:04,450 --> 00:33:08,940 here for our political problem that had a minimization-- 525 00:33:08,940 --> 00:33:11,660 and how would we convert that to standard form. 526 00:33:11,660 --> 00:33:27,090 So it's probably just one conversion here that's tricky. 527 00:33:27,090 --> 00:33:36,097 So suppose I want to minimize minus 2x1 plus 3x2, 528 00:33:36,097 --> 00:33:37,930 and I want to convert it to standard formal. 529 00:33:37,930 --> 00:33:39,720 All I have is a standard LP solver. 530 00:33:39,720 --> 00:33:40,290 What do I do? 531 00:33:43,846 --> 00:33:46,490 It should be easy. 532 00:33:46,490 --> 00:33:50,440 What do I do if I had a solver that was maximizing, 533 00:33:50,440 --> 00:33:53,410 but I want to minimize a quantity? 534 00:33:53,410 --> 00:33:55,840 Just switch the signs, right? 535 00:33:55,840 --> 00:34:04,710 So negate to 2x1 minus 3x2 and maximize. 536 00:34:07,680 --> 00:34:10,110 So that was easy. 537 00:34:10,110 --> 00:34:12,370 This is a tricky one. 538 00:34:12,370 --> 00:34:29,050 Suppose xj does not have a non-negativity constraint. 539 00:34:29,050 --> 00:34:32,620 So it just happens to be the case that it's not dollars 540 00:34:32,620 --> 00:34:37,620 but it's some other quantity that can go negative. 541 00:34:37,620 --> 00:34:39,880 It might be profit or loss. 542 00:34:39,880 --> 00:34:42,820 So that quantity represents profit and loss, 543 00:34:42,820 --> 00:34:45,300 so it could go negative if it's loss. 544 00:34:45,300 --> 00:34:49,250 So I don't have this constraint in my original problem 545 00:34:49,250 --> 00:34:50,800 specification. 546 00:34:50,800 --> 00:34:53,980 But my standard form and my LP solver 547 00:34:53,980 --> 00:34:59,560 requires this entire vector to be non-negative. 548 00:34:59,560 --> 00:35:01,380 So I got a problem here. 549 00:35:01,380 --> 00:35:04,020 I can't use my standard solver, because 550 00:35:04,020 --> 00:35:06,320 of this non-negativity constraint. 551 00:35:06,320 --> 00:35:08,490 So how do I fix that? 552 00:35:08,490 --> 00:35:14,840 How do I turn it into a problem that allows the standard solver 553 00:35:14,840 --> 00:35:16,655 to be used? 554 00:35:16,655 --> 00:35:17,280 Yeah, go ahead. 555 00:35:17,280 --> 00:35:19,710 AUDIENCE: You can break it up into two variables, 556 00:35:19,710 --> 00:35:25,542 like xj1 and xj2, so xj1 minus xj2 equals xj, 557 00:35:25,542 --> 00:35:26,876 and both could have [INAUDIBLE]. 558 00:35:26,876 --> 00:35:28,792 SRINIVAS DEVADAS: Perfect, great, that's good. 559 00:35:28,792 --> 00:35:29,350 Here you go. 560 00:35:32,140 --> 00:35:39,400 So what you do here is take xj and replace it with, 561 00:35:39,400 --> 00:35:44,520 let's say, xj prime minus xj double prime. 562 00:35:44,520 --> 00:35:48,660 And you have xj prime greater than or equal to 0, 563 00:35:48,660 --> 00:35:51,300 xj double prime greater than or equal to 0. 564 00:35:51,300 --> 00:35:54,720 But depending on the particular values 565 00:35:54,720 --> 00:35:57,340 in whatever solution you're exploring 566 00:35:57,340 --> 00:36:01,340 are the final solution, you may have an actual xj value 567 00:36:01,340 --> 00:36:03,290 that's negative or positive. 568 00:36:03,290 --> 00:36:07,060 So you added an extra variable here to your linear program. 569 00:36:09,770 --> 00:36:13,570 And a couple more real quick, suppose 570 00:36:13,570 --> 00:36:19,560 that I have an equality constraint corresponding 571 00:36:19,560 --> 00:36:24,150 to x1 plus x2 equals 7. 572 00:36:24,150 --> 00:36:25,980 What do I do with an equality constraint 573 00:36:25,980 --> 00:36:30,190 where I have x1 plus x2 equals 7? 574 00:36:30,190 --> 00:36:30,860 Yeah, go ahead. 575 00:36:30,860 --> 00:36:35,770 AUDIENCE: You can say x1 plus x2 is greater than or equal to 7, 576 00:36:35,770 --> 00:36:38,135 and x1 plus x2 is less than or equal to 7. 577 00:36:38,135 --> 00:36:40,510 SRINIVAS DEVADAS: No, you can't do less than or equal to. 578 00:36:40,510 --> 00:36:42,427 AUDIENCE: But then you can flip the signs to-- 579 00:36:42,427 --> 00:36:44,593 SRINIVAS DEVADAS: Ah, then you could flip the signs. 580 00:36:44,593 --> 00:36:46,350 So you have two steps there, good. 581 00:36:46,350 --> 00:36:50,710 So your less than or equal to needs another multiplier. 582 00:36:50,710 --> 00:36:55,410 So what you end up doing is something like x1 plus x2 583 00:36:55,410 --> 00:36:57,110 greater than or equal to 7. 584 00:36:57,110 --> 00:37:01,610 And then you need-- if you do what the gentleman just 585 00:37:01,610 --> 00:37:10,600 said-- and flip the sign, you get minus x1 minus x2 586 00:37:10,600 --> 00:37:11,995 greater than or equal to minus 7. 587 00:37:17,077 --> 00:37:17,660 Is that right? 588 00:37:17,660 --> 00:37:18,000 No? 589 00:37:18,000 --> 00:37:18,570 I messed up? 590 00:37:22,216 --> 00:37:23,590 Oh, I want less than or equal to. 591 00:37:23,590 --> 00:37:24,830 You're right, you're right. 592 00:37:24,830 --> 00:37:28,380 So I need less than or equal to-- that's right, of course, 593 00:37:28,380 --> 00:37:29,820 thank you. 594 00:37:29,820 --> 00:37:32,570 So I need less than or equal to in both places. 595 00:37:38,380 --> 00:37:40,750 So that's the standard form. 596 00:37:40,750 --> 00:37:42,210 I needed less than or equal to. 597 00:37:42,210 --> 00:37:42,710 Good. 598 00:37:44,629 --> 00:37:46,920 What you've done is increased the number of constraints 599 00:37:46,920 --> 00:37:48,760 by one. 600 00:37:48,760 --> 00:37:51,453 Did I get this right the second time? 601 00:37:51,453 --> 00:37:52,440 All right. 602 00:37:52,440 --> 00:37:53,572 So that's pretty much it. 603 00:37:53,572 --> 00:37:55,530 The last thing, which I won't really write out, 604 00:37:55,530 --> 00:37:59,130 is, which we've done here already, greater than 605 00:37:59,130 --> 00:38:05,150 or equal to constraint translated-- I won't give you 606 00:38:05,150 --> 00:38:07,580 an example of this; we have this already-- 607 00:38:07,580 --> 00:38:12,540 translates to less than or equal to by minus 1 multiplied. 608 00:38:15,290 --> 00:38:17,060 So we have to invoke that in order 609 00:38:17,060 --> 00:38:19,670 to do the equality anyway. 610 00:38:19,670 --> 00:38:21,800 So you're in business. 611 00:38:21,800 --> 00:38:23,960 If you have a standard LP solver, 612 00:38:23,960 --> 00:38:28,190 you can take pretty much any optimization problem that 613 00:38:28,190 --> 00:38:30,680 is linear in terms of its objective function 614 00:38:30,680 --> 00:38:34,340 and has linear constraints, and you can transform it into LP. 615 00:38:34,340 --> 00:38:36,940 If you had non-linear constraints, 616 00:38:36,940 --> 00:38:38,900 there's lots of work that goes on 617 00:38:38,900 --> 00:38:43,240 in linearizing those constraints and using LP solvers. 618 00:38:43,240 --> 00:38:44,990 It's a very practical thing to do. 619 00:38:44,990 --> 00:38:48,210 It may be something you'll end up doing, invoking 620 00:38:48,210 --> 00:38:51,030 these powerful LP solvers-- typically, they're commercially 621 00:38:51,030 --> 00:38:54,090 available; the best ones are commercial-- 622 00:38:54,090 --> 00:38:56,600 and use it to solve your particular problem. 623 00:38:56,600 --> 00:39:01,640 It turns your algorithm design problem into a reduction. 624 00:39:01,640 --> 00:39:07,590 And so you'll spend really the next couple of weeks 625 00:39:07,590 --> 00:39:08,900 thinking about reductions. 626 00:39:08,900 --> 00:39:10,710 We'll start that up right now, where 627 00:39:10,710 --> 00:39:13,070 we'll take existing combinatorial 628 00:39:13,070 --> 00:39:16,810 problems, for which we already know algorithms for, 629 00:39:16,810 --> 00:39:18,550 and you're going to reduce them to LP. 630 00:39:18,550 --> 00:39:20,910 Just to give you a sense of what the power of LP 631 00:39:20,910 --> 00:39:24,530 is, but this notion of reduction is very powerful, 632 00:39:24,530 --> 00:39:26,380 you can use it to do complexity proofs. 633 00:39:26,380 --> 00:39:29,880 Here, we're just using it as a convenience in today's lecture 634 00:39:29,880 --> 00:39:32,640 to use our LP hammer. 635 00:39:32,640 --> 00:39:39,360 So let's say that I have our favorite problem 636 00:39:39,360 --> 00:39:42,670 of the week, namely max flow. 637 00:39:42,670 --> 00:39:45,590 And I want to convert that to LP. 638 00:39:45,590 --> 00:39:51,970 So go back a week ago, and right about this time a week ago, 639 00:39:51,970 --> 00:39:54,050 and we'd set up the max flow problem. 640 00:39:54,050 --> 00:39:56,690 And let's assume that we went back there. 641 00:39:56,690 --> 00:39:58,640 And we didn't talk about augmenting paths, 642 00:39:58,640 --> 00:40:02,310 and we didn't talk about residual capacities or min-cut 643 00:40:02,310 --> 00:40:06,120 or anything like that, but we knew LP already. 644 00:40:06,120 --> 00:40:09,950 And we just want to solve max flow using LP. 645 00:40:09,950 --> 00:40:10,710 So let's do that. 646 00:40:27,250 --> 00:40:29,410 So this is maximum flow. 647 00:40:29,410 --> 00:40:35,450 I'm not going to bother with converting to standard form. 648 00:40:35,450 --> 00:40:37,640 We know how to do that, given what we just 649 00:40:37,640 --> 00:40:41,440 did here, over there. 650 00:40:41,440 --> 00:40:46,990 So I'll just do whatever I want to keep things simple. 651 00:40:46,990 --> 00:40:50,160 Max flow is obviously a maximization problem. 652 00:40:50,160 --> 00:40:55,150 And using the same notation we've used, 653 00:40:55,150 --> 00:40:58,100 it's not going to look like Ax and b, 654 00:40:58,100 --> 00:41:02,884 just because I want you to recall what max flow is. 655 00:41:02,884 --> 00:41:04,300 And we're going to translate that. 656 00:41:04,300 --> 00:41:05,758 And the values of these variables-- 657 00:41:05,758 --> 00:41:08,560 or the names of these variables, whether they're x or f, 658 00:41:08,560 --> 00:41:10,070 it should really matter. 659 00:41:10,070 --> 00:41:14,020 We know how to do LP at this point-- we 660 00:41:14,020 --> 00:41:16,430 know how to formulate LP, I should say, at this point. 661 00:41:16,430 --> 00:41:21,500 And we're assuming that we have an LP solver. 662 00:41:21,500 --> 00:41:24,630 So what I want to do here with the maximum flow problem 663 00:41:24,630 --> 00:41:26,850 is maximizing the flow value. 664 00:41:26,850 --> 00:41:29,130 And it's simply, you grab the source, 665 00:41:29,130 --> 00:41:34,480 you have a variable associated with the flow from the source 666 00:41:34,480 --> 00:41:40,320 to every other vertex of v. And you have to maximize that. 667 00:41:40,320 --> 00:41:42,000 So that was the setup for max flow. 668 00:41:42,000 --> 00:41:44,240 I'm not changing that. 669 00:41:44,240 --> 00:41:46,960 What do you think the three constraints, or whatever 670 00:41:46,960 --> 00:41:48,730 set of constraints that we have here, 671 00:41:48,730 --> 00:41:50,870 are going to correspond to the LP? 672 00:41:50,870 --> 00:41:53,650 You spent a week on max flow, looked 673 00:41:53,650 --> 00:41:55,410 at the problem set, what constraints am I 674 00:41:55,410 --> 00:41:56,660 going to have to put up there? 675 00:41:59,190 --> 00:42:02,060 I'm going to have to put up capacity constraints. 676 00:42:02,060 --> 00:42:03,240 That's an obvious one. 677 00:42:03,240 --> 00:42:06,380 What is another one? 678 00:42:06,380 --> 00:42:08,810 Conservation constraints. 679 00:42:08,810 --> 00:42:13,360 All flow entering a node that is not the source or the sink 680 00:42:13,360 --> 00:42:14,210 has to leave it. 681 00:42:17,280 --> 00:42:20,140 In the original network, is there 682 00:42:20,140 --> 00:42:25,920 a concept of negative flow? 683 00:42:25,920 --> 00:42:29,350 No, you will define it going in the other direction. 684 00:42:29,350 --> 00:42:31,880 So we did talk about negative numbers, et cetera, 685 00:42:31,880 --> 00:42:35,260 but you're going to have positive quantities, especially 686 00:42:35,260 --> 00:42:39,000 if you look at net flow, the version of the flow 687 00:42:39,000 --> 00:42:45,340 that we zoomed in on in the Tuesday lecture from last week. 688 00:42:45,340 --> 00:42:48,579 And you also have-- in the general setting, 689 00:42:48,579 --> 00:42:51,120 you're going to have these skew symmetry constraints as well. 690 00:42:51,120 --> 00:42:53,530 So the three things that you need here 691 00:42:53,530 --> 00:43:03,125 are skew symmetry, conservation, and capacity. 692 00:43:06,830 --> 00:43:18,332 So you have such that f u, v equals minus f v, u for all u, 693 00:43:18,332 --> 00:43:21,040 v belonging to V. 694 00:43:21,040 --> 00:43:24,120 And depending on the kind of network that you have, 695 00:43:24,120 --> 00:43:27,270 if you constrain it to a sudden type 696 00:43:27,270 --> 00:43:29,960 that you don't have these two-way edges, 697 00:43:29,960 --> 00:43:33,000 you could certainly remove some, if not all, 698 00:43:33,000 --> 00:43:35,370 of these skew symmetry constraints. 699 00:43:35,370 --> 00:43:39,730 Important ones are conservation and capacity. 700 00:43:39,730 --> 00:43:42,780 And this should seem pretty familiar to you. 701 00:43:42,780 --> 00:43:45,500 But the key-- the reason I'm writing these all out 702 00:43:45,500 --> 00:43:49,020 is primarily to ensure that you understand that these are all 703 00:43:49,020 --> 00:43:51,030 linear constraints. 704 00:43:51,030 --> 00:43:53,660 So that's pretty much the only thing 705 00:43:53,660 --> 00:43:55,132 that you need to observe here. 706 00:43:55,132 --> 00:43:56,590 Obviously, these constraints you've 707 00:43:56,590 --> 00:44:01,530 seen many a time from the two lectures last week. 708 00:44:01,530 --> 00:44:04,690 But notice that they're all linear. 709 00:44:04,690 --> 00:44:08,900 And finally, this one is f u, v less than or equal to c of u, 710 00:44:08,900 --> 00:44:18,300 v for all u, v belonging to cap V. 711 00:44:18,300 --> 00:44:20,050 So this is f. 712 00:44:20,050 --> 00:44:22,760 That's a variable that's less than another constant, 713 00:44:22,760 --> 00:44:24,280 clearly linear. 714 00:44:24,280 --> 00:44:26,460 Doing a bunch of sums here. 715 00:44:26,460 --> 00:44:29,050 I could obviously have multipliers, scalar 716 00:44:29,050 --> 00:44:29,966 multipliers. 717 00:44:29,966 --> 00:44:31,340 In this case, for conservation, I 718 00:44:31,340 --> 00:44:35,240 don't have scalar multipliers, but clearly linear. 719 00:44:35,240 --> 00:44:38,510 Skew symmetry, got two variables in here. 720 00:44:38,510 --> 00:44:41,140 One of them is a negation of the other, clearly linear, 721 00:44:41,140 --> 00:44:43,820 so that's why this is an LP. 722 00:44:43,820 --> 00:44:46,480 And so you might say, well, I know better, 723 00:44:46,480 --> 00:44:49,250 max flow is much more efficient than any LP 724 00:44:49,250 --> 00:44:50,550 solver that's out there. 725 00:44:50,550 --> 00:44:52,360 And you would be right. 726 00:44:52,360 --> 00:44:56,250 If you have a max flow problem of this variety, 727 00:44:56,250 --> 00:44:58,560 it's difficult to imagine that you 728 00:44:58,560 --> 00:45:02,540 would get performance, empirical performance from running an LP 729 00:45:02,540 --> 00:45:03,500 solver. 730 00:45:03,500 --> 00:45:10,220 But this generalization of max flow 731 00:45:10,220 --> 00:45:14,000 that's a multi-commodity max flow, where you just don't have 732 00:45:14,000 --> 00:45:16,190 one commodity flowing through. 733 00:45:16,190 --> 00:45:19,680 You may be counting cars and trucks on a road, 734 00:45:19,680 --> 00:45:21,740 or there's two different kinds of liquid 735 00:45:21,740 --> 00:45:28,730 flowing through the same pipe, whatever, gas or liquid. 736 00:45:28,730 --> 00:45:31,230 And so when you have multiple commodities, 737 00:45:31,230 --> 00:45:36,210 you may have a linear but more complicated cost function 738 00:45:36,210 --> 00:45:39,957 that's a function of the flow of each of the commodities. 739 00:45:39,957 --> 00:45:42,290 And they may have a certain weight associated with them. 740 00:45:42,290 --> 00:45:45,250 So there's a lot of things that could 741 00:45:45,250 --> 00:45:51,170 be more general-- there could be more general settings 742 00:45:51,170 --> 00:45:52,520 corresponding to max flow. 743 00:45:52,520 --> 00:45:57,550 And I'll just leave you with the thought 744 00:45:57,550 --> 00:46:01,210 that you could simply have two commodities. 745 00:46:06,470 --> 00:46:08,010 And we'll just call them 1 and 2. 746 00:46:10,640 --> 00:46:13,730 And so now, you have the f1's and the c1's and the f2's and 747 00:46:13,730 --> 00:46:15,750 the c2's. 748 00:46:15,750 --> 00:46:19,540 Each commodity has to be conserved. 749 00:46:19,540 --> 00:46:21,277 But what about the capacity? 750 00:46:21,277 --> 00:46:23,110 What do you think happens with the capacity? 751 00:46:23,110 --> 00:46:27,170 Let's just assume these are two different kinds of cars. 752 00:46:27,170 --> 00:46:29,699 So what would the capacity constraint look like? 753 00:46:33,701 --> 00:46:34,200 Yeah. 754 00:46:34,200 --> 00:46:36,829 AUDIENCE: You can say either c1 or f1 755 00:46:36,829 --> 00:46:43,022 plus f2 is-- for each edge, you can add them together 756 00:46:43,022 --> 00:46:44,730 or you might take the linear [INAUDIBLE]. 757 00:46:44,730 --> 00:46:46,396 SRINIVAS DEVADAS: Exactly, that's right. 758 00:46:46,396 --> 00:46:47,890 So good point. 759 00:46:47,890 --> 00:46:51,790 It may be the case that I have distinct capacities. 760 00:46:51,790 --> 00:46:55,720 And in fact, if you have completely disjoint problems, 761 00:46:55,720 --> 00:46:59,110 you're right in that you can solve them separately. 762 00:46:59,110 --> 00:47:01,790 But actually, the more interesting case 763 00:47:01,790 --> 00:47:05,560 would be that you have a single capacity c, 764 00:47:05,560 --> 00:47:09,610 so you'll have-- let me just write this out here. 765 00:47:16,250 --> 00:47:18,420 If in fact you had two distinct things, 766 00:47:18,420 --> 00:47:22,870 so if you had f1, c1, f1, c2, the question 767 00:47:22,870 --> 00:47:27,890 is, do you have two distinct, disjoint optimizations, 768 00:47:27,890 --> 00:47:32,920 in which case you just use max flow twice. 769 00:47:32,920 --> 00:47:35,380 On other hand, what's more interesting really-- 770 00:47:35,380 --> 00:47:38,140 and I should've used this example for starters-- 771 00:47:38,140 --> 00:47:39,580 but here's a better one. 772 00:47:39,580 --> 00:47:42,930 You have two commodities and a single capacity. 773 00:47:42,930 --> 00:47:44,870 So the road is a good example. 774 00:47:44,870 --> 00:47:48,650 Both the cars and the trucks share the same road. 775 00:47:48,650 --> 00:47:49,990 It has a certain capacity. 776 00:47:49,990 --> 00:47:52,480 And now, your capacity constraint 777 00:47:52,480 --> 00:47:58,390 is looking like f1 plus f2 over here. 778 00:47:58,390 --> 00:48:04,440 And that's the flow through the particular edge uv. 779 00:48:04,440 --> 00:48:07,380 So you have something like f1 u, v plus f2 u, 780 00:48:07,380 --> 00:48:11,365 v is less than or equal to c u, v for this total capacity. 781 00:48:14,190 --> 00:48:16,560 And that's pretty much it. 782 00:48:16,560 --> 00:48:18,900 So that is linear. 783 00:48:18,900 --> 00:48:20,440 The nice thing is that it's linear. 784 00:48:20,440 --> 00:48:22,350 You could put weights on it. 785 00:48:22,350 --> 00:48:28,280 If you wanted to claim that a particular commodity 786 00:48:28,280 --> 00:48:30,590 1 uses up-- because it's a truck, 787 00:48:30,590 --> 00:48:33,630 it uses up more space on the road. 788 00:48:33,630 --> 00:48:35,530 And you can accommodate fewer of them. 789 00:48:35,530 --> 00:48:37,120 You could put a multiplier in there. 790 00:48:37,120 --> 00:48:39,160 Still say it's linear. 791 00:48:39,160 --> 00:48:43,740 So that's the power of by having an LP engine. 792 00:48:43,740 --> 00:48:45,480 You could translate problems that 793 00:48:45,480 --> 00:48:48,670 are not exactly max flow, that are multi-commodity flow. 794 00:48:48,670 --> 00:48:52,120 You may have additional linear constraints that you could add, 795 00:48:52,120 --> 00:48:54,650 and you could still use your LP package. 796 00:48:54,650 --> 00:48:59,290 So that's the reason why this is interesting and powerful. 797 00:48:59,290 --> 00:49:04,530 So that is kind of an obvious, corresponding to max flow. 798 00:49:04,530 --> 00:49:08,380 Let's look at something that's a little less obvious. 799 00:49:08,380 --> 00:49:12,690 And it's going to be a little tricky to convert the shortest 800 00:49:12,690 --> 00:49:22,400 path problem to LP, not a lot of work 801 00:49:22,400 --> 00:49:25,624 but one little observation that's 802 00:49:25,624 --> 00:49:28,040 going to be important to make in order for the whole thing 803 00:49:28,040 --> 00:49:32,472 to flow through or actually work out. 804 00:49:32,472 --> 00:49:34,180 So we all know the shortest path problem. 805 00:49:38,230 --> 00:49:41,684 We want to find-- let's just call it the single source 806 00:49:41,684 --> 00:49:42,600 shortest path problem. 807 00:49:42,600 --> 00:49:44,050 You have a specific source. 808 00:49:44,050 --> 00:49:48,510 That's going to turn into the point from which you're 809 00:49:48,510 --> 00:49:52,210 going to start computing the distance. 810 00:49:52,210 --> 00:49:55,900 That's what Dijkstra does, and that's Bellman-Ford does. 811 00:49:55,900 --> 00:50:02,930 And so this is from vertex x-- s, excuse me, s. 812 00:50:02,930 --> 00:50:07,320 And what I want to do here is obviously set it up 813 00:50:07,320 --> 00:50:11,440 as a set of linear constraints. 814 00:50:11,440 --> 00:50:23,980 If I have dv corresponding to the distance from the source-- 815 00:50:23,980 --> 00:50:28,050 so dv represents the distance from the source-- 816 00:50:28,050 --> 00:50:32,030 and eventually I want dv to be the shortest 817 00:50:32,030 --> 00:50:33,900 distance from the source. 818 00:50:33,900 --> 00:50:37,260 That's our notation for shortest paths. 819 00:50:37,260 --> 00:50:40,510 dv represents an existing path-- it may not 820 00:50:40,510 --> 00:50:44,580 be the shortest path-- from s to v, the value of that. 821 00:50:44,580 --> 00:50:51,800 But dv monotonically decreases as you run through. 822 00:50:51,800 --> 00:50:54,160 It's initially infinity in Dijkstra 823 00:50:54,160 --> 00:50:55,690 going back to Dijkstra. 824 00:50:55,690 --> 00:50:59,740 And then we shrink it through a process of relaxation. 825 00:50:59,740 --> 00:51:02,650 Now I want to try and model that-- I want to try and model 826 00:51:02,650 --> 00:51:05,170 all of this as an LP. 827 00:51:05,170 --> 00:51:07,400 So it's not immediately obvious-- 828 00:51:07,400 --> 00:51:09,820 the thing that the flow networks had, 829 00:51:09,820 --> 00:51:11,620 where we had these constraints. 830 00:51:11,620 --> 00:51:14,640 We have capacity constraints and conservation constraints. 831 00:51:14,640 --> 00:51:18,680 And we could turn that constraint into an inequality. 832 00:51:18,680 --> 00:51:20,600 And it was pretty smooth. 833 00:51:20,600 --> 00:51:22,030 It's pretty easy. 834 00:51:22,030 --> 00:51:26,100 So what I need to do here with shortest paths 835 00:51:26,100 --> 00:51:30,630 is something that's a little more subtle. 836 00:51:30,630 --> 00:51:36,070 So what basic constraint do I have in a shortest path 837 00:51:36,070 --> 00:51:38,770 algorithm? 838 00:51:38,770 --> 00:51:41,970 What's an inequality-- you remember an inequality 839 00:51:41,970 --> 00:51:45,070 from shortest paths that we kept talking about? 840 00:51:45,070 --> 00:51:46,980 The triangle inequality. 841 00:51:46,980 --> 00:51:49,890 So we're going to have to go with the triangle inequality 842 00:51:49,890 --> 00:51:53,870 and take the triangle inequality and use that to create an LP 843 00:51:53,870 --> 00:51:56,410 formulation of shortest paths. 844 00:51:56,410 --> 00:51:59,500 In particular, what we have here is 845 00:51:59,500 --> 00:52:05,180 that I could write dv minus du is 846 00:52:05,180 --> 00:52:13,660 less than or equal to w u, v for all u, v belonging to E. 847 00:52:13,660 --> 00:52:15,810 And that's the triangle inequality. 848 00:52:19,500 --> 00:52:23,094 And I'm going to have d of s equals 0. 849 00:52:23,094 --> 00:52:24,760 That's the only thing that I start with. 850 00:52:27,590 --> 00:52:30,020 And so what's happening out here is simply 851 00:52:30,020 --> 00:52:32,340 that there's different ways of getting 852 00:52:32,340 --> 00:52:37,820 to v. And my shortest path is going to be the best 853 00:52:37,820 --> 00:52:40,760 way of getting to v. So in particular, 854 00:52:40,760 --> 00:52:44,252 the way you want to think about this is that if I have a v 855 00:52:44,252 --> 00:52:51,380 and I can get to it from, let's just say, u1 and u2. 856 00:52:51,380 --> 00:52:53,360 And maybe the source is over here. 857 00:52:53,360 --> 00:52:56,300 And these are the only two edges that can get to v. 858 00:52:56,300 --> 00:52:59,270 So I'm just looking at a fairly limited setting. 859 00:52:59,270 --> 00:53:03,030 u1 and u2 are going to have to be the two vertices. 860 00:53:03,030 --> 00:53:06,130 One of these two is going to get me to v. 861 00:53:06,130 --> 00:53:12,530 And I got w u1, v here. 862 00:53:12,530 --> 00:53:18,290 And I got w u2, v over here. 863 00:53:18,290 --> 00:53:20,340 And so what this is saying is, I'm 864 00:53:20,340 --> 00:53:23,770 going to have to write this out for each of these edges. 865 00:53:23,770 --> 00:53:29,230 For each of these edges, I'm going to have this constraint. 866 00:53:29,230 --> 00:53:33,050 And that's says that the dv value, if I want the shortest 867 00:53:33,050 --> 00:53:38,310 path, should obey both of these constraints. 868 00:53:38,310 --> 00:53:40,830 And if I want to obey both of these constraints, one of them 869 00:53:40,830 --> 00:53:43,030 is going to be my limiting constraint. 870 00:53:43,030 --> 00:53:45,790 And I'm going to get the min of those two. 871 00:53:45,790 --> 00:53:46,450 Correct? 872 00:53:46,450 --> 00:53:49,260 So in effect what this translates to 873 00:53:49,260 --> 00:53:50,950 is that it's an AND, right? 874 00:53:50,950 --> 00:53:57,550 So dv have minus du1 is less than or equal to w u1, 875 00:53:57,550 --> 00:54:06,920 v. dv minus du2 is less than or equal to w u2, v. That's 876 00:54:06,920 --> 00:54:09,760 an AND, because I'm putting both of those constraints in here. 877 00:54:09,760 --> 00:54:15,140 And that essentially means that dv 878 00:54:15,140 --> 00:54:23,190 is going to be the min of the two quantities-- 879 00:54:23,190 --> 00:54:27,500 the du1 quantity plus the w u1, v, 880 00:54:27,500 --> 00:54:33,460 and the du2 quantity plus the w u2, v. That make sense? 881 00:54:33,460 --> 00:54:37,577 Ask me questions if this is unclear. 882 00:54:37,577 --> 00:54:39,160 So that simply corresponds to the fact 883 00:54:39,160 --> 00:54:40,630 that I'm doing an AND over here. 884 00:54:40,630 --> 00:54:42,620 I'm adding all of these constraints in there. 885 00:54:42,620 --> 00:54:47,310 So I'm applying the triangle inequality to every edge, 886 00:54:47,310 --> 00:54:53,730 to every relationship between a vertex that 887 00:54:53,730 --> 00:54:55,880 has a path ending at it. 888 00:54:55,880 --> 00:54:58,780 And you're pushing it forward to this vertex v, 889 00:54:58,780 --> 00:55:01,880 all the different ways that you can get to v. In this case, 890 00:55:01,880 --> 00:55:05,510 there is two sets of ways-- one from u1 and one from u2. 891 00:55:05,510 --> 00:55:10,820 And the last step is a minimization step. 892 00:55:10,820 --> 00:55:13,820 So you think you're done or we're done, 893 00:55:13,820 --> 00:55:16,130 but we're not quite done, because what's 894 00:55:16,130 --> 00:55:20,167 missing here in terms of my formulation of LP? 895 00:55:20,167 --> 00:55:21,458 What else do I have to do here? 896 00:55:25,950 --> 00:55:30,154 Well, sure, non-negative-- let's do that. 897 00:55:30,154 --> 00:55:30,653 Sorry? 898 00:55:30,653 --> 00:55:31,486 AUDIENCE: Objective. 899 00:55:31,486 --> 00:55:35,390 SRINIVAS DEVADAS: Objective, who said objective? 900 00:55:35,390 --> 00:55:35,930 You again? 901 00:55:38,550 --> 00:55:42,210 So we are missing an objective function. 902 00:55:42,210 --> 00:55:46,310 Now shortest path is what kind of problem again? 903 00:55:46,310 --> 00:55:50,050 Short means minimum? 904 00:55:50,050 --> 00:55:52,540 Minimum, height whatever. 905 00:55:52,540 --> 00:55:56,240 So do I put a-- what happens if I put a minimum in here? 906 00:55:56,240 --> 00:56:00,070 And let's say that I do something like sigma 907 00:56:00,070 --> 00:56:04,470 over V, cap V, dv, because I want to minimize-- 908 00:56:04,470 --> 00:56:06,010 or I could pick a particular one. 909 00:56:06,010 --> 00:56:10,010 I could pick a particular single source, single destination, 910 00:56:10,010 --> 00:56:14,340 and I put a minimum there. 911 00:56:14,340 --> 00:56:15,900 What happens? 912 00:56:15,900 --> 00:56:18,230 Does this work? 913 00:56:18,230 --> 00:56:19,730 What's the solution to this problem, 914 00:56:19,730 --> 00:56:23,510 if I minimize the distance? 915 00:56:23,510 --> 00:56:28,170 0, because the zero value is going to work. 916 00:56:28,170 --> 00:56:31,990 So there's something-- I haven't put in the constraint 917 00:56:31,990 --> 00:56:33,810 that I do want a path. 918 00:56:33,810 --> 00:56:40,940 I do want a path from s to v for any v that 919 00:56:40,940 --> 00:56:46,700 matches one of the quantities in the min, 920 00:56:46,700 --> 00:56:49,920 because the min says I have equality. 921 00:56:49,920 --> 00:56:53,810 The big issue here is, this is a less than or equal to, 922 00:56:53,810 --> 00:56:57,620 and that's why the min doesn't work in the objective function. 923 00:56:57,620 --> 00:56:59,060 It's a less than or equal to. 924 00:56:59,060 --> 00:57:02,410 But this min over here, which is the definition of a shortest 925 00:57:02,410 --> 00:57:06,660 path, is saying that it's either equal to this quantity 926 00:57:06,660 --> 00:57:08,390 or equal to that quantity. 927 00:57:08,390 --> 00:57:12,610 There's an equality over here that is missing from this side. 928 00:57:12,610 --> 00:57:14,450 And that's the key observation. 929 00:57:14,450 --> 00:57:16,810 Once you observe that, that need equality 930 00:57:16,810 --> 00:57:21,850 for one of these constituent quantities of the min, 931 00:57:21,850 --> 00:57:24,930 then you'll see that what you have to do 932 00:57:24,930 --> 00:57:28,396 is simply change this min to a max. 933 00:57:30,834 --> 00:57:32,250 So you say, well, how the heck did 934 00:57:32,250 --> 00:57:34,240 a min get changed into a max? 935 00:57:34,240 --> 00:57:37,125 And I'm not sure I'm going to convince every on of you 936 00:57:37,125 --> 00:57:39,050 in the next minute or so. 937 00:57:39,050 --> 00:57:42,360 But the bottom line is, it comes down 938 00:57:42,360 --> 00:57:47,930 to I do have a min already in my inequalities, 939 00:57:47,930 --> 00:57:50,520 because I'm ANDing each of these inequalities, 940 00:57:50,520 --> 00:57:53,220 and I'm putting down each of those inequalities in there. 941 00:57:53,220 --> 00:57:57,020 So each of them is going to force me to find the best 942 00:57:57,020 --> 00:58:00,220 solution, because they're going to constrain me to not 943 00:58:00,220 --> 00:58:03,790 go via u2 if u1 is better, because the other constraint 944 00:58:03,790 --> 00:58:06,857 corresponding to u1 is going to force me down. 945 00:58:06,857 --> 00:58:08,440 So there's an additional min in there, 946 00:58:08,440 --> 00:58:11,170 because the ANDing of the less than or equal tos. 947 00:58:11,170 --> 00:58:16,780 And then in order to actually force the equality for one 948 00:58:16,780 --> 00:58:19,715 of those, I need to push up as hard as I can, 949 00:58:19,715 --> 00:58:21,830 or as high as I can. 950 00:58:21,830 --> 00:58:23,360 So think about it. 951 00:58:23,360 --> 00:58:25,880 Play around with a couple of examples. 952 00:58:25,880 --> 00:58:30,130 Choose a simple example for starters. 953 00:58:30,130 --> 00:58:34,210 And you'll see that this is the correct formulation. 954 00:58:34,210 --> 00:58:39,470 So you can see that it's not completely clear in some cases 955 00:58:39,470 --> 00:58:41,580 how to transform problems to LP. 956 00:58:41,580 --> 00:58:45,230 But even in those cases, sometimes you can. 957 00:58:45,230 --> 00:58:48,110 So there's just a ton of different problems, 958 00:58:48,110 --> 00:58:53,340 a good skill to have to be able to take 959 00:58:53,340 --> 00:58:55,390 combinatorial optimization engines, 960 00:58:55,390 --> 00:58:58,840 like LP or even max flow, and be able to translate problems 961 00:58:58,840 --> 00:58:59,362 to them. 962 00:58:59,362 --> 00:59:00,820 It's something that you'll probably 963 00:59:00,820 --> 00:59:04,970 do if you stick to algorithms in your careers 964 00:59:04,970 --> 00:59:11,830 or exploiting available algorithm packages. 965 00:59:14,900 --> 00:59:17,650 So the last thing I want to do here for the rest of the time 966 00:59:17,650 --> 00:59:24,920 is to give you some sense for how an LP program is actually 967 00:59:24,920 --> 00:59:25,550 optimized. 968 00:59:25,550 --> 00:59:30,370 How can you possibly take the standard LP formulation, 969 00:59:30,370 --> 00:59:31,720 which is a general setting. 970 00:59:31,720 --> 00:59:33,730 You know nothing about shortest paths, 971 00:59:33,730 --> 00:59:36,260 let's assume, nothing about max flow. 972 00:59:36,260 --> 00:59:37,950 It's not about a specific problem. 973 00:59:37,950 --> 00:59:39,360 It's about the general setting. 974 00:59:39,360 --> 00:59:41,216 How can we solve the general setting, 975 00:59:41,216 --> 00:59:42,840 because that was the theme here anyway. 976 00:59:42,840 --> 00:59:46,650 You had this engine, and you want to use this engine. 977 00:59:46,650 --> 00:59:48,780 But now, how do you build this engine? 978 00:59:48,780 --> 00:59:54,000 So what we're going to do is look at a fairly simple example 979 00:59:54,000 --> 00:59:56,520 of the simplex algorithm. 980 00:59:56,520 --> 01:00:01,260 And this algorithm is in the textbook. 981 01:00:01,260 --> 01:00:02,700 And it'll be in my notes. 982 01:00:02,700 --> 01:00:04,930 So I'll get as far as I can. 983 01:00:04,930 --> 01:00:07,840 It's not that complicated to describe, especially 984 01:00:07,840 --> 01:00:11,530 from an example standpoint. 985 01:00:11,530 --> 01:00:14,660 But I may not get through all of the steps 986 01:00:14,660 --> 01:00:17,580 to get you the optimum for this particular example 987 01:00:17,580 --> 01:00:20,260 given how much time we have. 988 01:00:20,260 --> 01:00:26,420 The most important concept in simplex 989 01:00:26,420 --> 01:00:31,020 is yet another form of representation 990 01:00:31,020 --> 01:00:36,350 for simplex, which says that you can represent the LP, not 991 01:00:36,350 --> 01:00:39,990 in standard form, but in slack form. 992 01:00:39,990 --> 01:00:42,292 So I'm going to tell you what slack form is. 993 01:00:42,292 --> 01:00:43,750 And then what we're going to do is, 994 01:00:43,750 --> 01:00:48,110 the flow of simplex, algorithmic flow, 995 01:00:48,110 --> 01:00:56,360 is to convert one slack form into and equivalent. 996 01:00:56,360 --> 01:01:01,000 Obviously, you don't want to do something that's incorrect, 997 01:01:01,000 --> 01:01:02,770 but it has to be an equivalent slack 998 01:01:02,770 --> 01:01:22,715 form, whose objective value has not decreased and has likely 999 01:01:22,715 --> 01:01:23,215 increased. 1000 01:01:27,340 --> 01:01:29,890 So you're guaranteed that the objective value has not 1001 01:01:29,890 --> 01:01:30,390 decreased. 1002 01:01:30,390 --> 01:01:32,890 You're not guaranteed that it's increased. 1003 01:01:32,890 --> 01:01:43,510 And then we're going to keep going till the optimal solution 1004 01:01:43,510 --> 01:01:46,030 becomes obvious. 1005 01:01:46,030 --> 01:01:49,994 And you might say, how is this obvious? 1006 01:01:49,994 --> 01:01:52,410 That's the reason why I talked about the short certificate 1007 01:01:52,410 --> 01:01:53,730 of optimality. 1008 01:01:53,730 --> 01:01:58,750 It's definitely a relationship between the termination 1009 01:01:58,750 --> 01:02:01,610 of simplex and the fact that you can now say, 1010 01:02:01,610 --> 01:02:04,930 hey, I know I'm done here, it's kind of obvious 1011 01:02:04,930 --> 01:02:06,590 that I can't do any better. 1012 01:02:06,590 --> 01:02:09,090 And hopefully, you'll see that by the end of this lecture 1013 01:02:09,090 --> 01:02:11,790 in this simple example. 1014 01:02:11,790 --> 01:02:12,490 So that's it. 1015 01:02:12,490 --> 01:02:14,040 It's an iterative algorithm. 1016 01:02:14,040 --> 01:02:17,330 It's exponential, unfortunately, because this 1017 01:02:17,330 --> 01:02:24,460 takes m plus n, choose n iterations in the worst case, 1018 01:02:24,460 --> 01:02:26,280 where n is the number of variables 1019 01:02:26,280 --> 01:02:29,080 and m is the number of constraints. 1020 01:02:29,080 --> 01:02:31,160 Most of the time, it does a lot better, 1021 01:02:31,160 --> 01:02:34,590 but that's the only bound that you can actually 1022 01:02:34,590 --> 01:02:37,290 prove in the worst case. 1023 01:02:37,290 --> 01:02:40,490 And so you're stuck with an exponential algorithm 1024 01:02:40,490 --> 01:02:43,500 if you're using simplex worst case. 1025 01:02:43,500 --> 01:02:46,350 We won't actually do much analysis on simplex. 1026 01:02:46,350 --> 01:02:50,290 It's really out of scope for 046 in terms of the analysis. 1027 01:02:50,290 --> 01:02:53,390 The actual algorithm is certainly within scope. 1028 01:02:53,390 --> 01:02:56,550 So what I want to do is give you some sense for what 1029 01:02:56,550 --> 01:02:58,002 the slack form looks like. 1030 01:02:58,002 --> 01:02:59,960 And we'll do a couple of iterations of simplex. 1031 01:02:59,960 --> 01:03:04,786 And we'll get as far as we can before the end the lecture. 1032 01:03:04,786 --> 01:03:06,160 So we'll take a different example 1033 01:03:06,160 --> 01:03:12,340 from our political example. 1034 01:03:12,340 --> 01:03:15,430 It's similar in size. 1035 01:03:15,430 --> 01:03:20,370 And I want to explain to you what the slack form is 1036 01:03:20,370 --> 01:03:22,720 and why it's interesting. 1037 01:03:22,720 --> 01:03:35,510 So what I want to do is maximize 3x1 plus x2 plus x3 subject 1038 01:03:35,510 --> 01:03:40,920 to the constraints that x1 plus x2 plus 3x2 1039 01:03:40,920 --> 01:03:47,800 is less than or equal to 30, 2x1 plus 2x2 plus 5x3 is 1040 01:03:47,800 --> 01:03:55,250 less than or equal to 24, 4x1 plus x2 plus 2x3 less 1041 01:03:55,250 --> 01:03:59,900 than or equal to 36, and then non-negativity constraints, 1042 01:03:59,900 --> 01:04:03,200 x1, x2, x3 greater than or equal to 0. 1043 01:04:03,200 --> 01:04:05,792 So that's our example problem. 1044 01:04:05,792 --> 01:04:06,750 I'll leave it up there. 1045 01:04:09,410 --> 01:04:12,070 You're going to convert this to slack form. 1046 01:04:12,070 --> 01:04:13,960 And so what is the slack form? 1047 01:04:13,960 --> 01:04:19,130 We're going to introduce an additional number of variables 1048 01:04:19,130 --> 01:04:22,230 that correspond to the number of equations that we have. 1049 01:04:22,230 --> 01:04:25,040 So we're going to introduce, in this case, three new labels, 1050 01:04:25,040 --> 01:04:27,640 because I have three equations. 1051 01:04:27,640 --> 01:04:33,300 And the slack for this problem looks like this. 1052 01:04:33,300 --> 01:04:39,480 I'm going to have z equals 3x1 plus x2 1053 01:04:39,480 --> 01:04:42,470 plus x3, same as before. 1054 01:04:42,470 --> 01:04:44,920 And then I'm going to have variables 1055 01:04:44,920 --> 01:04:48,000 that represent-- these are called basic variables. 1056 01:04:48,000 --> 01:04:51,520 And the original variables are called non-basic variables. 1057 01:04:51,520 --> 01:04:55,090 So I'm going to add three basic labels, x4, x5, 1058 01:04:55,090 --> 01:04:58,350 and x6, corresponding to these three constraints. 1059 01:04:58,350 --> 01:05:01,470 And they're going to represent slack in the sense 1060 01:05:01,470 --> 01:05:04,880 that they're going to correspond to how much slack you 1061 01:05:04,880 --> 01:05:09,290 have in the inequalities that you 1062 01:05:09,290 --> 01:05:11,400 have in the original problem. 1063 01:05:11,400 --> 01:05:17,220 So if x4 happens to be 0, then you're jammed up. 1064 01:05:17,220 --> 01:05:22,450 You have no slack, because x1 plus x2 plus 3x3 equals 30. 1065 01:05:22,450 --> 01:05:26,790 And increasing any one of them will violate the constraint. 1066 01:05:26,790 --> 01:05:29,130 So that's just simply the notion of slack. 1067 01:05:29,130 --> 01:05:32,310 It's how much room do you have. 1068 01:05:32,310 --> 01:05:42,220 x5 is 24 minus 2x1 minus 2x2 minus 5x3. 1069 01:05:42,220 --> 01:05:47,460 And the last one is 36 minus 4x1. 1070 01:05:47,460 --> 01:05:50,230 This is very mechanical up to this point. 1071 01:05:50,230 --> 01:05:53,880 And so I'll call this set of equations, equation I. 1072 01:05:53,880 --> 01:05:57,650 And what I'm going to do is I'm going 1073 01:05:57,650 --> 01:06:05,980 to now work on a space that corresponds to x1, x2, x3, x4, 1074 01:06:05,980 --> 01:06:08,370 x5, x6. 1075 01:06:08,370 --> 01:06:10,790 So I'm going to have these solutions that 1076 01:06:10,790 --> 01:06:13,516 now have six values associated with them, 1077 01:06:13,516 --> 01:06:15,390 as opposed to just three values, because I've 1078 01:06:15,390 --> 01:06:20,530 added three variables, the basic variables, to my non-basic set. 1079 01:06:20,530 --> 01:06:26,390 So the original variables are non-basic, 1080 01:06:26,390 --> 01:06:28,630 just to differentiate. 1081 01:06:28,630 --> 01:06:32,280 So so far it's just set up its definitions. 1082 01:06:32,280 --> 01:06:39,260 We can think about up running through iterations of simplex. 1083 01:06:39,260 --> 01:06:41,430 It takes about three iterations here 1084 01:06:41,430 --> 01:06:47,630 in order to get to the point where the optimum is obvious. 1085 01:06:47,630 --> 01:06:50,320 So you're going to convert through three slack forms. 1086 01:06:50,320 --> 01:06:53,680 And then finally, when you get to this fourth slack form, 1087 01:06:53,680 --> 01:06:56,100 you see that you have an optimal solution. 1088 01:06:56,100 --> 01:06:58,740 And how does that work? 1089 01:06:58,740 --> 01:07:04,660 You're going to have the notion of a basic solution, where 1090 01:07:04,660 --> 01:07:11,985 we set all non-basic variables to 0. 1091 01:07:19,200 --> 01:07:21,830 So in this case, what we're going to have-- 1092 01:07:21,830 --> 01:07:26,105 and then we're going to compute values of the basic variables. 1093 01:07:31,880 --> 01:07:36,900 So our objective function here is going to be 3 times 1094 01:07:36,900 --> 01:07:41,755 0 plus 1 times 0 plus 2 times 0, which is obviously 0. 1095 01:07:44,900 --> 01:07:53,430 And the values x4, of course, is going 1096 01:07:53,430 --> 01:07:56,590 to be-- because all of these are 0-- is going to be 30. 1097 01:07:56,590 --> 01:07:58,140 x5 is going to be 24. 1098 01:07:58,140 --> 01:07:59,920 And x6 is going to be 36. 1099 01:07:59,920 --> 01:08:02,450 So this is kind of a trivial starting point. 1100 01:08:02,450 --> 01:08:05,660 So you can think of this as 0, 0, 0, 1101 01:08:05,660 --> 01:08:13,360 the solution that you're looking at, 30 24, and 36. 1102 01:08:13,360 --> 01:08:18,140 So that's our starting point, which doesn't really 1103 01:08:18,140 --> 01:08:20,290 tell you much. 1104 01:08:20,290 --> 01:08:23,189 But now comes the key step, where 1105 01:08:23,189 --> 01:08:26,010 we're going to do something that's called pivoting. 1106 01:08:26,010 --> 01:08:29,830 And in pivoting, you're going to swap 1107 01:08:29,830 --> 01:08:33,160 a basic able with a non-basic variable. 1108 01:08:33,160 --> 01:08:37,630 It is a step that requires some intelligence. 1109 01:08:37,630 --> 01:08:41,609 But the basic step is a swap. 1110 01:08:41,609 --> 01:08:43,410 So one of the basic variables is going 1111 01:08:43,410 --> 01:08:47,470 to become a non-basic variable and vice versa. 1112 01:08:47,470 --> 01:08:51,359 And how do we select this? 1113 01:08:51,359 --> 01:08:54,500 Initially, you can kind of do this in an arbitrary way. 1114 01:08:54,500 --> 01:08:57,880 It gets a little more subtle as you go along. 1115 01:08:57,880 --> 01:09:02,840 You don't want to do things in a random way. 1116 01:09:02,840 --> 01:09:11,200 But let's just start with what pivoting actually 1117 01:09:11,200 --> 01:09:16,260 does in a more generic setting. 1118 01:09:16,260 --> 01:09:24,550 The basic step is select a non-basic variable-- let's 1119 01:09:24,550 --> 01:09:33,170 call it x sub e-- who's coefficient in the objective 1120 01:09:33,170 --> 01:09:38,174 function is positive. 1121 01:09:42,189 --> 01:09:45,930 You can always redefine things if you can't find something 1122 01:09:45,930 --> 01:09:46,890 like that. 1123 01:09:46,890 --> 01:09:48,523 But we won't go there. 1124 01:09:48,523 --> 01:09:49,939 And then what we're going to do is 1125 01:09:49,939 --> 01:09:57,745 we're going to increase the value of x as much as possible. 1126 01:10:00,430 --> 01:10:03,330 And we always have constraints, of course. 1127 01:10:03,330 --> 01:10:07,362 So we can do that without violating 1128 01:10:07,362 --> 01:10:08,320 any of the constraints. 1129 01:10:15,320 --> 01:10:24,910 And then at this point, variable xe becomes basic. 1130 01:10:24,910 --> 01:10:27,230 So it's going to turn into the left hand side. 1131 01:10:27,230 --> 01:10:28,230 It's going to move over. 1132 01:10:28,230 --> 01:10:31,890 You're going to swap-- the x1 might be over here, over here, 1133 01:10:31,890 --> 01:10:35,290 and you got to rewrite these equations so the x1 becomes 1134 01:10:35,290 --> 01:10:38,280 basic, for example, over to the left hand side, 1135 01:10:38,280 --> 01:10:41,250 and the variable that it replaced 1136 01:10:41,250 --> 01:10:42,760 goes over to the right hand side. 1137 01:10:42,760 --> 01:10:45,640 So you can think of this as Gaussian elimination, 1138 01:10:45,640 --> 01:10:47,700 except with inequalities. 1139 01:10:47,700 --> 01:10:49,960 There's definitely relationships there, 1140 01:10:49,960 --> 01:10:52,430 if you recall your Gaussian elimination. 1141 01:10:52,430 --> 01:10:55,280 If you don't, don't worry about it. 1142 01:10:55,280 --> 01:11:02,160 So xe becomes basic, and some other variable 1143 01:11:02,160 --> 01:11:03,350 becomes non-basic. 1144 01:11:10,420 --> 01:11:15,410 The values of other basic variables 1145 01:11:15,410 --> 01:11:24,023 and the objective function may change. 1146 01:11:28,920 --> 01:11:33,530 So we'll do one pivot step at least, 1147 01:11:33,530 --> 01:11:35,920 so you get a sense for the algebra involved. 1148 01:11:35,920 --> 01:11:40,420 And it becomes a little more concrete. 1149 01:11:40,420 --> 01:11:43,080 To motivate you further, you'll be 1150 01:11:43,080 --> 01:11:47,985 doing this in problem set 8 on a different example. 1151 01:11:51,580 --> 01:11:54,830 So what did I do here is we're going 1152 01:11:54,830 --> 01:11:59,720 to select a non-basic variable. 1153 01:11:59,720 --> 01:12:02,970 So let's just select x1, lexicographic order 1154 01:12:02,970 --> 01:12:09,600 or numeric order, let's select x1 is selected. 1155 01:12:09,600 --> 01:12:10,940 That's the non-basic variable. 1156 01:12:13,590 --> 01:12:19,680 And what I want to do is increase the value of x1. 1157 01:12:19,680 --> 01:12:23,080 So I want to increase it without violating constraints. 1158 01:12:29,450 --> 01:12:31,850 Now, which of these constraints do you think 1159 01:12:31,850 --> 01:12:35,710 is going to cause trouble first with respect 1160 01:12:35,710 --> 01:12:37,730 to increasing the value of x1? 1161 01:12:37,730 --> 01:12:39,360 x1 is now 0. 1162 01:12:39,360 --> 01:12:45,440 We're at ground level, we have all 0, things are feasible. 1163 01:12:45,440 --> 01:12:49,490 Now, as I start increasing x1, remember, 1164 01:12:49,490 --> 01:12:51,860 you have non-negativity constraints 1165 01:12:51,860 --> 01:12:55,240 associated with each of the x4 values as well. 1166 01:12:55,240 --> 01:12:57,840 That's what these basic variables represent. 1167 01:12:57,840 --> 01:13:02,080 So don't forget the fact that the constraints are violated. 1168 01:13:02,080 --> 01:13:06,860 You need all the xi's to be greater than or equal to 0. 1169 01:13:06,860 --> 01:13:09,760 That was true for the non-basic variables. 1170 01:13:09,760 --> 01:13:11,870 It's also true for the basic variables. 1171 01:13:11,870 --> 01:13:14,240 So a violation of a constraint implies 1172 01:13:14,240 --> 01:13:19,780 that one of the currently basic variables goes negative. 1173 01:13:19,780 --> 01:13:23,380 That's exactly equivalent to the original inequality 1174 01:13:23,380 --> 01:13:25,400 not being satisfied. 1175 01:13:25,400 --> 01:13:27,360 So which of these constraints, do you think, 1176 01:13:27,360 --> 01:13:29,240 is going to cause trouble here? 1177 01:13:29,240 --> 01:13:33,440 Just look at it, and should be able to look at the three 1178 01:13:33,440 --> 01:13:37,020 equations and tell me, as I increase the value of x1, 1179 01:13:37,020 --> 01:13:40,510 where am I going to hit my limit? 1180 01:13:40,510 --> 01:13:42,567 x6, yes, absolutely correct. 1181 01:13:42,567 --> 01:13:44,150 So that's because of the minus 4 here. 1182 01:13:44,150 --> 01:13:45,520 This is a big multiplier. 1183 01:13:45,520 --> 01:13:49,720 I got a minus 1 here, a minus 2 here, and a minus 4 here. 1184 01:13:49,720 --> 01:13:53,922 And if we just look at 4, the magnitudes, 4 is bigger than 2 1185 01:13:53,922 --> 01:13:55,380 and is bigger than 1, so it's going 1186 01:13:55,380 --> 01:13:57,090 to be that third constraint. 1187 01:13:57,090 --> 01:14:02,310 So third constraint-- you can obviously 1188 01:14:02,310 --> 01:14:04,510 compute this numerically or mechanically. 1189 01:14:04,510 --> 01:14:10,320 It was just easier to do this in this example by eyeballing it. 1190 01:14:10,320 --> 01:14:13,490 And so third constraint is the tightest one. 1191 01:14:13,490 --> 01:14:16,780 And it limits how much we can increase x1. 1192 01:14:16,780 --> 01:14:18,710 So I'm going to do my second step 1193 01:14:18,710 --> 01:14:22,610 up there, which corresponds to rewriting 1194 01:14:22,610 --> 01:14:26,860 x1 as these other variables. 1195 01:14:26,860 --> 01:14:31,480 And now, I got x1 on the left hand side and x6 1196 01:14:31,480 --> 01:14:33,100 on the right hand side. 1197 01:14:33,100 --> 01:14:36,020 And now, it's just merely a matter of substitution. 1198 01:14:36,020 --> 01:14:40,590 Once I've done this, I'm just going to jam through and go in 1199 01:14:40,590 --> 01:14:58,130 and I'm going to rewrite the other equations with x6 1200 01:14:58,130 --> 01:14:59,955 on the right hand side. 1201 01:15:02,770 --> 01:15:09,530 And that is, I'm going to replace x1 1202 01:15:09,530 --> 01:15:12,785 with the eyeball equation. 1203 01:15:12,785 --> 01:15:15,665 And that's really a simple substitution. 1204 01:15:15,665 --> 01:15:17,040 So at this point, what's happened 1205 01:15:17,040 --> 01:15:21,170 is, because the third constraint representing x6 1206 01:15:21,170 --> 01:15:23,420 was the one that was chosen, what's happened 1207 01:15:23,420 --> 01:15:28,440 is that x1 and x6 are going to interchange roles. 1208 01:15:28,440 --> 01:15:31,810 x1 was non-basic, it's going to become basic. 1209 01:15:31,810 --> 01:15:34,680 x6 was basic and is going to become non-basic. 1210 01:15:34,680 --> 01:15:38,910 And that's basically the essence of the simplex algorithm. 1211 01:15:38,910 --> 01:15:41,850 The iteration and then the convergence and all of that 1212 01:15:41,850 --> 01:15:48,400 is, as I mentioned, going to require getting to a point 1213 01:15:48,400 --> 01:15:50,880 where the optimality is obvious, but we 1214 01:15:50,880 --> 01:15:53,420 won't be doing any proofs corresponding 1215 01:15:53,420 --> 01:15:57,900 to conversions for simplex or any other specific LP 1216 01:15:57,900 --> 01:16:01,130 techniques in this class. 1217 01:16:01,130 --> 01:16:02,540 Maybe some constraint techniques, 1218 01:16:02,540 --> 01:16:06,010 I take back what I said, but certainly not for simplex. 1219 01:16:06,010 --> 01:16:10,080 But I just want to give you a sense for the flow here. 1220 01:16:10,080 --> 01:16:14,840 And so let's just go through this last thing here in terms 1221 01:16:14,840 --> 01:16:18,680 of finishing off the pivot xi's. 1222 01:16:18,680 --> 01:16:25,240 And I want to show you what the equations look like. 1223 01:16:25,240 --> 01:16:30,670 And if you just keep doing that, at some point, you'll converge. 1224 01:16:33,680 --> 01:16:35,590 And so what happens here is you have z 1225 01:16:35,590 --> 01:16:41,150 equals 27 plus x2 divided by 4 plus x3 divided 1226 01:16:41,150 --> 01:16:45,890 by 2 minus 3x6 divided by 4. 1227 01:16:45,890 --> 01:16:49,290 x1 equals-- so there's a bunch of algebra 1228 01:16:49,290 --> 01:16:57,470 here that I'm obviously skipping over, but it's simple algebra. 1229 01:16:57,470 --> 01:17:02,950 And I have x4 over here, 21 minus 3x2 1230 01:17:02,950 --> 01:17:09,372 divided by 4 minus 5x3 divided by 2 plus x6 divided by 4, 1231 01:17:09,372 --> 01:17:10,580 and then one last thing here. 1232 01:17:23,970 --> 01:17:29,870 So that's my pivot step, that I've flipped x1 and x6. 1233 01:17:29,870 --> 01:17:32,730 So now you ask, what was the point? 1234 01:17:32,730 --> 01:17:34,150 What was the point of this? 1235 01:17:34,150 --> 01:17:39,120 Well, the point of this was that you actually increased 1236 01:17:39,120 --> 01:17:42,150 the objective function value and, in this particular case, 1237 01:17:42,150 --> 01:17:46,280 quite significantly, while maintaining correctness. 1238 01:17:46,280 --> 01:17:53,140 And so let me just make these observations and conclude here, 1239 01:17:53,140 --> 01:17:55,760 because then that gets us to the point where you've seen 1240 01:17:55,760 --> 01:17:58,130 the details of one pivot step. 1241 01:17:58,130 --> 01:18:02,840 And you can imagine applying it over and over to a convergence. 1242 01:18:02,840 --> 01:18:09,590 And let's just look at the original basic solution, 1243 01:18:09,590 --> 01:18:17,380 which as you recall was 0, 0, 0, 30, 24, and 36. 1244 01:18:17,380 --> 01:18:20,370 And this is simply x1 through x6. 1245 01:18:20,370 --> 01:18:23,100 This original basic solution suddenly 1246 01:18:23,100 --> 01:18:28,140 satisfies these equations-- equations II, 1247 01:18:28,140 --> 01:18:30,250 if you just put them in there. 1248 01:18:30,250 --> 01:18:35,800 And it makes sense that it will have the objective value of 0, 1249 01:18:35,800 --> 01:18:38,030 given equivalents, but you can verify 1250 01:18:38,030 --> 01:18:41,410 that by saying that you have 27-- 1251 01:18:41,410 --> 01:18:43,950 the original had the objective value of 0, 1252 01:18:43,950 --> 01:18:47,230 because all the x sub 1, x2, x3 were 0, 1253 01:18:47,230 --> 01:18:50,930 so that was an easy check in the set of equations corresponding 1254 01:18:50,930 --> 01:18:53,360 to the first set, which I've erased at this point. 1255 01:18:53,360 --> 01:18:56,980 But no matter. 1256 01:18:56,980 --> 01:19:03,140 And if you look at what I have here, I have 36 equals 0. 1257 01:19:03,140 --> 01:19:05,580 So the objective value here is 0. 1258 01:19:05,580 --> 01:19:07,860 It matches what you had before. 1259 01:19:07,860 --> 01:19:16,670 But the basic solution for II, I'm 1260 01:19:16,670 --> 01:19:19,360 going to set the non-basic values. 1261 01:19:19,360 --> 01:19:22,170 So what is a solution here? 1262 01:19:22,170 --> 01:19:27,350 The non-basic values are 0. 1263 01:19:27,350 --> 01:19:35,970 So the solution is going to be 9, because 9 is non-basic. 1264 01:19:35,970 --> 01:19:37,950 x1 is now non-basic. 1265 01:19:37,950 --> 01:19:40,610 x2 and x3 are still basic. 1266 01:19:40,610 --> 01:19:42,840 And then I have 21 and 6. 1267 01:19:42,840 --> 01:19:45,080 And x6 now has become basic. 1268 01:19:45,080 --> 01:19:50,040 So the way I get this solution is simply by plugging in 0's 1269 01:19:50,040 --> 01:19:52,000 on this aside. 1270 01:19:52,000 --> 01:19:55,040 And I get 9, 21, and 6, because I just plugged 1271 01:19:55,040 --> 01:19:56,910 in 0's on the right hand side. 1272 01:19:56,910 --> 01:19:59,080 So that's how I got a new solution. 1273 01:19:59,080 --> 01:20:03,400 And if you look at the objective value for this, 1274 01:20:03,400 --> 01:20:09,670 the objective value, you can look at this objective value 1275 01:20:09,670 --> 01:20:13,200 simply by looking at the original problem. 1276 01:20:13,200 --> 01:20:20,670 And the original problem had 3x1 plus x2 plus x3 1277 01:20:20,670 --> 01:20:22,477 as the objective value. 1278 01:20:22,477 --> 01:20:24,810 And so if you go off and you see, well, that you had 0's 1279 01:20:24,810 --> 01:20:29,590 for the other ones, but you have 3 times 9, 1280 01:20:29,590 --> 01:20:33,230 so we have an objective value of 27. 1281 01:20:33,230 --> 01:20:37,250 So this flip of our pivot basically 1282 01:20:37,250 --> 01:20:39,920 got you from an objective value of 0, 1283 01:20:39,920 --> 01:20:45,190 while maintaining correctness, to an objective value of 27. 1284 01:20:45,190 --> 01:20:49,440 And you can look at this in the notes or in CLRS, 1285 01:20:49,440 --> 01:20:52,040 but you have to do two more pivots corresponding 1286 01:20:52,040 --> 01:20:54,820 to two other variables-- the same grungy 1287 01:20:54,820 --> 01:20:59,100 stuff that I went through here, substitution after selection. 1288 01:20:59,100 --> 01:21:01,420 And the objective value is going to increase. 1289 01:21:01,420 --> 01:21:05,690 And you might ask, how do I know that I'm done? 1290 01:21:05,690 --> 01:21:07,190 And so that was the last thing here, 1291 01:21:07,190 --> 01:21:13,700 which is increase the value until it becomes obvious-- no, 1292 01:21:13,700 --> 01:21:18,330 this was pivoting, I'm sorry-- increase the value pivot 1293 01:21:18,330 --> 01:21:23,240 until it becomes obvious what the optimum is. 1294 01:21:23,240 --> 01:21:26,420 And what ends up happening is you end up 1295 01:21:26,420 --> 01:21:28,320 with an objective function. 1296 01:21:28,320 --> 01:21:31,210 In this case, the objective function mind you, 1297 01:21:31,210 --> 01:21:34,100 this thing over here is the objective function. 1298 01:21:34,100 --> 01:21:37,790 And notice that it has a negative coefficient 1299 01:21:37,790 --> 01:21:43,570 on the variable that was actually 1300 01:21:43,570 --> 01:21:46,185 the part of the first pivot. 1301 01:21:46,185 --> 01:21:48,850 So x1 and x6 were a part of the first pivot. 1302 01:21:48,850 --> 01:21:51,307 And this got a negative coefficient. 1303 01:21:51,307 --> 01:21:53,890 So what ends up happening here is you end up getting something 1304 01:21:53,890 --> 01:22:01,810 like 30 minus something minus something 1305 01:22:01,810 --> 01:22:06,930 minus something, where you have xi values over here. 1306 01:22:06,930 --> 01:22:10,750 And when you set these to be 0, that's the best you can do, 1307 01:22:10,750 --> 01:22:13,870 because these are all negative quantities. 1308 01:22:13,870 --> 01:22:16,530 So I'll just leave you with that. 1309 01:22:16,530 --> 01:22:19,270 Hopefully, you understood how we do the pivoting-- 1310 01:22:19,270 --> 01:22:21,120 sub through it three times, and then 1311 01:22:21,120 --> 01:22:24,350 you get that objective function, and the optimum value is 30. 1312 01:22:24,350 --> 01:22:26,115 See you next time.