1 00:00:14,471 --> 00:00:18,070 GUEST SPEAKER: That says choices for your game project. 2 00:00:18,070 --> 00:00:22,552 These are the six choices that I want to make available to you. 3 00:00:22,552 --> 00:00:27,550 The next one is perhaps the most challenging one intellectually. 4 00:00:27,550 --> 00:00:29,135 It's about new funding mechanisms 5 00:00:29,135 --> 00:00:31,650 for disaster preparedness. 6 00:00:31,650 --> 00:00:33,180 I will explain this. 7 00:00:33,180 --> 00:00:37,076 We call it forecast-based financing, or FBF, 8 00:00:37,076 --> 00:00:42,364 and it's the one that has the most potential to transform 9 00:00:42,364 --> 00:00:45,500 the humanitarian world. 10 00:00:45,500 --> 00:00:48,640 All right, moving to the next one. 11 00:00:48,640 --> 00:00:52,095 The next one is the most complex. 12 00:00:56,010 --> 00:01:01,660 Do you see the drawing of the time bomb with the fuse? 13 00:01:01,660 --> 00:01:03,490 OK. 14 00:01:03,490 --> 00:01:07,380 So allow me to give a little bit of context. 15 00:01:07,380 --> 00:01:09,700 We did a little bit of this conversation 16 00:01:09,700 --> 00:01:11,880 when I showed up a few weeks ago, 17 00:01:11,880 --> 00:01:14,382 but I imagine you all forgot by now. 18 00:01:14,382 --> 00:01:20,230 The humanitarian sector has two main funding mechanisms. 19 00:01:20,230 --> 00:01:23,241 Imagine a pot of money. 20 00:01:23,241 --> 00:01:27,330 Even if there's almost no money there, if the pot exists, 21 00:01:27,330 --> 00:01:31,820 money can go in to then be taken out to spend. 22 00:01:31,820 --> 00:01:33,770 One of the pots of money that exists, 23 00:01:33,770 --> 00:01:35,640 one of the funding mechanisms that exists, 24 00:01:35,640 --> 00:01:40,820 is for disaster response, meaning hurricane or flood 25 00:01:40,820 --> 00:01:41,930 or volcanic eruption. 26 00:01:41,930 --> 00:01:46,030 Something has already happened, is happening, 27 00:01:46,030 --> 00:01:48,300 and is killing people. 28 00:01:48,300 --> 00:01:50,930 People are dying, but if you did something, 29 00:01:50,930 --> 00:01:53,840 you could save lives, for example, get on a boat 30 00:01:53,840 --> 00:01:56,990 and rescue people who are stranded on a roof. 31 00:01:56,990 --> 00:02:00,045 Or people are sick, go give them medicine. 32 00:02:00,045 --> 00:02:03,230 This is disaster response, and if you spend money, 33 00:02:03,230 --> 00:02:07,820 you can do very good things when the funding mechanism exists. 34 00:02:07,820 --> 00:02:10,986 There's another pot of money which is for a normal day. 35 00:02:10,986 --> 00:02:12,220 The Red Cross has vehicles. 36 00:02:12,220 --> 00:02:13,720 The vehicles need maintenance. 37 00:02:13,720 --> 00:02:15,270 They need to change tires. 38 00:02:15,270 --> 00:02:17,580 You need to buy a new computer. 39 00:02:17,580 --> 00:02:20,680 You need to pay rent and electricity bills. 40 00:02:20,680 --> 00:02:22,690 You need to train staff. 41 00:02:22,690 --> 00:02:26,233 So on a normal day, you have a budget for, you know, 42 00:02:26,233 --> 00:02:29,060 either maintenance or development of new projects 43 00:02:29,060 --> 00:02:30,740 or writing proposals and so on. 44 00:02:30,740 --> 00:02:32,920 And that pot of money exists. 45 00:02:32,920 --> 00:02:35,600 We call it the annual appeal. 46 00:02:35,600 --> 00:02:39,750 Every year we appeal for help, and people 47 00:02:39,750 --> 00:02:43,240 or donors or governments give us some money. 48 00:02:43,240 --> 00:02:47,530 What does not exist is what the drawing, the cartoon 49 00:02:47,530 --> 00:02:52,160 depicts, which is when there's a signal from science 50 00:02:52,160 --> 00:02:57,470 that a disaster-- that something bad is likely to happen soon. 51 00:02:57,470 --> 00:03:02,125 We don't have money to fund action before the disaster, 52 00:03:02,125 --> 00:03:04,870 after the forecast. 53 00:03:04,870 --> 00:03:08,030 So you can click to see the text. 54 00:03:08,030 --> 00:03:11,010 The context is that there's that missing money pot. 55 00:03:11,010 --> 00:03:12,737 We have my for after a disaster, we 56 00:03:12,737 --> 00:03:14,840 have money for the normal day, but we 57 00:03:14,840 --> 00:03:18,250 don't have money for before the disaster after the forecast. 58 00:03:18,250 --> 00:03:22,189 The issue is that if you spend money in that time window 59 00:03:22,189 --> 00:03:24,105 before the disaster and after the science says 60 00:03:24,105 --> 00:03:27,880 a disaster is likely, you can do very good things 61 00:03:27,880 --> 00:03:32,170 for very cheap and relatively simply and much more reliably. 62 00:03:32,170 --> 00:03:34,790 It's not only more expensive, but more difficult, 63 00:03:34,790 --> 00:03:38,270 to save a life when the storm waters are 64 00:03:38,270 --> 00:03:41,930 flashing through a city and people are about to drown. 65 00:03:41,930 --> 00:03:43,820 The boat is unsafe. 66 00:03:43,820 --> 00:03:46,520 Whereas if you do it the day before, 67 00:03:46,520 --> 00:03:49,050 when the water has come down from the sky, 68 00:03:49,050 --> 00:03:52,100 but it's still in the river and not in the city, 69 00:03:52,100 --> 00:03:54,930 people can take action by just walking away 70 00:03:54,930 --> 00:03:57,752 on their own means, with their valuables, 71 00:03:57,752 --> 00:04:01,200 and not just depending on someone to show up with a boat. 72 00:04:01,200 --> 00:04:03,720 So we came up with a financial mechanism. 73 00:04:03,720 --> 00:04:05,670 We call it forecast-based financing 74 00:04:05,670 --> 00:04:09,840 for disaster preparedness, which is [INAUDIBLE] 75 00:04:09,840 --> 00:04:11,090 in our particular method. 76 00:04:11,090 --> 00:04:13,610 There's a pot of money, so like a bank account. 77 00:04:13,610 --> 00:04:17,500 You need to have a threshold of disaster risk 78 00:04:17,500 --> 00:04:19,190 that needs to be exceeded. 79 00:04:19,190 --> 00:04:20,949 So it can't be two drops of rain. 80 00:04:20,949 --> 00:04:22,239 It has to be a lot of rain. 81 00:04:22,239 --> 00:04:23,280 It cannot be some breeze. 82 00:04:23,280 --> 00:04:26,100 It has to be a hurricane of Category 1 or more. 83 00:04:26,100 --> 00:04:28,735 But once the threshold is established and reached 84 00:04:28,735 --> 00:04:31,580 by the forecast, then there's some standard operating 85 00:04:31,580 --> 00:04:35,270 procedures, some actions that people have to take, 86 00:04:35,270 --> 00:04:37,860 spending money to save lives. 87 00:04:37,860 --> 00:04:40,630 And that is something that if you could come up 88 00:04:40,630 --> 00:04:46,940 with a game that captures that, this is the one that if we help 89 00:04:46,940 --> 00:04:49,280 communicate forecast-based financing-- 90 00:04:49,280 --> 00:04:52,850 I said to the public, because it can work for a broad audience 91 00:04:52,850 --> 00:04:53,980 digitally. 92 00:04:53,980 --> 00:04:57,020 Ideally it would reach those who can contribute either money 93 00:04:57,020 --> 00:05:00,440 or decisions, muscle power. 94 00:05:00,440 --> 00:05:02,030 The chances of embrace for this one 95 00:05:02,030 --> 00:05:04,800 are not as high as all the previous ones, 96 00:05:04,800 --> 00:05:08,570 but if it were to happen, it would have the highest impact. 97 00:05:08,570 --> 00:05:10,575 Because right now there is billions 98 00:05:10,575 --> 00:05:13,930 of dollars being spent in the two extremes, 99 00:05:13,930 --> 00:05:16,480 and if a small fraction were spent-- 100 00:05:16,480 --> 00:05:18,860 invested in that precious window of opportunity 101 00:05:18,860 --> 00:05:22,875 when you see the fuse getting close, that time of effort 102 00:05:22,875 --> 00:05:25,523 can really have a very, very high impact. 103 00:05:25,523 --> 00:05:27,023 There is a link to a journal article 104 00:05:27,023 --> 00:05:31,115 that should be very clear for MIT students. 105 00:05:31,115 --> 00:05:33,967 We also have some simpler readings on general. 106 00:05:33,967 --> 00:05:35,300 And I have been engaged with it. 107 00:05:35,300 --> 00:05:36,570 And I'm ready to help. 108 00:05:36,570 --> 00:05:39,176 This is, I insist, the one that is 109 00:05:39,176 --> 00:05:41,190 intellectually most challenging. 110 00:05:41,190 --> 00:05:43,177 But I think if you nail it, it will also 111 00:05:43,177 --> 00:05:45,295 be the most rewarding. 112 00:05:45,295 --> 00:05:45,795 Questions? 113 00:05:51,140 --> 00:05:52,140 PROFESSOR: So one thing. 114 00:05:52,140 --> 00:05:54,550 When we first started designing this class, 115 00:05:54,550 --> 00:05:56,735 this was actually the problem that we 116 00:05:56,735 --> 00:06:00,350 were most interested in, mainly because the other problems we 117 00:06:00,350 --> 00:06:02,380 just hadn't heard about since. 118 00:06:02,380 --> 00:06:06,920 The cholera was a recent proposal that was asked for, 119 00:06:06,920 --> 00:06:10,520 and Ebola, of course, very recent as well. 120 00:06:10,520 --> 00:06:13,060 So if you really enjoyed some of the challenges 121 00:06:13,060 --> 00:06:15,120 from projects two and three, and you really 122 00:06:15,120 --> 00:06:18,050 took on the planning aspects, the trade-offs 123 00:06:18,050 --> 00:06:20,640 aspects of those two games, if you're 124 00:06:20,640 --> 00:06:23,990 looking at kind of trying to communicate to another person 125 00:06:23,990 --> 00:06:25,717 how these kind of probabilities works, 126 00:06:25,717 --> 00:06:27,300 this is probably a really good project 127 00:06:27,300 --> 00:06:29,940 you might be interested in. 128 00:06:29,940 --> 00:06:33,950 And I should say also that working with a few MIT 129 00:06:33,950 --> 00:06:37,460 teams, including the Humanitarian Response Lab, 130 00:06:37,460 --> 00:06:40,105 including the Environmental Engineering, 131 00:06:40,105 --> 00:06:43,922 we have two generations of three students plus on professor 132 00:06:43,922 --> 00:06:47,820 go to Uganda to work on making this happen for real. 133 00:06:47,820 --> 00:06:51,230 We got German money to do it in Togo and in Uganda. 134 00:06:51,230 --> 00:06:52,850 So it's beginning to happen, but it's 135 00:06:52,850 --> 00:06:55,400 hard to explain because it's dry. 136 00:06:55,400 --> 00:06:56,710 It's like explaining insurance. 137 00:06:56,710 --> 00:06:59,370 People get bored before they get it. 138 00:06:59,370 --> 00:07:02,065 So maybe a game can help motivate. 139 00:07:08,250 --> 00:07:11,180 STUDENT 1: Our team is working on forecast-based financing. 140 00:07:11,180 --> 00:07:14,080 So because we're about planning for disasters, 141 00:07:14,080 --> 00:07:16,430 our game is pretty heavy on chance. 142 00:07:16,430 --> 00:07:19,990 But we want to teach the players to do is to use planning 143 00:07:19,990 --> 00:07:22,040 and forecasting in order to reduce 144 00:07:22,040 --> 00:07:25,510 the effects of that chance and sort of gain valuable skills 145 00:07:25,510 --> 00:07:26,760 in mitigating that. 146 00:07:32,040 --> 00:07:36,110 PROFESSOR: So basically if you have-- there's one, 147 00:07:36,110 --> 00:07:41,050 two, three, four, five testers, plus 148 00:07:41,050 --> 00:07:45,720 let's say each team send out two people to test other games. 149 00:07:45,720 --> 00:07:47,050 Remember to rotate. 150 00:07:47,050 --> 00:07:49,820 We're going to do this for about 20 minutes, 151 00:07:49,820 --> 00:07:52,570 as long as it takes, and then see where we are and do 152 00:07:52,570 --> 00:07:54,896 it again to get some just quick testing 153 00:07:54,896 --> 00:07:56,270 and make sure that the five of us 154 00:07:56,270 --> 00:07:58,765 get to play a good number of your games 155 00:07:58,765 --> 00:08:00,930 and give you some feedback on that. 156 00:08:00,930 --> 00:08:03,220 So remember if you're using digital-- 157 00:08:03,220 --> 00:08:05,780 if you're not-- If a computer's not being used for testing, 158 00:08:05,780 --> 00:08:08,780 close it, or make it look like it's not being used 159 00:08:08,780 --> 00:08:10,220 for testing by typing on it. 160 00:08:13,080 --> 00:08:13,580 All right. 161 00:08:13,580 --> 00:08:14,791 Let's get started. 162 00:08:14,791 --> 00:08:18,158 [INTERPOSING VOICES] 163 00:08:24,057 --> 00:08:24,640 STUDENT 1: Hi. 164 00:08:24,640 --> 00:08:26,560 We're forecast-based funding. 165 00:08:26,560 --> 00:08:29,640 The general idea behind our concept 166 00:08:29,640 --> 00:08:31,612 is that planning ahead for disasters 167 00:08:31,612 --> 00:08:33,559 is much better than trying to react to them. 168 00:08:33,559 --> 00:08:36,530 So if you can have operating procedures or ways of planning 169 00:08:36,530 --> 00:08:39,049 for them and money allocated for that, 170 00:08:39,049 --> 00:08:43,830 you can reduce loss of lives in the event of disasters. 171 00:08:43,830 --> 00:08:45,900 We are actually between two prototypes 172 00:08:45,900 --> 00:08:49,300 right now that we're testing to try and get at the ideas. 173 00:08:49,300 --> 00:08:53,810 The first one is a sort of higher-level city-based 174 00:08:53,810 --> 00:08:57,700 simulation of cities at risk of disaster, 175 00:08:57,700 --> 00:08:59,450 which you then have to fortify by training 176 00:08:59,450 --> 00:09:03,530 volunteers or preparing for upcoming disasters in order 177 00:09:03,530 --> 00:09:06,231 to prevent too much damage from happening to them. 178 00:09:06,231 --> 00:09:08,355 One of the problems that we're seeing with the game 179 00:09:08,355 --> 00:09:11,520 is that it's kind of abstract and not as interactive 180 00:09:11,520 --> 00:09:13,730 for players to connect with, but they 181 00:09:13,730 --> 00:09:17,427 are getting a good understanding of the idea of planning ahead. 182 00:09:17,427 --> 00:09:19,510 STUDENT 2: And so to try and address those issues, 183 00:09:19,510 --> 00:09:21,140 we have a second prototype right now, 184 00:09:21,140 --> 00:09:23,770 which is about trying to actually, like, 185 00:09:23,770 --> 00:09:26,550 rescue volunteers in a flooding-- rescue 186 00:09:26,550 --> 00:09:28,290 people in a flooding city. 187 00:09:28,290 --> 00:09:31,030 And so that hits the other end of the scale, where 188 00:09:31,030 --> 00:09:35,815 the player is told ahead of time this disaster was planned for, 189 00:09:35,815 --> 00:09:38,040 versus this disaster was not planned for, 190 00:09:38,040 --> 00:09:41,472 and the appropriate effects for each. 191 00:09:41,472 --> 00:09:42,930 And then they have to rescue people 192 00:09:42,930 --> 00:09:45,040 under those two different conditions, 193 00:09:45,040 --> 00:09:47,100 and then they get to directly compare 194 00:09:47,100 --> 00:09:51,230 what the experience is for, like, 195 00:09:51,230 --> 00:09:54,319 acting in one circumstance versus the other. 196 00:09:54,319 --> 00:09:55,860 STUDENT 1: Over the next couple days, 197 00:09:55,860 --> 00:09:59,136 we'll be looking at what we learned from both of them 198 00:09:59,136 --> 00:10:01,010 and trying to either combine them or pull out 199 00:10:01,010 --> 00:10:05,739 the parts that we thought were really useful to make one game. 200 00:10:05,739 --> 00:10:07,780 PROFESSOR: I can tell you've worked hard on this. 201 00:10:07,780 --> 00:10:09,250 This one's a hard one. 202 00:10:09,250 --> 00:10:10,230 STUDENT 1: Yeah. 203 00:10:10,230 --> 00:10:13,067 PROFESSOR: Did we assign you a target audience for this one? 204 00:10:13,067 --> 00:10:15,150 STUDENT 1: I think it was stated a couple of times 205 00:10:15,150 --> 00:10:17,460 that we should be looking at people like policymakers 206 00:10:17,460 --> 00:10:21,460 or donors in the sense of people who would be allocating funds 207 00:10:21,460 --> 00:10:25,100 from governments or nonprofits, things like that, 208 00:10:25,100 --> 00:10:28,196 basically to make it clear that this type of planning 209 00:10:28,196 --> 00:10:32,057 is a good idea. 210 00:10:32,057 --> 00:10:32,640 PROFESSOR: Oh. 211 00:10:32,640 --> 00:10:35,225 And what-- have you decided on a technology yet, 212 00:10:35,225 --> 00:10:37,215 or are you still pondering it? 213 00:10:37,215 --> 00:10:38,340 STUDENT 1: We're probably-- 214 00:10:38,340 --> 00:10:39,624 STUDENT 2: We're using Phaser. 215 00:10:39,624 --> 00:10:40,207 PROFESSOR: OK. 216 00:10:40,207 --> 00:10:43,616 [LAUGHTER] 217 00:10:45,064 --> 00:10:45,564 All right. 218 00:10:45,564 --> 00:10:46,538 Thank you. 219 00:10:46,538 --> 00:10:49,947 [APPLAUSE] 220 00:10:55,320 --> 00:10:57,610 STUDENT 1: Our game is called Hello Waves. 221 00:10:57,610 --> 00:10:59,270 We don't have the title screen in yet, 222 00:10:59,270 --> 00:11:04,150 but the idea is still forecast-based financing. 223 00:11:04,150 --> 00:11:07,710 So it's a little washed out on there, but if you can see, 224 00:11:07,710 --> 00:11:11,120 there are these five different sand castles, with workers 225 00:11:11,120 --> 00:11:14,982 associated with each, and up here in the corner, 226 00:11:14,982 --> 00:11:16,940 we have a forecast of what the water level will 227 00:11:16,940 --> 00:11:18,792 be like in a couple days. 228 00:11:18,792 --> 00:11:21,250 This is the water level right here, and throughout the game 229 00:11:21,250 --> 00:11:23,710 it'll rise and fall, and the idea 230 00:11:23,710 --> 00:11:28,340 is not to let your workers get drowned by the rising water. 231 00:11:28,340 --> 00:11:29,985 The other aspect of the game, though, 232 00:11:29,985 --> 00:11:32,750 is that you have a certain amount of supplies, 233 00:11:32,750 --> 00:11:36,040 and whenever you move someone from their castle 234 00:11:36,040 --> 00:11:38,460 in order to prevent them from getting drowned, 235 00:11:38,460 --> 00:11:40,950 they will consume supplies from that stock, 236 00:11:40,950 --> 00:11:42,620 and if you don't have enough supplies 237 00:11:42,620 --> 00:11:44,660 then they'll take damage. 238 00:11:44,660 --> 00:11:47,367 So as you can see right now, everybody's 239 00:11:47,367 --> 00:11:48,200 in their own castle. 240 00:11:48,200 --> 00:11:51,010 If we click Next Turn, the water level will change, 241 00:11:51,010 --> 00:11:53,330 and we'll see that the forecast will update. 242 00:11:53,330 --> 00:11:56,800 We have a range of values on this forecast, 243 00:11:56,800 --> 00:11:58,932 a high prediction and a low prediction, 244 00:11:58,932 --> 00:12:00,640 and if you mouse over any of the castles, 245 00:12:00,640 --> 00:12:02,690 you'll see a red line appear on the forecast, 246 00:12:02,690 --> 00:12:07,430 just to help you orient what heights are on the forecast, 247 00:12:07,430 --> 00:12:09,080 or if you're looking at the forecast, 248 00:12:09,080 --> 00:12:11,040 what it is in the real world. 249 00:12:11,040 --> 00:12:14,530 So you can see the water move over these couple days. 250 00:12:14,530 --> 00:12:16,112 Right now the forecast is pretty low. 251 00:12:16,112 --> 00:12:17,820 If we see that it's kind of getting close 252 00:12:17,820 --> 00:12:20,278 and we're worried about someone, we can click and drag them 253 00:12:20,278 --> 00:12:25,190 to another castle, and on the next turn they'll move over. 254 00:12:25,190 --> 00:12:26,810 You can also click people to toggle 255 00:12:26,810 --> 00:12:28,809 between building and gathering. 256 00:12:28,809 --> 00:12:30,350 It's very hard to see the status text 257 00:12:30,350 --> 00:12:31,558 right now because it's small. 258 00:12:31,558 --> 00:12:33,940 We threw that in quickly for right now. 259 00:12:33,940 --> 00:12:36,120 But he is set to building and these three 260 00:12:36,120 --> 00:12:37,350 are set to gathering. 261 00:12:37,350 --> 00:12:39,370 So we'll see that the each turn these three will 262 00:12:39,370 --> 00:12:41,680 gather one supply each. 263 00:12:41,680 --> 00:12:43,620 This teddy bear will consume one. 264 00:12:43,620 --> 00:12:45,500 So we'll see supplies go up by two 265 00:12:45,500 --> 00:12:49,450 and victory progress will go up by one. 266 00:12:49,450 --> 00:12:51,400 What we have planned for the rest of the game 267 00:12:51,400 --> 00:12:53,310 is we've made a lot of progress in terms 268 00:12:53,310 --> 00:12:56,620 of the intuitiveness of our game once you understand it. 269 00:12:56,620 --> 00:12:59,200 But right now, there's a lot in the game that isn't 270 00:12:59,200 --> 00:13:00,950 explained from the get go. 271 00:13:00,950 --> 00:13:03,590 And so if I give that spiel to anyone, 272 00:13:03,590 --> 00:13:07,210 they can play the game fine, and they do pretty well with it, 273 00:13:07,210 --> 00:13:10,100 but you just open up the game you're lost. 274 00:13:10,100 --> 00:13:12,470 So a lot of our work is going to be on the UI, 275 00:13:12,470 --> 00:13:14,460 of making things more self-evident 276 00:13:14,460 --> 00:13:17,420 and making sure that people can understand what's going on 277 00:13:17,420 --> 00:13:19,422 and what their actions will do without me having 278 00:13:19,422 --> 00:13:20,825 to stand there and tell them. 279 00:13:24,919 --> 00:13:25,960 PROFESSOR: Any questions? 280 00:13:29,356 --> 00:13:29,856 All right. 281 00:13:29,856 --> 00:13:32,291 So I'm going to ask you again, Tom, what is your biggest 282 00:13:32,291 --> 00:13:34,260 risk going forward? 283 00:13:34,260 --> 00:13:35,460 STUDENT 1: So-- 284 00:13:35,460 --> 00:13:37,110 STUDENT 2: I mean, I think in terms 285 00:13:37,110 --> 00:13:39,880 of our technical implementation, we've 286 00:13:39,880 --> 00:13:42,090 already got good playtests of our work done, 287 00:13:42,090 --> 00:13:44,610 and we have all of our major graphics in, 288 00:13:44,610 --> 00:13:45,610 and things like that. 289 00:13:45,610 --> 00:13:47,830 So I think our major risk going ahead 290 00:13:47,830 --> 00:13:51,300 is not being able to explain the game properly, 291 00:13:51,300 --> 00:13:54,000 and so even if we have a finished product, 292 00:13:54,000 --> 00:13:56,030 it may still not be playable if we 293 00:13:56,030 --> 00:13:57,989 don't write good instructions. 294 00:13:57,989 --> 00:13:59,030 GUEST SPEAKER: All right. 295 00:13:59,030 --> 00:14:05,350 So on that front, think of incremental additional 296 00:14:05,350 --> 00:14:05,850 features. 297 00:14:05,850 --> 00:14:08,620 So the game begins with a super-simple, even maybe 298 00:14:08,620 --> 00:14:11,150 too boring choice, but then a new choice arrives. 299 00:14:11,150 --> 00:14:15,130 So the learning by playing can be staggered 300 00:14:15,130 --> 00:14:16,790 in a way that is intuitive. 301 00:14:16,790 --> 00:14:17,290 Good. 302 00:14:17,290 --> 00:14:21,192 Good luck to you, and we'll talk more about choices later, 303 00:14:21,192 --> 00:14:24,815 but-- I should say that this high versus low prediction 304 00:14:24,815 --> 00:14:27,275 is something that I wish scientists did like you're 305 00:14:27,275 --> 00:14:29,656 doing, because most people say one value 306 00:14:29,656 --> 00:14:31,808 and then it's not that value, it's up here, 307 00:14:31,808 --> 00:14:33,753 and then everyone doesn't believe forecasts. 308 00:14:33,753 --> 00:14:34,253 Good. 309 00:14:34,253 --> 00:14:34,753 Thank you. 310 00:14:34,753 --> 00:14:35,720 STUDENT 1: Thank you. 311 00:14:35,720 --> 00:14:39,132 [APPLAUSE] 312 00:14:39,538 --> 00:14:40,121 PROFESSOR: OK. 313 00:14:40,121 --> 00:14:40,621 One down. 314 00:14:45,917 --> 00:14:46,500 STUDENT 1: OK. 315 00:14:46,500 --> 00:14:51,230 So it's a little light on there, but our game is Hello Waves. 316 00:14:51,230 --> 00:14:53,370 It's a game about forecast-based financing 317 00:14:53,370 --> 00:14:55,590 that we're developing for the Red Cross. 318 00:14:55,590 --> 00:14:57,310 The idea of forecast-based financing 319 00:14:57,310 --> 00:15:00,120 is using the idea of forecasts about the future in order 320 00:15:00,120 --> 00:15:04,010 to make decisions that are more effective than just reacting 321 00:15:04,010 --> 00:15:06,940 to disasters when they happen. 322 00:15:06,940 --> 00:15:12,700 So I'd like to show you a playthrough of our game. 323 00:15:12,700 --> 00:15:15,590 So, as I said, Hello Waves. 324 00:15:15,590 --> 00:15:18,060 Here are the instructions. 325 00:15:18,060 --> 00:15:20,560 We would have done a tutorial if we had enough time, 326 00:15:20,560 --> 00:15:23,190 but this gets the idea across. 327 00:15:23,190 --> 00:15:26,950 And designing a good tutorial that actually teaches 328 00:15:26,950 --> 00:15:30,840 the player well is kind of hard to do in a logical flow, 329 00:15:30,840 --> 00:15:33,790 so we have these instructions that explain how to play, 330 00:15:33,790 --> 00:15:37,300 what the point of the game is, and a couple of tips 331 00:15:37,300 --> 00:15:40,280 about how the game works. 332 00:15:40,280 --> 00:15:42,290 So if we go into the Play screen, 333 00:15:42,290 --> 00:15:45,380 you can see something similar to what we showed you 334 00:15:45,380 --> 00:15:48,870 a couple weeks ago, where you have all these different toys 335 00:15:48,870 --> 00:15:52,860 at their castles, and as you go through the days 336 00:15:52,860 --> 00:15:55,070 you'll see a forecast of what the water level is 337 00:15:55,070 --> 00:15:57,140 going to be at over time. 338 00:15:57,140 --> 00:15:59,830 And so you can see that right now they're gathering candy. 339 00:15:59,830 --> 00:16:01,720 The water level will change. 340 00:16:01,720 --> 00:16:04,470 This character is underwater, and so he takes damage. 341 00:16:04,470 --> 00:16:06,260 And so what you actually want to do 342 00:16:06,260 --> 00:16:09,380 is you want to move toys out of the water 343 00:16:09,380 --> 00:16:11,050 so they don't get damaged. 344 00:16:11,050 --> 00:16:13,580 However, they can only move one castle over at a time. 345 00:16:13,580 --> 00:16:15,775 So because we didn't think ahead well enough, 346 00:16:15,775 --> 00:16:17,400 we'll see that this truck will actually 347 00:16:17,400 --> 00:16:21,259 take damage on the next turn, because he's underwater. 348 00:16:21,259 --> 00:16:22,800 Actually, he ended up not underwater. 349 00:16:22,800 --> 00:16:24,120 We got lucky there. 350 00:16:24,120 --> 00:16:27,470 But theoretically he would have. 351 00:16:27,470 --> 00:16:30,020 And so as you go through it, the end goal of the game 352 00:16:30,020 --> 00:16:33,250 is to build a castle. 353 00:16:33,250 --> 00:16:34,850 And so every toy when they're home 354 00:16:34,850 --> 00:16:36,640 can either choose between gathering candy 355 00:16:36,640 --> 00:16:40,150 to feed evacuated toys or building the castle 356 00:16:40,150 --> 00:16:42,040 to reach your end goal of the game. 357 00:16:42,040 --> 00:16:43,716 And then the idea of the forecast 358 00:16:43,716 --> 00:16:45,730 is that you want to use the forecast in order 359 00:16:45,730 --> 00:16:47,730 to know how much candy you're going to need over 360 00:16:47,730 --> 00:16:51,180 the next couple days, and to use it to know when you're going 361 00:16:51,180 --> 00:16:54,740 to need to evacuate various toys from their variously-heighted 362 00:16:54,740 --> 00:16:55,240 homes. 363 00:16:58,000 --> 00:17:01,065 So back to the presentation. 364 00:17:01,065 --> 00:17:04,079 Full screen. 365 00:17:04,079 --> 00:17:07,550 So we had a couple of challenges in the design process. 366 00:17:07,550 --> 00:17:10,170 The first-- and the major one-- is really 367 00:17:10,170 --> 00:17:13,990 that we were trying to teach about forecast-based financing, 368 00:17:13,990 --> 00:17:16,869 which was a bit of an abstract topic. 369 00:17:16,869 --> 00:17:19,520 It's a little different than just thinking long term, 370 00:17:19,520 --> 00:17:22,220 because you have to use the idea of there's some information we 371 00:17:22,220 --> 00:17:23,636 know about the future that we want 372 00:17:23,636 --> 00:17:25,200 to use to make optimal decisions, 373 00:17:25,200 --> 00:17:29,070 or at least decisions based on some idea of the risk that's 374 00:17:29,070 --> 00:17:30,490 out there. 375 00:17:30,490 --> 00:17:32,504 But we also wanted to avoid things 376 00:17:32,504 --> 00:17:33,920 like just pushing a button to win, 377 00:17:33,920 --> 00:17:35,460 where you have all the information 378 00:17:35,460 --> 00:17:37,260 you ever need, and there's just one option 379 00:17:37,260 --> 00:17:39,360 that you know you're going to pick every time, 380 00:17:39,360 --> 00:17:41,879 and the game has no thought whatsoever. 381 00:17:41,879 --> 00:17:43,420 And we also needed to think about how 382 00:17:43,420 --> 00:17:45,770 we were going to communicate the forecast to the player 383 00:17:45,770 --> 00:17:49,980 so that they could then use that to make decisions. 384 00:17:49,980 --> 00:17:53,010 One of the problems that we also ran into related to this 385 00:17:53,010 --> 00:17:54,670 was that we focused a lot on the idea 386 00:17:54,670 --> 00:17:57,250 of forecast-based financing as the topic 387 00:17:57,250 --> 00:18:00,350 and then tried to build a game built on top of that topic 388 00:18:00,350 --> 00:18:03,210 instead of building a game that used forecast-based financing. 389 00:18:03,210 --> 00:18:05,590 So that held us back a lot in the beginning 390 00:18:05,590 --> 00:18:07,950 when we were trying to come up with ideas. 391 00:18:07,950 --> 00:18:10,220 We also had a really difficult initial target audience 392 00:18:10,220 --> 00:18:16,430 of policymakers, 50, 60-year-old government officials or people 393 00:18:16,430 --> 00:18:18,770 at NGOs who were going to be using forecasts 394 00:18:18,770 --> 00:18:20,390 to make decisions of some sort, and it 395 00:18:20,390 --> 00:18:22,848 was supposed to teach them about how they can use forecasts 396 00:18:22,848 --> 00:18:24,237 to make these decisions. 397 00:18:24,237 --> 00:18:26,070 Except this was a really difficult audience, 398 00:18:26,070 --> 00:18:28,040 because they don't generally play games 399 00:18:28,040 --> 00:18:30,404 and they don't have a lot of time 400 00:18:30,404 --> 00:18:31,820 to learn about this kind of thing, 401 00:18:31,820 --> 00:18:33,940 and so we couldn't really expect to get 402 00:18:33,940 --> 00:18:35,690 them to sit down for a long period of time 403 00:18:35,690 --> 00:18:38,650 and play around with our game. 404 00:18:38,650 --> 00:18:40,342 In order to deal with these problems, 405 00:18:40,342 --> 00:18:41,800 we came up with a couple solutions. 406 00:18:41,800 --> 00:18:44,580 The first thing was that we had no idea what kind of game 407 00:18:44,580 --> 00:18:47,160 would work or would make sense or anything like that. 408 00:18:47,160 --> 00:18:49,990 So what we did is we just broke our team up 409 00:18:49,990 --> 00:18:52,390 into multiple groups and came up with a bunch 410 00:18:52,390 --> 00:18:54,160 of different prototypes. 411 00:18:54,160 --> 00:18:55,990 On paper we had two different prototypes, 412 00:18:55,990 --> 00:18:58,040 one that focused on the idea of managing 413 00:18:58,040 --> 00:19:00,010 a city and its resources and its response 414 00:19:00,010 --> 00:19:02,640 to disasters, versus focusing on individual people 415 00:19:02,640 --> 00:19:04,640 and how you're going to move them around to keep 416 00:19:04,640 --> 00:19:06,340 them safe from the disasters. 417 00:19:06,340 --> 00:19:08,300 And then we went into a whole bunch 418 00:19:08,300 --> 00:19:09,820 of different digital prototypes, one 419 00:19:09,820 --> 00:19:12,540 that was a text-based game about managing a city. 420 00:19:12,540 --> 00:19:17,010 We had one that looked like this, where 421 00:19:17,010 --> 00:19:18,550 you had two different cities and you 422 00:19:18,550 --> 00:19:20,590 were specifying what workers were going to do 423 00:19:20,590 --> 00:19:23,880 or when they would leave the city in order to stay safe. 424 00:19:23,880 --> 00:19:25,630 And then we would take the different ideas 425 00:19:25,630 --> 00:19:27,260 that we were learning from both of these prototypes, 426 00:19:27,260 --> 00:19:29,270 combine them together, take things out, 427 00:19:29,270 --> 00:19:35,150 and our final prototype actually uses ideas from all of these. 428 00:19:35,150 --> 00:19:38,150 Another solution that we kind of got lucky with 429 00:19:38,150 --> 00:19:41,029 is when Pablo came to play our game, 430 00:19:41,029 --> 00:19:42,570 he told us that we actually shouldn't 431 00:19:42,570 --> 00:19:44,819 focus on the policymakers, because he wasn't confident 432 00:19:44,819 --> 00:19:47,720 that he could actually get them to play the game. 433 00:19:47,720 --> 00:19:50,430 And so we switched our target to being grade schoolers, which 434 00:19:50,430 --> 00:19:53,250 is why you saw the sort of cutesy art 435 00:19:53,250 --> 00:19:55,310 with the beach and the toys. 436 00:19:55,310 --> 00:19:57,700 This actually made it a lot easier for us, 437 00:19:57,700 --> 00:20:00,039 because we could target people who probably 438 00:20:00,039 --> 00:20:01,830 had some experience with games, or at least 439 00:20:01,830 --> 00:20:05,790 wanted to do something fun and would be curious to learn 440 00:20:05,790 --> 00:20:07,475 about our topic. 441 00:20:10,530 --> 00:20:15,380 STUDENT 2: So another for our development process-- 442 00:20:15,380 --> 00:20:16,990 Or I can just speak here, right? 443 00:20:16,990 --> 00:20:17,656 PROFESSOR: Yeah. 444 00:20:17,656 --> 00:20:19,568 Just speak right there. 445 00:20:19,568 --> 00:20:22,490 STUDENT 2: So another big issue-- 446 00:20:22,490 --> 00:20:24,520 another set of challenges that we ran into 447 00:20:24,520 --> 00:20:26,780 was through our development process. 448 00:20:26,780 --> 00:20:31,145 And so we had initially issues with communication 449 00:20:31,145 --> 00:20:32,390 and facilitation. 450 00:20:32,390 --> 00:20:36,780 Our team had a wide variety of experiences and backgrounds. 451 00:20:36,780 --> 00:20:40,600 Some were hardcore gamers, some mostly mobile gamers. 452 00:20:40,600 --> 00:20:42,430 And so there were initially a lot 453 00:20:42,430 --> 00:20:46,975 of disagreements on what level of game we wanted to create 454 00:20:46,975 --> 00:20:50,540 and what sort of game, casual versus hardcore, that we wanted 455 00:20:50,540 --> 00:20:54,000 to create, and so we needed to overcome challenges 456 00:20:54,000 --> 00:20:57,010 of facilitation and communication within our team. 457 00:20:57,010 --> 00:20:59,810 Another major challenge in our development process 458 00:20:59,810 --> 00:21:04,240 that we had to face came from our design issues, 459 00:21:04,240 --> 00:21:07,820 where for a very long time we had 460 00:21:07,820 --> 00:21:09,660 a vague vision of what to do. 461 00:21:09,660 --> 00:21:12,140 We didn't know what kind of game we wanted to make, 462 00:21:12,140 --> 00:21:15,440 and so we purposely tried to keep our game ideas vague 463 00:21:15,440 --> 00:21:17,490 as we built prototypes. 464 00:21:17,490 --> 00:21:21,110 But then we ran into issues where 465 00:21:21,110 --> 00:21:24,210 we would have solutions but, like, no consensus on which 466 00:21:24,210 --> 00:21:29,612 solution was best, and where we went for long periods of time 467 00:21:29,612 --> 00:21:31,320 without having a clear direction of where 468 00:21:31,320 --> 00:21:34,110 we wanted our final game to be. 469 00:21:34,110 --> 00:21:40,295 So our solutions for the challenges 470 00:21:40,295 --> 00:21:44,420 posed by the development were a team structure. 471 00:21:44,420 --> 00:21:48,640 And so we structured our team loosely 472 00:21:48,640 --> 00:21:50,950 into three subteams, a production subteam which 473 00:21:50,950 --> 00:21:56,500 would take care of production, like the deliverables 474 00:21:56,500 --> 00:21:59,640 and making sure that all the game 475 00:21:59,640 --> 00:22:02,950 ideas are being communicated properly, 476 00:22:02,950 --> 00:22:07,540 a technical team which worked primarily with the code 477 00:22:07,540 --> 00:22:09,250 and making sure the game got done, 478 00:22:09,250 --> 00:22:11,060 and then a user experience team which 479 00:22:11,060 --> 00:22:14,800 handled art, UI, and sound. 480 00:22:14,800 --> 00:22:20,320 And so we kept the responsibilities flexible. 481 00:22:20,320 --> 00:22:24,710 So as team members got busy over the semester 482 00:22:24,710 --> 00:22:29,170 or as changing conditions led to different people contributing, 483 00:22:29,170 --> 00:22:33,910 we kept-- responsibilities were able to easily flow 484 00:22:33,910 --> 00:22:37,610 between teams and team members. 485 00:22:37,610 --> 00:22:40,780 Additionally, another idea we started with in the beginning 486 00:22:40,780 --> 00:22:44,700 was the idea of subteam leaders, which were the two 487 00:22:44,700 --> 00:22:46,020 people marked with the Ls. 488 00:22:46,020 --> 00:22:48,360 But that was an idea we later abandoned 489 00:22:48,360 --> 00:22:51,480 in favor of just having a more flexible team structure 490 00:22:51,480 --> 00:22:54,760 or flat team structure. 491 00:22:54,760 --> 00:22:59,330 Another solution for helping our development process was the use 492 00:22:59,330 --> 00:23:03,360 of good code practices, and I cannot emphasize enough that 493 00:23:03,360 --> 00:23:05,780 this really helped speed up our development, 494 00:23:05,780 --> 00:23:09,110 because we didn't run into trouble with code. 495 00:23:09,110 --> 00:23:10,780 It was mostly with design. 496 00:23:10,780 --> 00:23:18,020 So we used Yeoman, which is a JavaScript module system, 497 00:23:18,020 --> 00:23:21,610 and basically it allowed our code to be interoperable. 498 00:23:21,610 --> 00:23:25,200 We could write one module separately from another module. 499 00:23:25,200 --> 00:23:31,380 So that solved a lot of issues with dependencies or people 500 00:23:31,380 --> 00:23:33,690 working in parallel, because it allowed people 501 00:23:33,690 --> 00:23:38,230 to work in parallel without overwriting each other's code. 502 00:23:38,230 --> 00:23:42,690 We also used good code practice with state machines and MVC, 503 00:23:42,690 --> 00:23:45,080 which is model-view-control, and so we 504 00:23:45,080 --> 00:23:48,720 had a very object-oriented code, very modular. 505 00:23:48,720 --> 00:23:52,980 And when we did need to change our code, rip it all out 506 00:23:52,980 --> 00:23:54,980 and put it back in, it actually didn't turn out 507 00:23:54,980 --> 00:23:57,580 to be too painful, because we just had to switch a couple 508 00:23:57,580 --> 00:23:58,320 objects around. 509 00:24:01,590 --> 00:24:08,130 And then one final solution that we used for our development 510 00:24:08,130 --> 00:24:11,320 processes were Slack and Scrum. 511 00:24:11,320 --> 00:24:15,460 So Slack is like a modernized IRC chat room, 512 00:24:15,460 --> 00:24:16,670 and it's very feature rich. 513 00:24:16,670 --> 00:24:20,320 It has a lot of integrations with GitHub and Google Drive 514 00:24:20,320 --> 00:24:22,530 and things like that. 515 00:24:22,530 --> 00:24:25,260 And so that sort of real time communication 516 00:24:25,260 --> 00:24:27,180 actually made it so that we didn't really 517 00:24:27,180 --> 00:24:29,979 have to meet outside of class too often. 518 00:24:29,979 --> 00:24:31,520 If someone was working, we would just 519 00:24:31,520 --> 00:24:34,770 email out saying I'm on the Slack, 520 00:24:34,770 --> 00:24:36,790 and then people could meet on the Slack, 521 00:24:36,790 --> 00:24:38,430 and it was full-featured enough-- 522 00:24:38,430 --> 00:24:41,490 like we could send attachments and things like that-- 523 00:24:41,490 --> 00:24:45,670 that most of our in person communication 524 00:24:45,670 --> 00:24:47,660 could be done in class. 525 00:24:47,660 --> 00:24:52,960 And in class we adopted a sort of, like, daily Scrum format, 526 00:24:52,960 --> 00:24:58,430 where we simply said what we had done since the previous class 527 00:24:58,430 --> 00:25:03,500 and what our goals were until next class. 528 00:25:03,500 --> 00:25:06,190 So in the end, though, we did have to cut some features. 529 00:25:09,070 --> 00:25:12,150 These features mainly were the idea 530 00:25:12,150 --> 00:25:15,890 of multiple-- again, a tutorial or multiple levels, 531 00:25:15,890 --> 00:25:18,700 simply because there would just be too much content that we 532 00:25:18,700 --> 00:25:21,010 would need to playtest in order to make sure 533 00:25:21,010 --> 00:25:23,520 that it was of consistent quality 534 00:25:23,520 --> 00:25:27,370 and got our message across. 535 00:25:27,370 --> 00:25:32,010 And also trying to add more individuality to the toys 536 00:25:32,010 --> 00:25:33,440 that you saw. 537 00:25:33,440 --> 00:25:36,040 They have different graphics, but that's as much 538 00:25:36,040 --> 00:25:38,055 as we could do given the time constraints. 539 00:25:40,830 --> 00:25:45,360 So kind of just bring it back. 540 00:25:45,360 --> 00:25:51,590 Our three worst decisions were first, 541 00:25:51,590 --> 00:25:54,280 we did end up spending a lot of time on code 542 00:25:54,280 --> 00:25:57,120 and assets that never got used. 543 00:25:57,120 --> 00:26:02,430 We maybe, like, used 10% to 20% of our final work 544 00:26:02,430 --> 00:26:04,170 in our final project. 545 00:26:04,170 --> 00:26:07,390 Actually, maybe that is a bit overkill. 546 00:26:07,390 --> 00:26:08,120 But OK. 547 00:26:08,120 --> 00:26:14,040 Maybe 30% of our final code and assets in our final project. 548 00:26:14,040 --> 00:26:17,860 This really was due to this second bad decision, 549 00:26:17,860 --> 00:26:20,750 that we kept the game and its direction 550 00:26:20,750 --> 00:26:22,470 too vague for too long. 551 00:26:22,470 --> 00:26:24,760 We always were holding out for, oh, maybe we'll 552 00:26:24,760 --> 00:26:26,870 be able to come up with a better game idea, 553 00:26:26,870 --> 00:26:29,550 or maybe we'll find some magic solution 554 00:26:29,550 --> 00:26:32,710 to how we can make forecast-based financing 555 00:26:32,710 --> 00:26:34,170 into a game. 556 00:26:34,170 --> 00:26:38,230 And because of this mindset, we spent probably 557 00:26:38,230 --> 00:26:42,390 the first half of the project, like, just staying too vague. 558 00:26:42,390 --> 00:26:45,410 And that hurt us in the end, because we spent so much time 559 00:26:45,410 --> 00:26:47,710 going in all these different directions. 560 00:26:47,710 --> 00:26:53,110 And really the decision that kind of 561 00:26:53,110 --> 00:26:54,530 captures those first two, though, 562 00:26:54,530 --> 00:26:59,270 is the fact that we tried to make a game on top 563 00:26:59,270 --> 00:27:00,710 of forecast-based financing. 564 00:27:00,710 --> 00:27:03,110 So we had forecast-based financing, and we were like, 565 00:27:03,110 --> 00:27:05,290 how can we skin this as a game? 566 00:27:05,290 --> 00:27:08,300 Whereas once we switched that mindset and thought, 567 00:27:08,300 --> 00:27:11,580 let's have a game, and how can we put forecast-based financing 568 00:27:11,580 --> 00:27:12,430 into it? 569 00:27:12,430 --> 00:27:16,640 I think that was the moment that we then came together as a team 570 00:27:16,640 --> 00:27:22,100 and really started making the final game that we wanted. 571 00:27:22,100 --> 00:27:23,310 So our best decisions-- 572 00:27:25,940 --> 00:27:28,300 STUDENT 1: So one of our best decisions 573 00:27:28,300 --> 00:27:31,016 was that we chose good tools at the beginning, which 574 00:27:31,016 --> 00:27:32,595 meant that as we went through all 575 00:27:32,595 --> 00:27:34,400 these different digital prototypes, 576 00:27:34,400 --> 00:27:37,574 we actually didn't have to completely rewrite our game. 577 00:27:37,574 --> 00:27:39,990 We could pull out the way that workers worked in one game. 578 00:27:39,990 --> 00:27:42,084 We could pull out the view that we were using in another game, 579 00:27:42,084 --> 00:27:44,340 and then we could just combine them together, 580 00:27:44,340 --> 00:27:46,842 and that allowed us to move quickly whenever we 581 00:27:46,842 --> 00:27:49,180 were changing our prototype. 582 00:27:49,180 --> 00:27:51,716 We also weren't afraid to trust each other, 583 00:27:51,716 --> 00:27:54,439 both in terms of what everyone was working on, 584 00:27:54,439 --> 00:27:56,730 but also in terms of the decisions that we were making. 585 00:27:56,730 --> 00:27:59,021 And so when we said that we had to throw something out, 586 00:27:59,021 --> 00:28:03,240 we all understood that it was for the good of the project. 587 00:28:03,240 --> 00:28:05,230 And we didn't have a lot of complaining 588 00:28:05,230 --> 00:28:07,330 or hurt feelings when something didn't get put in 589 00:28:07,330 --> 00:28:11,472 or when we decided to throw out certain assets or code. 590 00:28:11,472 --> 00:28:13,930 And when we got to the end, we had 591 00:28:13,930 --> 00:28:16,650 been through enough of this vagueness 592 00:28:16,650 --> 00:28:18,710 that we were all kind of frustrated with it, 593 00:28:18,710 --> 00:28:21,460 and we realized that we had an idea that we all liked 594 00:28:21,460 --> 00:28:23,710 and we really got on board with it and made it happen. 595 00:28:23,710 --> 00:28:25,500 Once we started working on our final idea, 596 00:28:25,500 --> 00:28:29,198 and we saw that it worked, basically every decision 597 00:28:29,198 --> 00:28:31,811 from that point on was how do we make this game better, 598 00:28:31,811 --> 00:28:33,560 and we were all on board with that vision. 599 00:28:37,900 --> 00:28:38,400 Thank you. 600 00:28:38,400 --> 00:28:39,852 Any questions? 601 00:28:39,852 --> 00:28:43,240 [APPLAUSE] 602 00:28:44,692 --> 00:28:47,596 PROFESSOR: Anyone in the audience have feedback for them 603 00:28:47,596 --> 00:28:49,306 first, before-- 604 00:28:49,306 --> 00:28:52,549 AUDIENCE: You can't see those sand castles very well. 605 00:28:52,549 --> 00:28:55,090 STUDENT 1: Yeah, we'll have to make all of our slides darker. 606 00:28:55,090 --> 00:28:56,596 PROFESSOR: Yellow and white. 607 00:28:56,596 --> 00:28:57,220 AUDIENCE: Yeah. 608 00:28:57,220 --> 00:29:00,320 Like, the yellow and white, on your early screens. 609 00:29:00,320 --> 00:29:02,494 I was doing this on your instruction screen. 610 00:29:02,494 --> 00:29:03,410 PROFESSOR: It's also-- 611 00:29:03,410 --> 00:29:07,365 AUDIENCE: I realize that's the game, not the presentation, 612 00:29:07,365 --> 00:29:08,299 but wow. 613 00:29:08,299 --> 00:29:10,590 PROFESSOR: It's also a little faded in the game itself. 614 00:29:10,590 --> 00:29:12,589 It could be the display resolution you're using. 615 00:29:12,589 --> 00:29:15,186 So we're all done with everything, 616 00:29:15,186 --> 00:29:16,310 try a different resolution. 617 00:29:16,310 --> 00:29:17,518 See if it makes a difference. 618 00:29:20,750 --> 00:29:22,960 AUDIENCE: The music in the game level was too high. 619 00:29:22,960 --> 00:29:25,370 It was overpowering your voice. 620 00:29:25,370 --> 00:29:27,880 So I think you were almost hollering just 621 00:29:27,880 --> 00:29:29,317 to be able to be heard. 622 00:29:29,317 --> 00:29:30,650 it doesn't need to be that loud. 623 00:29:30,650 --> 00:29:33,191 So you can just crank down the volume, either on the computer 624 00:29:33,191 --> 00:29:36,940 or on the controls over there. 625 00:29:36,940 --> 00:29:39,720 Something that I had a question for, you 626 00:29:39,720 --> 00:29:41,634 don't have to answer it today, but you 627 00:29:41,634 --> 00:29:43,550 might want to put it in your presentation was, 628 00:29:43,550 --> 00:29:48,600 how long was the design of what you eventually 629 00:29:48,600 --> 00:29:51,110 established on the table before you 630 00:29:51,110 --> 00:29:53,610 decided that this was the thing that you were going for? 631 00:29:53,610 --> 00:29:55,950 Because I'm assuming it was one of your vague-- 632 00:29:55,950 --> 00:29:58,440 it came up during your vague idea phase, 633 00:29:58,440 --> 00:30:02,200 and you're implied that you were in that mode for too long. 634 00:30:02,200 --> 00:30:05,220 But I don't know how long it was on the table, 635 00:30:05,220 --> 00:30:07,300 or was it only something you figured out 636 00:30:07,300 --> 00:30:09,782 at the end of the vague idea phase? 637 00:30:09,782 --> 00:30:11,240 Because otherwise you wouldn't have 638 00:30:11,240 --> 00:30:13,330 been able to switch to this idea if it was not 639 00:30:13,330 --> 00:30:14,663 on the table in the first place. 640 00:30:16,830 --> 00:30:21,010 And how did you decide this was it, that the sand castle 641 00:30:21,010 --> 00:30:22,420 game was going to be it? 642 00:30:22,420 --> 00:30:26,290 I understand-- I think you very clearly explained the benefits 643 00:30:26,290 --> 00:30:27,770 of deciding this was it. 644 00:30:27,770 --> 00:30:30,570 But how did you come to that conclusion? 645 00:30:30,570 --> 00:30:32,620 So that's stuff I wanted to hear more about. 646 00:30:32,620 --> 00:30:35,780 But you don't have to answer it now. 647 00:30:35,780 --> 00:30:36,875 AUDIENCE: Yeah. 648 00:30:36,875 --> 00:30:38,250 Just a little bit of specificity. 649 00:30:38,250 --> 00:30:41,036 You mentioned you spent too long on that design phase. 650 00:30:41,036 --> 00:30:42,160 I wonder how long that was. 651 00:30:44,989 --> 00:30:47,280 PROFESSOR: You said Pablo switched your target audience 652 00:30:47,280 --> 00:30:49,170 for you. 653 00:30:49,170 --> 00:30:52,065 Telling us whether that was because of the game that he saw 654 00:30:52,065 --> 00:30:55,090 or because of something else-- if you know that information, 655 00:30:55,090 --> 00:30:55,910 throw it in there. 656 00:30:55,910 --> 00:30:59,150 If you don't know that information, that's fine. 657 00:30:59,150 --> 00:31:01,339 Knowing why you dropped the team lead. 658 00:31:01,339 --> 00:31:03,880 You mentioned you dropped it, but you didn't exactly say why. 659 00:31:03,880 --> 00:31:05,400 Again, really quick. 660 00:31:05,400 --> 00:31:08,270 This is what-- we weren't getting blah out of it, 661 00:31:08,270 --> 00:31:11,080 or the flexibility was more, whatever. 662 00:31:11,080 --> 00:31:12,270 And then, yeah, to-- 663 00:31:12,270 --> 00:31:13,050 Oh. 664 00:31:13,050 --> 00:31:13,990 Defining your terms. 665 00:31:13,990 --> 00:31:16,550 You were saying things like casual versus hardcore. 666 00:31:16,550 --> 00:31:19,424 Give a little bit a definition of what you mean by that. 667 00:31:19,424 --> 00:31:21,090 It means different things for everybody. 668 00:31:21,090 --> 00:31:22,765 So what is your use of that. 669 00:31:22,765 --> 00:31:23,390 AUDIENCE: Yeah. 670 00:31:23,390 --> 00:31:25,970 I think he specifically said hardcore and mobile, 671 00:31:25,970 --> 00:31:28,140 but there are hardcore mobile players out there, 672 00:31:28,140 --> 00:31:28,849 so this is like-- 673 00:31:28,849 --> 00:31:30,473 PROFESSOR: And casual can be considered 674 00:31:30,473 --> 00:31:32,440 a version of hardcore, just in a different way. 675 00:31:32,440 --> 00:31:33,880 So just be really-- just be a little more 676 00:31:33,880 --> 00:31:35,630 clear about-- because I think you were talking 677 00:31:35,630 --> 00:31:37,671 about the target audience and the kind of players 678 00:31:37,671 --> 00:31:39,350 and the kind of games they might play. 679 00:31:39,350 --> 00:31:42,040 So just be a little bit more focused on what exactly 680 00:31:42,040 --> 00:31:45,507 you mean by that. 681 00:31:45,507 --> 00:31:46,343 That's it from me. 682 00:31:46,343 --> 00:31:46,968 AUDIENCE: Yeah. 683 00:31:46,968 --> 00:31:48,916 So in terms of technical observation, 684 00:31:48,916 --> 00:31:52,325 Yeoman is not really a module framework. 685 00:31:52,325 --> 00:31:55,247 It's just a system to generate the project 686 00:31:55,247 --> 00:31:58,169 and it puts different frameworks. 687 00:31:58,169 --> 00:32:00,130 Again, just check that. 688 00:32:00,130 --> 00:32:03,340 AUDIENCE: So that's a terminology issue, then. 689 00:32:03,340 --> 00:32:05,332 Clearly it worked out for your team. 690 00:32:05,332 --> 00:32:07,040 So we're not saying don't mention Yeoman. 691 00:32:07,040 --> 00:32:08,123 It's clearly a good thing. 692 00:32:08,123 --> 00:32:11,530 But just check your definition of what it is. 693 00:32:11,530 --> 00:32:13,580 Because it'll be more helpful for other people 694 00:32:13,580 --> 00:32:15,224 to understand what it is so that they 695 00:32:15,224 --> 00:32:17,640 can think about whether they want to use it in the future. 696 00:32:17,640 --> 00:32:19,910 PROFESSOR: Oh, and actually-- So your demo, 697 00:32:19,910 --> 00:32:22,150 you spent about three minutes doing it. 698 00:32:22,150 --> 00:32:24,690 We are going to have you have a player from the audience 699 00:32:24,690 --> 00:32:29,470 play your game live, without getting a lot of help. 700 00:32:29,470 --> 00:32:31,030 You can talk over it. 701 00:32:31,030 --> 00:32:33,115 You can, after a while, start helping them. 702 00:32:33,115 --> 00:32:34,490 We want to see them at least just 703 00:32:34,490 --> 00:32:36,080 to start playing on their own. 704 00:32:36,080 --> 00:32:37,980 Decide when you're going to put that demo in. 705 00:32:37,980 --> 00:32:39,970 You could actually combine them both together 706 00:32:39,970 --> 00:32:42,470 if you do it at the beginning or end. 707 00:32:42,470 --> 00:32:45,990 But if you do that, give the player a little bit of time 708 00:32:45,990 --> 00:32:47,080 before you start talking. 709 00:32:49,835 --> 00:32:50,960 AUDIENCE: That's all I got. 710 00:32:50,960 --> 00:32:51,680 PROFESSOR: Great. 711 00:32:51,680 --> 00:32:52,560 Thanks. 712 00:32:52,560 --> 00:32:55,850 [APPLAUSE] 713 00:32:55,850 --> 00:32:57,060 Snap, come on down. 714 00:33:02,030 --> 00:33:03,020 STUDENT 1: Hello. 715 00:33:03,020 --> 00:33:04,610 We are Hello Waves. 716 00:33:04,610 --> 00:33:07,620 We're a game about forecasting, specifically 717 00:33:07,620 --> 00:33:11,610 this idea called forecast-based financing, of using forecasts 718 00:33:11,610 --> 00:33:14,760 to make decisions about possible disasters in the future 719 00:33:14,760 --> 00:33:16,452 and how to prepare for them. 720 00:33:16,452 --> 00:33:18,910 We'd actually like to start with a playthrough of our game. 721 00:33:18,910 --> 00:33:23,924 So if anybody would like to be a volunteer to try it out. 722 00:33:23,924 --> 00:33:25,740 AUDIENCE: The guest from not our class. 723 00:33:25,740 --> 00:33:28,915 Would you please come down, Andrew? 724 00:33:28,915 --> 00:33:34,117 [LAUGHTER] 725 00:33:34,117 --> 00:33:35,700 PROFESSOR: Thank you for volunteering. 726 00:33:35,700 --> 00:33:39,074 [LAUGHTER] 727 00:33:39,074 --> 00:33:41,472 ANDREW: Hi, I'm Andrew. 728 00:33:41,472 --> 00:33:42,930 PROFESSOR: Oh, do you need a chair? 729 00:33:42,930 --> 00:33:43,930 There's one right there. 730 00:33:43,930 --> 00:33:44,454 All right. 731 00:33:44,454 --> 00:33:44,954 [? 732 00:33:44,954 --> 00:33:47,078 STUDENT 1: I'm not sure how to make it full screen. 733 00:33:47,078 --> 00:33:50,102 But anyways, this is our game. 734 00:33:50,102 --> 00:33:51,810 I'd recommend looking at the instructions 735 00:33:51,810 --> 00:33:53,101 first and reading through them. 736 00:33:57,640 --> 00:34:00,170 And we'll keep the description minimal 737 00:34:00,170 --> 00:34:03,770 as you read through, just to show the player's 738 00:34:03,770 --> 00:34:05,400 initial reaction to it. 739 00:34:05,400 --> 00:34:07,817 [MUSIC PLAYING] 740 00:34:07,817 --> 00:34:08,400 ANDREW: Music. 741 00:34:34,959 --> 00:34:35,750 STUDENT 1: So yeah. 742 00:34:35,750 --> 00:34:36,375 Start the game. 743 00:34:38,889 --> 00:34:41,900 So as you can see, our game, as I said before, 744 00:34:41,900 --> 00:34:46,220 you control some toys on a day at the beach. 745 00:34:53,000 --> 00:34:56,190 And so by dragging and dropping them between the castles, 746 00:34:56,190 --> 00:34:58,920 you'll see that their status is changed, and say 747 00:34:58,920 --> 00:35:01,010 that they want to move on their next turn 748 00:35:01,010 --> 00:35:02,567 or that they want to be collecting-- 749 00:35:02,567 --> 00:35:04,150 or that they're going to be collecting 750 00:35:04,150 --> 00:35:08,430 candy or anything like that. 751 00:35:08,430 --> 00:35:12,400 And the game is turn-based, so all of the actions 752 00:35:12,400 --> 00:35:13,915 will resolve on the next turn. 753 00:35:42,096 --> 00:35:45,398 ANDREW: Am I reading this right? 754 00:35:45,398 --> 00:35:46,064 STUDENT 1: Yeah. 755 00:35:46,064 --> 00:35:47,689 STUDENT 2: You can also access the Help 756 00:35:47,689 --> 00:35:49,536 by clicking in the bottom right corner. 757 00:36:06,920 --> 00:36:09,330 STUDENT 1: And so when you go to the next turn, 758 00:36:09,330 --> 00:36:14,580 you'll see all the toys move as specified by the status 759 00:36:14,580 --> 00:36:16,285 bubble above their heads. 760 00:36:16,285 --> 00:36:18,326 ANDREW: So go ahead to the next turn? [INAUDIBLE] 761 00:36:18,326 --> 00:36:19,754 and Next Turn. 762 00:36:22,002 --> 00:36:23,460 STUDENT 1: But unfortunately you'll 763 00:36:23,460 --> 00:36:28,070 find that when toys have been evacuated, they're unhappy 764 00:36:28,070 --> 00:36:30,510 and need candy to survive, so they'll all take damage. 765 00:36:30,510 --> 00:36:33,480 ANDREW: OK. 766 00:36:33,480 --> 00:36:36,450 I don't want to be in that [INAUDIBLE]. 767 00:36:36,450 --> 00:36:39,415 STUDENT 2: So you can try returning them to their homes. 768 00:36:39,415 --> 00:36:39,915 ANDREW: Ah. 769 00:36:56,250 --> 00:36:59,710 STUDENT 1: And so you'll see that on this turn instead now 770 00:36:59,710 --> 00:37:01,710 they have their statuses set to gathering candy, 771 00:37:01,710 --> 00:37:03,895 except for the dump truck, who's still evacuated. 772 00:37:12,290 --> 00:37:13,290 ANDREW: Gotcha. 773 00:37:13,290 --> 00:37:15,290 OK. 774 00:37:15,290 --> 00:37:16,790 I think he needs to be evacuated. 775 00:37:24,290 --> 00:37:27,090 STUDENT 1: Exactly. 776 00:37:27,090 --> 00:37:29,787 And so the idea is that as the player plays through it, 777 00:37:29,787 --> 00:37:32,120 they get better at understanding how to use the forecast 778 00:37:32,120 --> 00:37:35,087 to make decisions about the future, 779 00:37:35,087 --> 00:37:37,670 both in terms of how much candy that they need to have stocked 780 00:37:37,670 --> 00:37:40,520 up in order to weather out the rising tides, 781 00:37:40,520 --> 00:37:42,740 and also in terms of when they're 782 00:37:42,740 --> 00:37:46,967 going to need to move their workers out-- their toys out 783 00:37:46,967 --> 00:37:48,300 of the areas that are in danger. 784 00:37:48,300 --> 00:37:50,280 ANDREW: They're gonna be really affected. 785 00:38:24,419 --> 00:38:26,210 STUDENT 1: I think you have those two guys. 786 00:38:26,210 --> 00:38:28,400 ANDREW: Oh, right. 787 00:38:28,400 --> 00:38:30,310 So is there a reminder of where they started? 788 00:38:30,310 --> 00:38:30,976 STUDENT 1: Yeah. 789 00:38:30,976 --> 00:38:32,100 It's on the castle. 790 00:38:32,100 --> 00:38:33,720 It's actually blocked there right now 791 00:38:33,720 --> 00:38:36,570 by the toy in front of it. 792 00:38:36,570 --> 00:38:40,035 But there's a little shadow on there, an imprint. 793 00:38:45,720 --> 00:38:51,220 And because he ran out of candy and then-- Actually, sorry. 794 00:38:51,220 --> 00:38:53,630 Because he went to a place that was underwater, 795 00:38:53,630 --> 00:38:57,518 he took too much damage and then was swept away by the waves. 796 00:38:57,518 --> 00:39:00,920 [LAUGHTER] 797 00:39:08,054 --> 00:39:09,470 STUDENT 1: And so on the forecast, 798 00:39:09,470 --> 00:39:13,730 you can see that high water is coming for quite a while, which 799 00:39:13,730 --> 00:39:15,360 is going to be a danger for the toys, 800 00:39:15,360 --> 00:39:18,600 both in terms of possibly getting swept away 801 00:39:18,600 --> 00:39:22,270 and not having enough candy for all the toys that you're going 802 00:39:22,270 --> 00:39:23,683 to have to move out of the way. 803 00:39:23,683 --> 00:39:24,474 ANDREW: [INAUDIBLE] 804 00:39:28,910 --> 00:39:30,670 PROFESSOR: Ooh. 805 00:39:30,670 --> 00:39:32,494 He may be out of luck. 806 00:39:32,494 --> 00:39:34,730 ANDREW: Yeah. 807 00:39:34,730 --> 00:39:40,022 PROFESSOR: And when you've lost two toys, you lose the game. 808 00:39:40,022 --> 00:39:41,106 ANDREW: Pretty good. 809 00:39:41,106 --> 00:39:42,480 STUDENT 1: Thank you for playing. 810 00:39:42,480 --> 00:39:45,588 [APPLAUSE] 811 00:39:46,125 --> 00:39:47,500 STUDENT 1: So that's Hello Waves. 812 00:39:53,290 --> 00:40:01,100 So in our game we had a few challenges to overcome. 813 00:40:01,100 --> 00:40:04,700 The first was that our game was based 814 00:40:04,700 --> 00:40:10,150 on forecast-based financing, which is a very abstract topic. 815 00:40:10,150 --> 00:40:11,730 It's a pretty understandable idea 816 00:40:11,730 --> 00:40:14,510 of using information about the future and ideas of risk 817 00:40:14,510 --> 00:40:17,180 in order to decide where to allocate resources. 818 00:40:17,180 --> 00:40:21,160 But it's still a bit abstract and building a game around it 819 00:40:21,160 --> 00:40:23,300 took a little bit of work. 820 00:40:23,300 --> 00:40:25,550 It's useful to note that it's different than long term 821 00:40:25,550 --> 00:40:26,060 planning. 822 00:40:26,060 --> 00:40:28,790 It's not just thinking about what will happen in the future, 823 00:40:28,790 --> 00:40:30,410 you know, building a dam to prevent 824 00:40:30,410 --> 00:40:32,050 water or things like that. 825 00:40:32,050 --> 00:40:34,630 It's actually about using the information 826 00:40:34,630 --> 00:40:37,020 you have to make the best decision for events 827 00:40:37,020 --> 00:40:41,360 that may be upcoming in the semi-near future. 828 00:40:41,360 --> 00:40:43,370 We also wanted to avoid making a game that 829 00:40:43,370 --> 00:40:46,980 was overly preachy or simplified, where it was clear 830 00:40:46,980 --> 00:40:49,610 exactly how you're going to win and you could just basically 831 00:40:49,610 --> 00:40:51,470 push the forecast-based financing 832 00:40:51,470 --> 00:40:52,700 button to win the game. 833 00:40:52,700 --> 00:40:54,783 We wanted players to actually think and understand 834 00:40:54,783 --> 00:40:57,470 the concept there, instead of just coming up 835 00:40:57,470 --> 00:41:00,110 with the buzzword of forecast-based financing. 836 00:41:00,110 --> 00:41:01,642 And finally, we had the challenge 837 00:41:01,642 --> 00:41:04,100 of actually communicating that forecast to players in a way 838 00:41:04,100 --> 00:41:05,650 that they would be able to understand 839 00:41:05,650 --> 00:41:06,980 and then make use of. 840 00:41:06,980 --> 00:41:09,260 You could see in that game that we had water levels, 841 00:41:09,260 --> 00:41:11,010 and it would show on the map, and we 842 00:41:11,010 --> 00:41:13,280 found that players were pretty good at using that 843 00:41:13,280 --> 00:41:15,590 in order to make decisions about what 844 00:41:15,590 --> 00:41:17,110 was going to happen in the future 845 00:41:17,110 --> 00:41:18,934 and how to allocate their resources. 846 00:41:18,934 --> 00:41:21,100 The other big challenge that we had in the beginning 847 00:41:21,100 --> 00:41:24,950 was that our initial target audience was policymakers. 848 00:41:24,950 --> 00:41:28,702 Like for Snap, Pablo had come in and pitched us this game idea, 849 00:41:28,702 --> 00:41:30,660 and originally he had wanted us to build a game 850 00:41:30,660 --> 00:41:33,280 for policymakers that would help them understand 851 00:41:33,280 --> 00:41:35,349 the benefits of forecast-based financing 852 00:41:35,349 --> 00:41:37,890 and therefore convince them that they should develop policies 853 00:41:37,890 --> 00:41:40,880 that would give resources to plans based 854 00:41:40,880 --> 00:41:43,410 on forecast-based financing. 855 00:41:43,410 --> 00:41:45,279 So I would like to take you through a couple 856 00:41:45,279 --> 00:41:47,320 of our prototypes, just to show you the evolution 857 00:41:47,320 --> 00:41:52,210 and comment on our process. 858 00:41:52,210 --> 00:41:56,930 Actually, to preface that, we had a lot of prototypes, 859 00:41:56,930 --> 00:41:59,980 because our idea was so abstract and because we weren't sure 860 00:41:59,980 --> 00:42:01,550 how to address our audience. 861 00:42:01,550 --> 00:42:04,310 So we built a lot of prototypes to start with. 862 00:42:04,310 --> 00:42:08,094 We had ideas that ranged across levels of scope 863 00:42:08,094 --> 00:42:10,510 of what you controlled, where you controlled entire cities 864 00:42:10,510 --> 00:42:12,259 or where you controlled individual workers 865 00:42:12,259 --> 00:42:13,676 and moved them around, and then we 866 00:42:13,676 --> 00:42:16,009 would pull from all these different kinds of ideas, what 867 00:42:16,009 --> 00:42:18,180 worked, what didn't, what did people understand, 868 00:42:18,180 --> 00:42:19,400 what confused them. 869 00:42:19,400 --> 00:42:22,680 And from that we got a really good idea of what concepts 870 00:42:22,680 --> 00:42:24,300 helped people understand the idea 871 00:42:24,300 --> 00:42:28,350 and brought them into this final game that we ended up with. 872 00:42:28,350 --> 00:42:31,170 So the project started on October 15th, 873 00:42:31,170 --> 00:42:32,750 and this was our first prototype. 874 00:42:32,750 --> 00:42:34,850 It was a terminal-based game where 875 00:42:34,850 --> 00:42:37,970 you had some kind of information about a future rainfall, 876 00:42:37,970 --> 00:42:40,780 and then you had to type in your commands of how you controlled 877 00:42:40,780 --> 00:42:42,520 different cities. 878 00:42:42,520 --> 00:42:45,360 This-- Actually, people found it fun, 879 00:42:45,360 --> 00:42:48,050 but as you might expect the feedback wasn't very good 880 00:42:48,050 --> 00:42:49,860 and people didn't quite understand 881 00:42:49,860 --> 00:42:51,470 how to move forward with it. 882 00:42:51,470 --> 00:42:54,750 It put a lot of cognitive load on people. 883 00:42:54,750 --> 00:42:56,980 So when we moved forward, we tried 884 00:42:56,980 --> 00:43:01,300 to give people more easily understandable actions to use. 885 00:43:01,300 --> 00:43:04,350 But the problem with this game was that it was time-based 886 00:43:04,350 --> 00:43:05,580 and it updated every second. 887 00:43:05,580 --> 00:43:06,680 And so there were so many numbers 888 00:43:06,680 --> 00:43:09,054 flying at people that even MIT students who playtested it 889 00:43:09,054 --> 00:43:10,320 couldn't understand. 890 00:43:10,320 --> 00:43:12,550 So we figured that people like policymakers 891 00:43:12,550 --> 00:43:14,920 who didn't have much experience with games really 892 00:43:14,920 --> 00:43:17,230 wouldn't be able to understand the game at all. 893 00:43:17,230 --> 00:43:19,570 So instead we went to turn-based. 894 00:43:19,570 --> 00:43:22,870 And that helped, but at the same time-- 895 00:43:22,870 --> 00:43:24,370 it's tough to see on this projector, 896 00:43:24,370 --> 00:43:26,190 but we have a forecast underneath that says 897 00:43:26,190 --> 00:43:27,900 how much rainfall is expected. 898 00:43:27,900 --> 00:43:29,900 And again, that wasn't understandable to people, 899 00:43:29,900 --> 00:43:32,483 because they couldn't understand what three inches of rainfall 900 00:43:32,483 --> 00:43:34,520 meant for their city and they couldn't 901 00:43:34,520 --> 00:43:37,292 understand how that contributed to a possible disaster. 902 00:43:37,292 --> 00:43:39,250 At this point, Pablo actually came by the class 903 00:43:39,250 --> 00:43:41,840 and played the game, and then he told us 904 00:43:41,840 --> 00:43:44,270 that he wasn't even sure that he could get policymakers 905 00:43:44,270 --> 00:43:46,730 to play the game, because they may not have enough time. 906 00:43:46,730 --> 00:43:49,429 Instead he wanted us to switch to grade schoolers, 907 00:43:49,429 --> 00:43:50,970 because we could teach them something 908 00:43:50,970 --> 00:43:53,300 about forecast-based financing and help them understand 909 00:43:53,300 --> 00:43:55,687 as they grew up. 910 00:43:55,687 --> 00:43:57,395 And this was great for us, because making 911 00:43:57,395 --> 00:43:59,870 a game that was serious, easy for somebody 912 00:43:59,870 --> 00:44:02,400 who didn't have experience with games to play, and also 913 00:44:02,400 --> 00:44:06,320 fun and engaging was too difficult for us, actually. 914 00:44:06,320 --> 00:44:09,560 So moving to grade schoolers was awesome. 915 00:44:09,560 --> 00:44:13,950 He also suggested that we try to make the idea of rainfall 916 00:44:13,950 --> 00:44:16,960 or water levels more visceral, and that's 917 00:44:16,960 --> 00:44:20,080 when we came upon this idea of the rising waters. 918 00:44:20,080 --> 00:44:22,470 Whenever players looked at this, they instantly 919 00:44:22,470 --> 00:44:24,120 understood the concept of the game. 920 00:44:24,120 --> 00:44:25,700 The feedback might not be there. 921 00:44:25,700 --> 00:44:28,360 The beautiful UI might not be there. 922 00:44:28,360 --> 00:44:30,660 But the idea of having cities with workers in them 923 00:44:30,660 --> 00:44:33,377 and a rising water level coming towards them, everybody 924 00:44:33,377 --> 00:44:35,710 understood that, and it made it a lot easier for players 925 00:44:35,710 --> 00:44:37,860 to reason about the game. 926 00:44:37,860 --> 00:44:40,810 From there we added things like nicer art, better feedback, 927 00:44:40,810 --> 00:44:44,410 which you can't quite see in a static picture, more 928 00:44:44,410 --> 00:44:46,930 improvements to how the forecast worked, 929 00:44:46,930 --> 00:44:50,955 and eventually we ended up with our final version today. 930 00:44:53,574 --> 00:44:54,990 STUDENT 2: So to talk a little bit 931 00:44:54,990 --> 00:44:57,270 about our actual development process. 932 00:44:57,270 --> 00:45:00,430 So our team was structured into three main subteams-- 933 00:45:00,430 --> 00:45:03,300 production, which was in charge of managerial roles, 934 00:45:03,300 --> 00:45:06,600 deliverables, and playtesting, and so there was a shared 935 00:45:06,600 --> 00:45:10,950 responsibility there, a technical team, which 936 00:45:10,950 --> 00:45:13,700 was in charge of the bulk of the coding work, 937 00:45:13,700 --> 00:45:16,140 and a user experience team, which would be in charge 938 00:45:16,140 --> 00:45:19,800 of assets and UI design. 939 00:45:19,800 --> 00:45:23,390 We also initially envisioned subteam leaders, 940 00:45:23,390 --> 00:45:26,100 where we'd be kind of communicating through them. 941 00:45:26,100 --> 00:45:28,090 But we found the concept kind of redundant, 942 00:45:28,090 --> 00:45:31,060 and so we worked pretty much with, like, a flat structure 943 00:45:31,060 --> 00:45:34,210 between the three teams. 944 00:45:34,210 --> 00:45:37,770 So from the beginning, we encouraged good coding 945 00:45:37,770 --> 00:45:42,060 practice, and so we used good tools available to us. 946 00:45:42,060 --> 00:45:45,110 One of these is Yeoman, which is a JavaScript scaffolding 947 00:45:45,110 --> 00:45:47,110 framework. 948 00:45:47,110 --> 00:45:50,530 And this helped us out a lot by basically automating 949 00:45:50,530 --> 00:45:55,320 a lot of our JavaScript tasks and making our code modular. 950 00:45:55,320 --> 00:45:58,360 We also used Phaser's state machine, 951 00:45:58,360 --> 00:46:03,070 which is this kind of badly documented new feature 952 00:46:03,070 --> 00:46:06,030 in the Phaser game engine, which was the JavaScript game 953 00:46:06,030 --> 00:46:08,570 engine that we used. 954 00:46:08,570 --> 00:46:12,319 It's a bit badly documented, but it did save us 955 00:46:12,319 --> 00:46:14,360 a lot of headaches, and once we figured that out, 956 00:46:14,360 --> 00:46:16,840 that proved immensely helpful. 957 00:46:16,840 --> 00:46:22,890 And we also used MVC, which is a software engineering 958 00:46:22,890 --> 00:46:25,450 term standing for Model-View-Controller. 959 00:46:25,450 --> 00:46:28,990 And again, by encouraging these good coding practices, 960 00:46:28,990 --> 00:46:32,590 we reduced dependencies and made sure that our team was 961 00:46:32,590 --> 00:46:34,760 productive. 962 00:46:34,760 --> 00:46:37,280 In terms of communication, we-- also 963 00:46:37,280 --> 00:46:39,504 similar to Snap-- used Slack, which is kind 964 00:46:39,504 --> 00:46:40,795 of like a modernized chat room. 965 00:46:40,795 --> 00:46:43,180 It's very feature-rich, and so you 966 00:46:43,180 --> 00:46:46,110 can share files in channel and things like that. 967 00:46:46,110 --> 00:46:49,510 And we also used the idea of the daily Scrum. 968 00:46:49,510 --> 00:46:52,050 We implemented it in class, and we 969 00:46:52,050 --> 00:46:56,700 would say what we had done that class, what we would be doing, 970 00:46:56,700 --> 00:47:00,950 and what we wanted to do until next class. 971 00:47:00,950 --> 00:47:05,280 And so the major challenges that we faced during the development 972 00:47:05,280 --> 00:47:08,760 process, though, were that our team members 973 00:47:08,760 --> 00:47:10,610 came from very different backgrounds 974 00:47:10,610 --> 00:47:12,875 and had very different preferences about games. 975 00:47:12,875 --> 00:47:15,290 You know, some of our team members 976 00:47:15,290 --> 00:47:18,795 were very hardcore StarCraft players and very good 977 00:47:18,795 --> 00:47:21,610 at RTS games, while other members of our team 978 00:47:21,610 --> 00:47:25,570 preferred like a more laid-back mobile game, Fruit Ninja 979 00:47:25,570 --> 00:47:26,760 kind of approach. 980 00:47:26,760 --> 00:47:29,680 And so trying to mediate those two viewpoints 981 00:47:29,680 --> 00:47:32,300 and trying to create a game that would 982 00:47:32,300 --> 00:47:37,200 engage both types of gamers was a challenge 983 00:47:37,200 --> 00:47:40,300 that we had to overcome. 984 00:47:40,300 --> 00:47:43,640 And so another big challenge that we had in our development 985 00:47:43,640 --> 00:47:47,735 process, as you saw through our progression 986 00:47:47,735 --> 00:47:49,110 through our different prototypes, 987 00:47:49,110 --> 00:47:53,280 was that our direction was not very clear until about halfway 988 00:47:53,280 --> 00:47:54,570 through the project. 989 00:47:54,570 --> 00:47:57,950 And so partially because we had different ideas 990 00:47:57,950 --> 00:47:59,760 on what the game should look like, 991 00:47:59,760 --> 00:48:02,930 and also partially because we had such an abstract idea 992 00:48:02,930 --> 00:48:05,780 of forecast-based financing that even 993 00:48:05,780 --> 00:48:09,530 we didn't have that good of a grasp on initially, 994 00:48:09,530 --> 00:48:11,860 it took us a while to really get settled 995 00:48:11,860 --> 00:48:13,530 on what we wanted to build. 996 00:48:13,530 --> 00:48:18,050 And so this really challenged our development process 997 00:48:18,050 --> 00:48:23,990 and made us have to build a lot before we got something 998 00:48:23,990 --> 00:48:26,760 that we liked. 999 00:48:26,760 --> 00:48:31,720 And so eventually we did end up having to cut some features, 1000 00:48:31,720 --> 00:48:36,320 like multiple levels, or a guided tutorial for the player. 1001 00:48:36,320 --> 00:48:40,140 We thought that this would introduce too much new content 1002 00:48:40,140 --> 00:48:42,220 that would need to be playtested, balanced, 1003 00:48:42,220 --> 00:48:47,030 and tested to ensure consistency with the rest 1004 00:48:47,030 --> 00:48:49,660 of our game, which we viewed as taking up too much time. 1005 00:48:49,660 --> 00:48:52,910 And we also cut the idea of adding more individuality 1006 00:48:52,910 --> 00:48:57,930 to the workers or to the toys that you saw, 1007 00:48:57,930 --> 00:48:59,780 other than different graphics for each. 1008 00:49:02,300 --> 00:49:06,277 STUDENT 1: So some of the worst things that we did on our team 1009 00:49:06,277 --> 00:49:07,860 is that we spent a lot of time on work 1010 00:49:07,860 --> 00:49:09,780 that got thrown out entirely. 1011 00:49:09,780 --> 00:49:13,070 All the prototypes we did, they were actually pretty useful 1012 00:49:13,070 --> 00:49:16,070 because of the things we learned about the concept and about 1013 00:49:16,070 --> 00:49:17,720 how people would play the game. 1014 00:49:17,720 --> 00:49:19,570 But we did spend a lot of time on things 1015 00:49:19,570 --> 00:49:23,050 like art or nitty gritty details that didn't really 1016 00:49:23,050 --> 00:49:25,320 need to be figured out and that we could have put off 1017 00:49:25,320 --> 00:49:27,950 until later in the project. 1018 00:49:27,950 --> 00:49:32,280 We also kept the game direction to vague for too long. 1019 00:49:32,280 --> 00:49:35,280 As Norman said, we spent a lot of time with that, 1020 00:49:35,280 --> 00:49:38,470 and it probably ate up too much of our time. 1021 00:49:38,470 --> 00:49:41,200 Although it helped us learn, we could've 1022 00:49:41,200 --> 00:49:43,720 moved faster in the beginning to get to a solid idea. 1023 00:49:43,720 --> 00:49:47,320 Because once we got to a solid idea of the rising waters, 1024 00:49:47,320 --> 00:49:50,160 our team started to centralize around a lot 1025 00:49:50,160 --> 00:49:53,610 better because we could actually deal with something concrete. 1026 00:49:53,610 --> 00:49:55,680 While we were dealing with the abstract ideas, 1027 00:49:55,680 --> 00:49:58,180 everybody was all over the place and arguing about things 1028 00:49:58,180 --> 00:50:00,680 that didn't quite line up. 1029 00:50:00,680 --> 00:50:04,070 And the worst decision of all that we started with, 1030 00:50:04,070 --> 00:50:06,860 and that sort of made the problem of going too vague 1031 00:50:06,860 --> 00:50:08,472 and all these other things happen, 1032 00:50:08,472 --> 00:50:09,930 is that we were originally thinking 1033 00:50:09,930 --> 00:50:13,670 how do we skin forecast-based financing as a game? 1034 00:50:13,670 --> 00:50:17,490 How do we take this idea of using forecasting decisions 1035 00:50:17,490 --> 00:50:19,880 and then just gamify it? 1036 00:50:19,880 --> 00:50:23,280 Which we eventually realized wasn't fun, 1037 00:50:23,280 --> 00:50:26,156 and didn't help us actually come up with any ideas. 1038 00:50:26,156 --> 00:50:27,780 Instead, when we flipped it and started 1039 00:50:27,780 --> 00:50:30,910 talking about what game could we create and use 1040 00:50:30,910 --> 00:50:34,760 forecast-based financing to improve it and teach players 1041 00:50:34,760 --> 00:50:38,030 how to play the game, and therefore allow them to come 1042 00:50:38,030 --> 00:50:40,860 out of the game having learned about forecast-based financing, 1043 00:50:40,860 --> 00:50:42,180 the world sort of opened up. 1044 00:50:42,180 --> 00:50:43,846 Everything became a lot more interesting 1045 00:50:43,846 --> 00:50:47,532 and we found that we started to move faster. 1046 00:50:47,532 --> 00:50:49,990 Some of the best decisions that we made, on the other hand. 1047 00:50:49,990 --> 00:50:52,120 As Norman said, we had good tools, 1048 00:50:52,120 --> 00:50:55,260 which meant that even when we threw out prototypes, 1049 00:50:55,260 --> 00:50:58,660 although we wasted things like art resources and things 1050 00:50:58,660 --> 00:51:01,990 like that, we actually didn't end up wasting very much code, 1051 00:51:01,990 --> 00:51:04,640 because things like the idea of workers or cities 1052 00:51:04,640 --> 00:51:07,140 could literally be pulled out of the old games we had, 1053 00:51:07,140 --> 00:51:09,510 put into our new game, and then reworked 1054 00:51:09,510 --> 00:51:13,300 to build our new structure. 1055 00:51:13,300 --> 00:51:15,120 We also weren't afraid to trust each other 1056 00:51:15,120 --> 00:51:17,840 and throw out the things that didn't work. 1057 00:51:17,840 --> 00:51:19,530 Once we started moving fast, we had 1058 00:51:19,530 --> 00:51:21,210 a lot of ideas that would come out, 1059 00:51:21,210 --> 00:51:23,320 and we would say, OK, this doesn't work. 1060 00:51:23,320 --> 00:51:24,970 We're actually going to scrap it. 1061 00:51:24,970 --> 00:51:28,840 Or we think that this isn't the direction we need to go in. 1062 00:51:28,840 --> 00:51:32,281 And everybody was willing to go along with it. 1063 00:51:32,281 --> 00:51:34,530 It's not a good feeling to see your things thrown out, 1064 00:51:34,530 --> 00:51:36,950 but everybody understood that was for the best of the game, 1065 00:51:36,950 --> 00:51:38,741 and I really appreciate their understanding 1066 00:51:38,741 --> 00:51:40,340 with everything too. 1067 00:51:40,340 --> 00:51:42,360 And part of that all comes down to the fact 1068 00:51:42,360 --> 00:51:44,550 that we were on board with our final idea. 1069 00:51:44,550 --> 00:51:47,115 We were all excited about the concept that we had. 1070 00:51:47,115 --> 00:51:48,740 Part of that might have been the relief 1071 00:51:48,740 --> 00:51:50,531 of coming to a concrete idea after spending 1072 00:51:50,531 --> 00:51:51,950 so much time being vague. 1073 00:51:51,950 --> 00:51:53,720 But once we had that concrete idea, 1074 00:51:53,720 --> 00:51:57,760 we really moved fast and worked well around it. 1075 00:51:57,760 --> 00:52:00,695 So thank you, and any questions? 1076 00:52:00,695 --> 00:52:04,160 [APPLAUSE] 1077 00:52:07,572 --> 00:52:08,780 AUDIENCE: Who did your sound? 1078 00:52:08,780 --> 00:52:11,090 It's awesome. 1079 00:52:11,090 --> 00:52:13,200 STUDENT 1: That was from our UI team. 1080 00:52:13,200 --> 00:52:14,185 AUDIENCE: Oh. 1081 00:52:14,185 --> 00:52:14,685 OK. 1082 00:52:14,685 --> 00:52:17,015 It's really cool. 1083 00:52:17,015 --> 00:52:18,390 PROFESSOR: Where did you find it? 1084 00:52:18,390 --> 00:52:20,015 STUDENT 2: So it's on our credits page. 1085 00:52:20,015 --> 00:52:22,590 Most of it was online. 1086 00:52:22,590 --> 00:52:23,950 The credits page in our game. 1087 00:52:27,798 --> 00:52:31,165 [LAUGHTER] 1088 00:52:35,747 --> 00:52:37,830 STUDENT 1: It's a little small, I guess, up there. 1089 00:52:37,830 --> 00:52:39,449 PROFESSOR: Oh, OK. 1090 00:52:39,449 --> 00:52:40,490 STUDENT 1: But it looks-- 1091 00:52:40,490 --> 00:52:41,450 PROFESSOR: [INAUDIBLE]. 1092 00:52:41,450 --> 00:52:43,622 STUDENT 2: Yeah. "Hold My Hand," AJP, by-- 1093 00:52:43,622 --> 00:52:45,122 AUDIENCE: Would it be possible maybe 1094 00:52:45,122 --> 00:52:47,569 to create a short URL for this? 1095 00:52:47,569 --> 00:52:48,110 For the game? 1096 00:52:48,110 --> 00:52:49,470 STUDENT 2: Yeah, a bit.ly link? 1097 00:52:49,470 --> 00:52:50,180 STUDENT 1: Yeah, sure. 1098 00:52:50,180 --> 00:52:52,530 We can make a bit.ly link, and we'll send it out to everyone. 1099 00:52:52,530 --> 00:52:54,030 AUDIENCE: Yeah, just so we've go it. 1100 00:52:54,030 --> 00:52:55,314 Yeah. 1101 00:52:55,314 --> 00:52:58,010 STUDENT 1: That's a good call. 1102 00:52:58,010 --> 00:52:59,004 Yes. 1103 00:52:59,004 --> 00:53:01,045 AUDIENCE: Having watched the early crash and burn 1104 00:53:01,045 --> 00:53:04,126 of the playthrough, how often have people-- 1105 00:53:04,126 --> 00:53:06,250 I don't know if you've actually had a lot of people 1106 00:53:06,250 --> 00:53:08,490 to play your current version of your game-- 1107 00:53:08,490 --> 00:53:10,720 do people usually take a playthrough or two 1108 00:53:10,720 --> 00:53:13,720 before they start getting the concept? 1109 00:53:13,720 --> 00:53:14,700 STUDENT 1: Yes. 1110 00:53:14,700 --> 00:53:17,882 That's why something like a tutorial would be really nice. 1111 00:53:17,882 --> 00:53:20,090 Unfortunately, we didn't have the time to put it in-- 1112 00:53:20,090 --> 00:53:21,048 AUDIENCE: Yeah, no, no. 1113 00:53:21,048 --> 00:53:22,510 I was just wondering how that-- 1114 00:53:22,510 --> 00:53:23,176 STUDENT 1: Yeah. 1115 00:53:23,176 --> 00:53:27,302 Usually what happens is even if after maybe a couple turns 1116 00:53:27,302 --> 00:53:29,260 of playing through, they start to get the idea. 1117 00:53:29,260 --> 00:53:32,410 The problem is that as the water starts to rise, 1118 00:53:32,410 --> 00:53:36,880 they haven't prepared enough, and so all their toys will 1119 00:53:36,880 --> 00:53:39,757 starve or get carried away by the waves, which 1120 00:53:39,757 --> 00:53:42,090 is a bit unfortunate and probably makes the players feel 1121 00:53:42,090 --> 00:53:43,006 bad on the first time. 1122 00:53:43,006 --> 00:53:44,632 But then they-- It actually teaches 1123 00:53:44,632 --> 00:53:45,840 them to think ahead about it. 1124 00:53:45,840 --> 00:53:48,204 So the next playthrough, they're much more careful 1125 00:53:48,204 --> 00:53:48,953 and understanding. 1126 00:53:52,817 --> 00:53:55,725 AUDIENCE: So I was wondering if you 1127 00:53:55,725 --> 00:54:00,530 were playing any other games or thinking about other games 1128 00:54:00,530 --> 00:54:03,090 as inspiration or thinking about how 1129 00:54:03,090 --> 00:54:05,592 to deal with some of-- hitting the right level 1130 00:54:05,592 --> 00:54:09,860 of strategic thinking in your game. 1131 00:54:09,860 --> 00:54:13,760 STUDENT 2: Well, so I guess in terms of early on, because we 1132 00:54:13,760 --> 00:54:17,110 had a very different idea of what we wanted to do it 1133 00:54:17,110 --> 00:54:18,340 early on in the process. 1134 00:54:18,340 --> 00:54:21,390 We were thinking about games like Civilization and how 1135 00:54:21,390 --> 00:54:24,680 did they communicate all these complex worker movements 1136 00:54:24,680 --> 00:54:27,050 and managing multiple cities. 1137 00:54:27,050 --> 00:54:31,050 But once we actually came up with this game concept, 1138 00:54:31,050 --> 00:54:35,610 I think we had much, much smaller goals, 1139 00:54:35,610 --> 00:54:40,780 and so did we have any specific game models, do you think, or? 1140 00:54:40,780 --> 00:54:43,660 STUDENT 1: There's none that I specifically think of. 1141 00:54:43,660 --> 00:54:46,800 There were some things that we were sort of inspired by, 1142 00:54:46,800 --> 00:54:49,260 standard tricks like when you hover 1143 00:54:49,260 --> 00:54:51,380 over one of the characters, they got bigger. 1144 00:54:51,380 --> 00:54:54,202 Some idea of showing off that this is clickable, 1145 00:54:54,202 --> 00:54:54,910 things like that. 1146 00:54:54,910 --> 00:54:59,143 But specific games themselves, not really. 1147 00:54:59,143 --> 00:54:59,684 AUDIENCE: OK. 1148 00:54:59,684 --> 00:55:02,660 I was just thinking it ended up being kind of board-game-y. 1149 00:55:02,660 --> 00:55:05,340 And I feel like there's a lot of board games 1150 00:55:05,340 --> 00:55:08,384 where they've been thinking a lot about getting 1151 00:55:08,384 --> 00:55:14,168 that right level of tactics rather than strategy. 1152 00:55:14,168 --> 00:55:15,614 But yeah. 1153 00:55:15,614 --> 00:55:17,060 I guess it worked out too. 1154 00:55:20,450 --> 00:55:22,090 AUDIENCE: So you mentioned that you 1155 00:55:22,090 --> 00:55:24,950 were able to change how your workers looked 1156 00:55:24,950 --> 00:55:26,465 and just keep your models. 1157 00:55:26,465 --> 00:55:29,928 So that's the holy grail of object-oriented programming, 1158 00:55:29,928 --> 00:55:31,792 that you have an object that's reusable 1159 00:55:31,792 --> 00:55:34,220 and you don't have to throw out the code. 1160 00:55:34,220 --> 00:55:36,387 Do you think there's a reason why in particular you 1161 00:55:36,387 --> 00:55:37,470 were able to achieve that? 1162 00:55:37,470 --> 00:55:40,212 Because I think that's not common, necessarily, 1163 00:55:40,212 --> 00:55:41,503 in object-oriented programming. 1164 00:55:45,450 --> 00:55:48,050 STUDENT 2: Partially a little bit of OCD-ness, 1165 00:55:48,050 --> 00:55:50,240 like, very early on, very strictly 1166 00:55:50,240 --> 00:55:53,610 saying we're going to write this object-oriented code, 1167 00:55:53,610 --> 00:55:55,860 and we're not just going to hack things together. 1168 00:55:55,860 --> 00:55:58,660 I think that helped a lot, because we actually 1169 00:55:58,660 --> 00:56:01,680 spent the time in the very beginning to think about, like, 1170 00:56:01,680 --> 00:56:05,780 which objects were responsible for what 1171 00:56:05,780 --> 00:56:07,840 and what their purpose should be. 1172 00:56:07,840 --> 00:56:11,592 So basically, I think because we moved more slowly in the start 1173 00:56:11,592 --> 00:56:13,800 and thought more carefully about how that code should 1174 00:56:13,800 --> 00:56:18,780 be structured, we ended up with having an easier time later on. 1175 00:56:18,780 --> 00:56:21,470 STUDENT 1: There's also the fact that because we had learned 1176 00:56:21,470 --> 00:56:23,700 these things from the prototype that we were putting 1177 00:56:23,700 --> 00:56:27,880 into this game, that also meant that the objects that we 1178 00:56:27,880 --> 00:56:31,570 created-- because we wanted similar functionality to things 1179 00:56:31,570 --> 00:56:33,970 that we had already seen and we knew worked, 1180 00:56:33,970 --> 00:56:36,490 it meant that we were comfortable pulling out 1181 00:56:36,490 --> 00:56:38,970 the functionality into that. 1182 00:56:38,970 --> 00:56:42,840 So I don't want to say it was designed to fit, 1183 00:56:42,840 --> 00:56:48,419 but there was the fact that we moved it on purpose, really. 1184 00:56:48,419 --> 00:56:50,252 AUDIENCE: I would like to something on that. 1185 00:56:50,252 --> 00:56:54,110 So we had a very good MVC model. 1186 00:56:54,110 --> 00:56:58,119 So models were in this tree-like structure, 1187 00:56:58,119 --> 00:57:00,584 and it was very easy to change models. 1188 00:57:00,584 --> 00:57:03,570 So models knew about-- Sorry. 1189 00:57:03,570 --> 00:57:05,620 User knew about models, but models 1190 00:57:05,620 --> 00:57:06,870 had no idea about [INAUDIBLE]. 1191 00:57:06,870 --> 00:57:08,370 So basically, it was quite easy. 1192 00:57:14,070 --> 00:57:15,270 PROFESSOR: Thank you. 1193 00:57:15,270 --> 00:57:16,170 STUDENT 1: Thank you. 1194 00:57:16,170 --> 00:57:19,520 [APPLAUSE]