Did Active Learning Work?

• Download English-US transcript (PDF)

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation or view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at OCW.MIT.edu.

PROFESSOR: So I am going to start this talk by explaining the title. So we'll start with active learning. What is active learning? In my mind, it's this pretty broad term. It's a form of learning that attempts to involve students more directly in the learning process. And there are all sorts of things that fall under the umbrella of active learning.

So things like having students engage in group work, in class-wide discussions, in debates, acting out certain concepts, group problem solving sessions, all of these things. And so in all of those activities, students are engaged. They're not passively listening to a lecture, they're actively participating.

And there is some evidence that this style of learning is good for a lot of things. There is evidence to support the effectiveness of student engagement in exam scores, failure rates, how well students remember content, student attitudes, study habits. And there's also evidence that active learning has a disproportionate benefit for minorities, students from disadvantaged backgrounds, female students, and male dominated fields.

So there's evidence that active learning creates this more inclusive environment. So it's this wide, wide variety of techniques reported to have a lot of benefits. So that's active learning.

Now, what is 6.033? So 6.033 is computer systems engineering. It's a required class for all computer science majors, which means as a result, at MIT it's quite large. So like Dipa said, we have about 400 students. It runs once a year. So every spring we get 400 new students. Most of them are juniors and seniors.

And in some ways, 033 is a typical MIT class. We have lectures twice a week. We have recitations twice a week. But in most other ways, 033 is not a typical MIT class. For one, it's a CIM class. So it's a Communications Intensive in the Major class. That means that most of our assignments are based around writing or oral presentations.

It's not a class where students are implementing or building a lot of things. We focus heavily on reading and on writing. In particular, we have tutorial sections each week, dedicated to the communication material, taught by dedicated communication staff.

The major assignment in 033 is a design project. It takes most of the semester. The students completed in teams, usually teams of three. And it's comprised of a few written assignments and oral presentations.

One of our big focuses for that project is getting students to recognize the trade-offs they need to make when they're designing a system, and getting them to defend those trade-offs. So this is not just a class about how computer systems work, but how we design them, how we decide what choices to make when we're building those systems.

So the last thing that makes 033 pretty unique, and is what I'm largely going to talk about here, is that our recitation sections are pretty intense. They are taught by faculty, senior research staff. We have about 25 students per section. So you are right to think that, oh, my gosh, they have like 18 sections per semester. We do. It's a lot.

Each recitation focuses around a technical paper that the students have read beforehand. And the goal for those discussion sections, those recitations, is for them to be largely discussion based. And in fact, that has always been the goal of 033. 033 has been around for almost 50 years at this point. It's been around since before I was born.

And the goal has always been for these recitations to be discussion based. During the recitations, we're talking about, we talk about the system that the paper describes-- does it work? What were the trade-offs that the authors made? Do we like this system? How would you design this system? All right, so that is 033.

Now, why put active learning into 033? And you might think that it's just because, well, there's evidence that active learning can be a good idea, so why not try it? That's fair. We were actually trying to solve a pretty specific problem in the class though.

So like I said, ideally the recitations are based heavily around discussion. And again, that has always been the goal. Because this is a class about system design, not just about how these systems work, one way that we teach that is by having the students analyze and critique existing systems before they go out and try to design their own.

And so, what are the goals of the designers? Does the system meet those goals? Are those goals important? If you were redesigning the system, how would you do it? What would your goals be? What trade-offs would you have to make? Things like that.

A large part of that instruction happens in the recitation, where we're having these discussions. And they go and apply those skills to their design projects. So to get that to happen, it's important that the students are engaged in recitation, that they're reading the paper beforehand, they're asking questions, they're answering questions, they're coming prepared, things like that.

And for a long time, that was not happening for us. We had a lot of trouble with this. We found that students weren't willing to volunteer answers. When we'd ask them a question, nobody would raise their hand. We found that some of them weren't even reading the paper beforehand. So they're just kind of coming in confused.

As a result of all of this, our instructors would end up spending the first 15, 20, 25 minutes of the recitation lecturing about the paper, because so many of the students didn't come prepared. And then that just made things worse, as you can imagine.

Students know that they're going to come into recitation, that they're going to get a lecture, so why bother reading the paper, if your instructor is just going to explain it to you. And so it was just kind of making things worse.

We weren't getting to teach the material that we really needed to teach, because we were spending time rehashing these papers. The students weren't learning how to read these papers, how to analyze them, how to critique them. And honestly, it wasn't much fun. It wasn't all that much fun to teach this way.

Occasionally, we could get a discussion going in a class. But we would then encounter another problem, which is pretty common, where we'd have a discussion, but it would be dominated by two or three students in the room, the same two or three students. And then other students are tuning out, or even becoming sort of visibly uncomfortable, not wanting to volunteer these answers. So that's where we were.

And we had this problem. We really wanted the recitations to be this really fun place, where the students were coming, and they're wrestling with all of these new ideas and proposing these new things. But we didn't have that. We had this place where students aren't coming prepared, or if they're prepared, if we can get a discussion going, it's not the kind of discussion we want. It's sort of dominated by one or two people.

So after some thinking, I thought active learning might be a good approach to solve both of these problems. So, as I mentioned earlier, there's some evidence that active learning can be good for creating an inclusive environment in a classroom. So for instance, one common technique, which I will talk a bit about later, is you pose a question to the class, but put everybody in small groups, have the students discuss in small groups, and then kind of bring the classroom back together for a wide discussion.

Students are in their groups, they get a chance to kind of vet their ideas with their peers. It's sort of hard if you're in a group of two or three to not participate. You sort of have to say something. So they'll get to vet their ideas with people, they'll get some reassurance.

And in theory, they are then more willing to bring that idea back to the whole class. That type of activity can also give you a relatively easy way to make sure one person doesn't dominate the discussion, by saying, OK, I split you guys up into five groups, and let's hear from each group now.

Moreover, I thought if we did a lot of active learning, it might keep the students engaged for longer. It would help us avoid lecturing. It would help us set this precedent that you come to recitation having read the paper, we're not going to lecture for you.

So that's kind of where we're starting. That's the baseline for this talk. Now my goal for all of you is that this talk is, I hope, exceedingly practical. I'm going to talk about some specific ways that I trained my staff to do this, the types of active learning activities we do, some lessons we've learned, how we've evaluated our success.

I hope you'll be able to apply some of these ideas to your own classes if you want. I do want to start, though, with a disclaimer. This is really a case study of how active learning has worked in 033. I'm not here to make claims that active learning is a solution to all of your teaching problems. I think it's very good. It's probably not the solution to everything.

Nor am I even-- I'm not even here to really evangelize it over any other particular pedagogical schemes. I think it works well in 033, I think we've done some really cool things. I hope you're excited to hear about them.

So active learning. As you've probably picked up at this point, what I'm going to talk about, we're infusing the recitations in 033 with active learning. And so that means that really it is my recitation instructors who are on the ground doing a lot of this.

And I will admit, when we were starting out, I sort of thought-- I had this amazing idea to do active learning recitations. And I was like, well, I am done. I have solved the problem. I just told my instructors to implement it. It was like, guys, active learning.

It didn't work. I was perplexed, because I told my instructors correctly that these types of activities would help increase participation. My instructors were the ones complaining about the lack of participation. And so I just thought come on, what's the problem.

Well it turns out you cannot tell your instructors to do a thing that they've never done before and just have them magically do it. In particular, you can't tell your instructors to fundamentally change the way they teach and magically have that happen. I would say it's difficult enough for us to change the way we teach, much less to get other people to change the way they teach.

So the first part of my talk is changing the way other people teach in four easy steps. So step one is get everyone on board. So this is the first thing that I did, last year in 033. I will keep saying last semester, but last year.

I presented active learning as our primary goal for the entire semester. So we did not wait until the course got settled down to start doing it. In fact, our very first staff meeting, which happens before the semester has even begun, was about active learning. We had everyone on the staff there. So me, all of my recitation instructors, all of my TAs, all of the communication instructors. This is about 30 some people here.

And we talked about active learning. We talked about why it's a good idea, the evidence that it's a good idea, evidence that it is better than lecturing. We talked about why running the recitations in this way would support the other learning objectives in 033, in particular the objective of getting students to design their own systems and evaluate their own tradeoffs.

I explained to my instructors that there would be extensive support for this. And that we'd be checking in throughout the semester to make sure this was really working for them and working for the students. Now I did expect some pushback on this in this meeting.

Partly, I am asking a lot of my staff here. And also, I am a lecturer in charge of a lot of faculty, who have been here for a very long time. So I was not quite sure how this would go. I do not know if my staff knew about this. I actually went so far in preparing for this staff meeting to not only write out all of my objectives and why active learning was a good idea, but to write my own rebuttal to myself and then rebut that.

So I sort of had, OK, if I didn't want to do this, I'd be asking-- well, I don't think active learning is a good idea, I don't think this is going to work. And I had prepared answers for all of these questions. I looked at the notes for that meeting to prepare for this. And it really went on for quite a while.

I did not actually get that much pushback here, because my staff is amazing. But also, I talked to a few of my instructors ahead of time about this, people who had been with the class for a while, who I knew would be maybe a little bit more game for this idea. I kind of got them on board first.

I also appealed to everyone's scientific nature, explaining that this was somewhat of a research project, which was true. I did want to know would this work in 033. I was heavily biased towards the fact that I thought it was going to work. But we had methods to evaluate this throughout the semester.

And I told them if this goes horribly, we won't keep doing it. I'm not going to burden you with this. And I enthusiastically mandated that this is what we were trained for this semester. In previous semesters, I had suggested some of these ideas, encouraged my staff to try it. And that really wasn't enough.

And so on the one hand, get everyone on board. It was one staff meeting. I will say, it was really tempting for me to not have that staff meeting so soon. 033 is such a big class. It has so many people involved. It feels like we're just scrambling to get the logistics worked out in the first week or two. Even for us assigning our students to sections is this whole thing.

But doing this really set the tone for the whole semester. I was kind of surprised at how helpful this was. Everybody knew the active learning was important. Even the people who weren't the recitation instructors, all the TAs, all the communications instructors, they knew that it wasn't optional. It was their primary goal of the semester. Any other changes that we made in the class were largely going to be in support of this goal. So step one, get everybody on board with your mission.

Step two is do a lot of work, do a lot of planning. So step one was our first staff meeting. Step two, what's happening in our second staff meeting. So one of the things that we did is we talked about a pretty well-known list of something like 278 different active learning activities. I knew it was a lot to ask really anyone to read through all 278 active learning activities.

And so I annotated this list. And we kind of discussed the highlights in the staff meeting. I am going to go into detail in the second part of this class about some of the specific activities we do in 033, but I'll give you some examples from this list just for concreteness.

So one of them, which I really like to do in lecture actually, is-- they all have funny names-- OK, so fingers on chest. So this is having students vote on a multiple choice question, but using like fingers on their chest. So option one, option two, option three.

And that way, the students can't see what other people are voting really. They're not quite like, oh, everybody else is voting for one. They can't really see, so they tend to be more truthful. A lot of times in lecture, I ask them just do they feel good about something or bad about something.

Another one, a fairly common one-- think, pair, share. So posing a question to the class, asking students to think about it, then to share their ideas with a partner. And then maybe go and address the larger class. And then a third one, which I like just because it has a nice name-- the name for it is human Tableau.

And so this is having groups create living scenes which relate to the classroom concepts or discussion. So that one's a little bit more out there. But these are the types of activities we're thinking about.

And of course, we went through this list. We looked at a whole bunch of activities. We're not going to end up using all of them in 033. But going through those list, it was a really good exercise for all of us. It gave everyone a much better sense of what is active learning or what can active learning be.

But even kind of armed with that list, it's still a lot to tell your instructors, here's your 278 activities, now go, now do active learning. So in addition to this, I pre-planned a lot of active learning techniques into each recitation.

So normally, before active learning, the way that I would plan recitations was strictly just for technical content. I would give my instructors a handful of major technical issues they needed to hit on. It helps keep our recitations somewhat uniform so that we can be fair on exams. We do test the technical material on exams in this class.

Beyond that though, my instructors have a lot of leeway to teach these topics however they want, a lot of leeway on how they teach in general. I had never before tried to steer them in a particular direction. But now, in addition to that technical content, what I would do is I would put two or three active learning activities or techniques into each recitation.

So things like pointing out places where they could have students break into groups and think about a particular question, pointing out places where they could hold a debate in the class. And again, I'm going to talk about these specific activities we do in the second part of this talk.

But planning activities in this way meant that my instructors has had a lot of ideas of how they can make this work in their sections. Giving them multiple ideas and multiple types of ideas really meant that they could pick the ones that they were the most comfortable with.

So instead of saying, hey, here's this really, really out there idea, everybody go try it, some of them might try some of the more extreme ones, but everybody could kind of find something that they were comfortable with.

So get everybody on board, plan things, And the next thing that I did was support my staff as individuals. So I didn't plan these activities and set the staff free. I actually observed them throughout the semester. And I made sure to stress that it wasn't any sort of evaluation. I was not there to say, oh, boy, Karen, this is really not working out. I was just there to see what was working and what wasn't, so that we could all iterate on these things.

In practice, this was great. I, of course, found way more good things in doing this than anything problematic. Most of my feedback to my instructors was, hey, you did this really awesome thing, you should do more of that, and can we share it with everybody else. And so, I really, in the end kind of thought of myself more as a cheerleader for them and what they were doing, than someone who was coming in are really critiquing anything.

I also got a lot of great ideas from them, both in how to present certain technical content and twists on some of the active learning techniques that I had suggested. So related to that, I'm observing them as individuals, but we would also talk a lot as a group about how this was going. Every staff meeting we'd take time to talk about what went well the previous week.

Our instructors would share active learning ideas that they tried, report on how they went. This was really nice. It got some of my more hesitant instructors kind of on board with this, to say like, oh, well, that person tried it and it seemed like it went really well. Maybe I'll try that next time.

And of course, the thing that turned out to be so great about this, is that all of my instructors have these amazing ideas on how to teach certain things. But to them, it doesn't seem special, like this is just always the way I've taught operating systems. And so, now we have this space where we're getting to share all of these ideas.

A lot of them relate to how our active learning has gone. But sometimes it's just, hey, I taught this paper this really cool way. It has also-- the fact that we've now kind of fostered this space for discussion at every staff meeting, everybody is pretty willing to bring up things that aren't going as well, to maybe come in and say, hey, I tried this active learning thing and it did not work at all.

What do you all think? Is there something I could have done differently? Is this just not something that will work for our class? And so that has been really, really great, having this space for my staff to kind of reflect on this and not feel judged in any way.

So this is the first part of getting active learning to work in 033, which was a lot of staff training. But now the second part-- what do we actually do? What are these activities? I would like to give you some specific examples from 033, but I will also try to give you some general feedback as well.

I realize you probably won't be able to take the 033 activities exactly into your classes, so let me talk about some of the common things that we do. So these are things that we've seen work really well in our classes.

So one of the most common types of active learning that we do is one I have already referenced a lot, which is to put students in small groups, have them do something, and then bring them back for a class-wide discussion. So let me give you some really specific examples.

We have a recitation where we talk about content distribution networks. The paper they read describes various goals of those networks, so performance, reliability, scalability, things like this. And what we'll do in that recitation is we'll divide the class up into groups. And we'll have each group think about what aspects of the system support a particular goal.

So you think about performance, you think about reliability, you think about scalability, et cetera. And then we'll bring the class back to a discussion once they've thought about that for a while.

Another example, when we talk about naming in the Unix file system, we start by asking them, what things have names? What things in Unix have names? And again, we'll put them in groups. We'll try to get every group to come up with as many things as possible.

Because when we bring them back together, we'll try to get a different thing from every group. So you can't all just say files, you have to say like files, and directories, and processes. And there's a lot of things in Unix that have names. I'm not here to make you Unix experts.

And so, in both of those examples, like I said, we'll bring the class back together for a discussion. We'll try to have each group say something. And as I said before, we've seen very clearly, talking in these small groups before hand, really makes the shyer students a lot more confident. And asking each group to contribute to the discussion means that we don't end up with something that is dominated by one or two groups.

Often, we'll even be a little bit more relaxed about it. We will just ask a big question, some sort of thought question, not just a tell me this fact from the paper. We'll break them up in the small groups, have them think about it, and come back together for a discussion. It's nice, it breaks up the recitation. It gets them kind of talking and doing things.

We occasionally will have our TAs teach in this class. And we encourage them to use these techniques as well. And one of them asked me a really good question that I realized I had never thought about. She said, what do I do when they're in groups? Like, do I just stand there and stare at them? Probably not, that's probably not good.

So to tell you what we do while they're in groups, sometimes our instructors might actually check in with each group, to say, oh, hey, what are you guys thinking about? Sometimes, you're kind of lightly monitoring, you're walking around.

I know when I cover recitations, sometimes I use it as time to prep for the thing that is coming next. So if I need to draw something on the board that's going to take a minute, I'll put you all in groups so you can discuss while I draw that. So this works really well as a technique to get students engaged and talking, and honestly, it's also nice from a classroom management perspective. It breaks things up, it gives you a little break from talking and you can drink some water.

So that is kind of our starting point. This is one thing where I would say we do something like this in almost every single recitation, every Tuesday and Thursday. The second one is one that I really like, which is to have the students debate. It's a little bit harder to work into a recitation. You need a question that can be debated.

For us, we have one recitation where the students read two short papers. And the papers actually come to opposing conclusions. And so it's an interesting lesson for them, well, how could this happen in science? How could data say two different things? It is mind-blowing for them, honestly, it's pretty wild.

And so we'll split them into two teams. And we'll give them each a paper, have them debate. The students usually really like this. They love to argue, so they're very excited to do this. It does involve a bit more monitoring. You want to make sure that one team, or one person, or one team doesn't dominate.

To combat that, we'll always have the teams converse beforehand, kind of get your argument together so you're ready to prepare it for the class. I personally think something like that could also work in small groups. Maybe you have groups of four, teams of two, debating.

I will say, when I've taught this recitation, of these two papers, there's like a clear winner in my mind. The last time I taught this, the opposing team made some very salient points. So they really do well with this. So this one is one that I think is really fun, just a little bit more difficult to work in because of what it requires.

A third one, something that we've actually started doing more of this year, is having them draw pictures. So we'll ask them, in groups, to draw pictures on the board of whatever system we're talking about, or maybe some component of that system. And what we end up with is a wide variety of drawings of varying levels of artistic skill.

And we can come together and talk about, what is each drawing showing? What are the common things that we're seeing in all of these depictions? What level of abstraction is each drawing capturing? That's something that comes up a lot for us in 033.

I like this for a few reasons. For 033 specifically, part of our communication curriculum involves teaching the students how to design and draw figures. So this, for us, this is just a great way for them to practice. It also forces them to figure out exactly what the system is doing. And so we can expose some areas where they might have missed something, which always happens.

We expect them to read the papers before they come in. They should be able to draw something. It might not be perfect, and that gives us a great space to explore that, like why was this confusing? What is it about the paper that made this confusing? It's also just fun. So the students get a kick out of it.

Now the last activity that I want to talk about, this is, I'm going to say that this is one of our more extreme activities. We actually do it pretty frequently. We will have students act things out. Also, the hardest to make an icon for, I just didn't know what to do there. They're acting, you can tell.

Now, I tell people about this a lot, and whenever I say it, like, oh, we act things out in 033 all the time, they are for sure intrigued. They are not sure exactly what that would entail. So I was going to have you all do this activity, but we're half going to do this activity.

So I kind of want to go through one of our examples actually in detail. We have them act out a system called MapReduce. But for this to make any sense, you need to understand a little bit about what MapReduce does. So you're going to become-- I don't want to say MapReduce experts, MapReduce novices for the next five minutes.

All right, so, brief aside of technical material. MapReduce is this system for performing parallel computation. And so here's the canonical example. We have a large piece of text. The Library of Congress has a lot of books in it. These books, we're going to digitize this, and so the data from these books is spread out on all different machines.

So we have a lot of computers. Let's say, you all can be computers. So each one of you has a book from the Library of Congress. And here's the problem that we want to solve, the canonical MapReduce problem. We want to count the number of occurrences of each word in this entire corpus.

So I want to know, how many times does the word apple appear? How many times does the word ball appear? How many times does the word the appear? OK? It's the canonical problem. You all don't look like you actually want to know that. You do, you want to know how many times the word apple occurs in the Library of Congress.

So you can imagine doing that with one computer, where that computer just reads every book in sequence and keeps track of these counts. And that would be very slow. So instead, what we can do is we can have multiple computers participate. You all have a portion of the Library of Congress. You all can read your data. You can make accounts for your pieces of data. This is the map phase.

And then we need some way to kind of combine those results. So if Dipa finds that the word apple occurred five times in her text, if Jesse found that it occurred six times, I need some way to add those up. That would be the reduce phase. And so part of the insight of this system is that there are a lot of common problems in computer science that can be described this way.

This is a system that came from Google. So Google is doing this on your data all the time. They are mapping and reducing everything about you. So let's see how that would actually work, how this system would work.

Jessie, come play your role in this thing.

JESSIE: Do I have to wear the hat?

PROFESSOR: You do have to wear the hat, it's active learning. I've got to write master on it.

JESSIE: I'm ready.

PROFESSOR: I think a lot of things in life should be more fun. So there's really no reason for Jessie to wear this hat. But there's also not a reason for her not to wear this hat. So--

JESSIE: There might be a reason.

AUDIENCE: This is about who this is fun for.

JESSIE: Fortunately, I can't see you, so now I feel comfortable.

PROFESSOR: Well, you can tilt the hat back a little bit maybe. So in MapReduce, there is the one machine that acts as the coordination for this, the master machine, as Jessie is so-- doing such a great job acting for us.

And so what the master's job would be-- you don't have-- you're just-- we just want to visualize you. You're just a prop, yes. So it would start by assigning jobs to workers. Now you all, I didn't make hats for all of you, there wasn't-- I know, I'm very sorry, but you're all workers. Only the master gets a hat.

All right, so you are all workers. So one of the first things Master Jessie would do, would be to assign map jobs to workers. And she would tell you things like exactly what data to operate on. So imagine a scenario where you, a worker machine, has data from a ton of different corpuses on it.

So Jessie actually has to say, OK, worry about the Library of Congress part of your data. And you would begin operating on that. You'd count your words. And you'd write out some results to your disk, that you also have as a machine in this. And then when you were done, you would report back to Jessie.

And you'd tell her some various meta information, kind of like, these are the words that I found. You don't have to send her all of your results, because the Library of Congress is big, you have a lot of data. But you might tell Jessie, hey, I found apple.

Now, once all of you are finished doing your map jobs, Jessie has all of this kind of meta information, she can start assigning the reduce jobs. And the reduce jobs are a little bit different. So Dipa if it was your job to reduce, say, the word apple, you need to know a couple things.

You need to know, well, which one of these machines actually found the word apple. It's not necessarily they're going to be in every piece of data. And Jessie would be able to tell you that. She'd basically say, hey, Dipa contact these workers, they have the data you need, you can add them together, in this example.

MapReduce functions can be much more complex than this. I thought let's start slow. All right, so the reduce workers would go, you'd reduce, you'd report results back to Jessie, and then we'd have our full counts.

I'm trying to decide how long I want you to stand here, and the answer is for the rest of the talk, but you can almost sit down. All right, so this is the basic--

JESSIE: I thought I'd have more work to do.

PROFESSOR: I thought that would be a lot. I didn't prep you for that. Don't go yet, though.

JESSIE: I'm just moving over.

PROFESSOR: So this is the basic idea of MapReduce. This is the type of thing we'd actually expect the students to know coming in. They've read the paper, and they should understand this sort of level of coordination. But there are a lot of details that we left out.

For instance, I didn't ask you all, hey, what happens if a machine fails? Worker Dipa crashes, what do we do?

AUDIENCE: [INAUDIBLE]

PROFESSOR: All the time, right? Is that data that she was storing, is it anywhere else in the system? How do we know? What if she had half finished the job? What do we do? Or even, what if there was no crash, but like let's say the worker Janet is just running really slow, like she's going, she's working hard, but everyone else is finished and we're just waiting on Janet.

And these are those types of things. Those are the interesting things about MapReduce. That's the point where there are tradeoffs for the students to think about. So it's not just important to think about, oh gosh, what does happen if a machine crashes or is slow, but why does it happen that way.

So to do this, we actually have them act out this system. And from the way that I've described it here, you might think, well, I have one student be the master and everybody else is a worker, that's kind of what I was setting you all up for. This is not actually how we do it.

Normally-- for one, it doesn't work with the size of our recitations. Even here, this is a lot of workers for one student to be in charge of. And also, if Jessie was maybe more of a shy student, she's like this is the worst day of my entire life. Katrina has made me wear a hat, and I have to stand-- I am in charge of everything, it's not good. It's not good.

So in practice, here's how we do this activity. You can sit down if you want. You don't have to be here anymore.

JESSIE: Should I keep the hat on?

PROFESSOR: I think, definitely, yes. I think absolutely. So we do start by assigning students roles. But we will usually have them in teams. So there will be a master team. Each worker will be a team of maybe two or three students.

This is good for a lot of reasons. Again, for the reason of getting the shyer students to be a little bit more comfortable, no one student is in charge of something. From a logistical standpoint, again, I glazed over many a detail in MapReduce, but you can imagine that the master has a lot to do, a lot to keep track of.

And so we will often, we'll have our master team. And we'll actually give each student in that team a task. So one might run the clock, timing in the system is important. We'll see an example of that in a second. One keeps track of what map tasks to complete. One assigns the tasks, things like that.

We have found-- we've actually done this activity a number of times. It gets better every time. One of the things we learned the most quickly is that the master-- we had to give the students who were doing the master extremely detailed instructions. If we just said do the master's job, they were not ready for that.

In some sections, in fact, some of my instructors prefer to do this where the instructor just is the master, and so they can be kind of in charge of coordination. Workers in this example get a little bit of an easier job. Again, we're putting them in teams of two to three. Sometimes we might give them a sheet with words on it, and actually ask them to count the words.

Sometimes we'll give them a sheet that kind of just says here's the words in your data and here are the counts. That might seem like, oh, gosh what's the point, they're not actually doing the worker job. Counting the words in the text is not the interesting part of this system. So sometimes we'll just skip that. It also, again, lets us do some very precise things with timing.

So once these roles are assigned, we're going to run the protocol. So the person keeping track of the master clock will advance it, one click, two clicks, et cetera. Various map jobs will complete. Most of the time, when we give them a paper that says, here's your words and your counts, we'll also say, hey, it takes three clock cycles for you to complete.

And so once the clock has ticked three times, they'll report back to the master team. Because of how we set up those map tasks, some of them will also fail. Some of them will have papers that say, oh, I failed after click two. And they have to announce it to the class. And at that point, we pause.

What happens? What happens when a machine fails? What does MapReduce do? Why does it do that? The spoiler for you all in MapReduce, the master will reassign that task to somewhere else. And part of that challenge is making sure that the new worker has access to the data that was on the crashed machine. MapReduce replicates data, so we talk about how that happens.

And the protocol continues. The students will find that one map task, it hasn't crashed, but it's taking a very long time. And that's another good point for discussion. What does MapReduce do? Does it just wait? In practice, it restarts the task in another machine.

And then, so, oh, there's a question, these two machines are running the same thing. What do we do? Who do we listen to? How do we get the data? And this is why the timing of that activity is so important, because a lot of the discussion points about MapReduce come from what we do when things are slow or when things fail. So being in control of that timing is crucial.

We do have some variations on this activity though. One of our instructors this year had students count characters instead of words, same idea. But of the workers was counting Russian characters instead of English. So they just organically were much slower than the rest of the class. And that was kind of a nice way to show that.

Many of our instructors have the students count types of candy instead of words. And we bring them candy. That's a big hit. In fact, after we did this recitation this year, one of our students asked why can't every recitation be like the MapReduce one. It is not clear how much the candy factored into that question.

So it has taken us quite a few iterations to get that activity working. And again, I know you're not all teaching MapReduce in your courses. You're not going to take that activity to you. But the things that were interesting for us, having the students in teams for each of these roles made a huge difference, both in terms of making sure they were all doing the correct things, but just getting them more comfortable with it.

I will tell you, the students go for this. They are into it. They map and reduce. They do not think it's stupid, which is very, very nice. And making sure that the master has these exceedingly detailed instructions, figuring out how to make the timing work, so that all of the things we wanted to discuss would actually happen.

And so really the activity is the way I describe it-- Karen, this is how Karen does it. Karen, you're the authoritative source on how we do MapReduce-- yeah. I want to talk briefly about another system we act out because it's fun. It's one of my favorite things that we do. It's much shorter.

We do this for a system called Raft. And Raft, like MapReduce is this large distributed system on a lot of machines. Timing is also important in Raft. In particular, machines set randomized timers. I picked some random number of seconds, milliseconds, whatever. And when that timer expires, they do something.

So being able to act that out is actually really crucial. And we thought, gosh, how are we going to randomize them? And so one of my instructors came up with the best idea, which is to have the students spin around in a circle. So like literally, they're standing here, and then they kind of-- they do this, and it takes a while.

Because he figured out that students just walk around naturally at different paces. So they've effectively randomized themselves. Someone will finish first. So I need you to imagine feel like you're walking around MIT, and you're just kind of like, oh, what's going on in this class.

And you see seven or eight students up at the front of the room just very quietly spinning around in place, while all of the other students are like-- the person who finishes first wins. And so who-- who is it? Who's going to be the next leader of this election cycle in Raft. It's really-- it looks ridiculous, but it works so well. And it brings me a lot of joy.

So to kind of wrap up our activities before I get to results, effectively, how has this worked for us-- again, we do all of these things a lot. This one in particular, small groups, a class wide discussion-- although acting things out, we act a lot of things out now. We've gotten pretty good at that.

And the key component for us has been recognizing in all of these cases, the students are more comfortable if they're not doing something alone, if they have at least one other person on their team doing whatever it is they need to do in that recitation.

And since we do something like this in every recitation, the students just come to expect it. I mean, they go for it. Sometimes, I'll come in and cover a recitation. And despite observations, I'm never exactly sure how my instructors are leaving things.

And so, I always start by asking, like, hey, I'm going to put you all in groups, is that something you're used to. Every time, they're like, yeah, we do that every day, like of course we can get into groups. So again, even acting things out, they really go for it.

We try to vary the types of activities that we do each week, to keep everyone engaged, to keep things interesting. It's hard to act ever system out. And there's occasions where an activity doesn't work.

The first time we tried MapReduce, it was chaos. But we thought it was a cool thing. So we just iterated. We just tried it again the next semester. And in planning all of these, we're still very careful to make sure that our activities actually teach them things.

Like I said, all of the recitations have technical points that they have to hit on. A MapReduce activity where they just act out the system in its normal use case, it would be fun, but they're supposed to come in knowing that. So we really want to design these activities so that we're hitting on the points we need to hit on. So there is a lot of planning that goes into this.

So now, for the final part, did it work? What got better? I'll start with a few actually quantitative things. Sometimes when I tell people about this endeavor, they want to know, are grades better? Have the students' grades improved? How are they doing on exams?

I don't personally think that that's the right question. There is one aspect of grading that has improved-- participation grades. We assign a pretty large portion of the final grade to participation in recitation. Again, we're asking a lot of them. We're expecting them to demonstrate certain skills in recitation, so they are graded accordingly. And we're seeing students earn higher participation grades now.

I'm comfortable interpreting that as they are participating more. We are also giving them a lot more opportunities to participate. Related to that, one of the things that we've gained from this experience is just a better understanding of what it means to participate in our class.

For a long time to us, it kind of meant, well, does this student answer questions when I ask them a question. But now it means, are they coming prepared to class? Are they engaged in group work? Are they listening to other students and to their instructor? Are they participating in class wide discussions?

There is more opportunities for students to participate. We are much clearer about how we assign this grade. So not only have the grades improved, but the students are noticeably more relaxed about this grade. I get fewer complaints about this grade, which is great for me. And they know that if they come prepared and ready to do the work, they'll be fine.

AUDIENCE: The grade for the recitation is separate from the course grade?

PROFESSOR: It's part of the final grade in the course. A percentage of the final grade is from participation in recitation.

So we also surveyed the students during the semester. And we were largely asking questions about do you feel comfortable in the recitations. Largely, yes, they did. In our final survey, overwhelmingly the students felt that these activities improved their engagement. So these things are all good.

I don't think they really captured though what has been so great about this endeavor for our class. And so in preparing this talk, I spoke to a couple of instructors who have been with me for a number of years. So they've seen the class through its non-active learning era, through the dark times, as well as the current era.

And so I stole a few quotes from them to kind of sum up what I think is made this approach better for the students. So let's turn to number one. The first is that I think the students actually understand the algorithms better. In particular, if we're acting something out, it gives them a visceral experience of the finer points and edge conditions. It's really easy to slide over things if one reads a paper quickly, and we're asking them to read a lot.

Second instructor, I think that it's much better when there are pairs discussing a solution or when we're doing a game. People are more likely to notice a flaw in something I've said or in someone else's answer. And they ask better questions. Plus, I think it's reassuring either to find out that a classmate is just as confused or to find out that a classmate knows a ton about the subject.

And then the last one, which I think really sums up how things have been for our students, is this one. I think the students are more engaged because they are asked to be more active participants. There's a greater sense of camaraderie between them because of their interactions.

So we're seeing these students understanding the details of the systems better, while developing this sense of camaraderie. And honestly, that has been huge for us. Largely, I would say, from my perspective as the person in charge, students are buying into this class more than they used to.

I get fewer complaints across the board. We're seeing them work better together in their team projects. We're seeing them be more engaged in lecture and in tutorial, even though recitations was really where this active learning push happened. And this whole thing has been beneficial, not just for the students, but also for the staff.

There was a little hesitance at first. Active learning can be a hard sell. It involves more work for everyone. It takes a lot of planning. It can be a nerve-wracking way to teach if you've never taught it before.

Imagine coming into a room of students you don't know very well, and say, OK, five of your are going to be on the master team, let's go, I made you all hats, right? But now, one of my instructors said, there's a modest amount of competition among the recitation instructors to work out schemes and games. So it feels like there's both a reward for applying myself, but there's also less need for perfection.

It's helped that we're encouraging active learning is an approach, because it makes it feel like less of a risk for me to try some new unconventional approach that occurs to me. So I'll tell you one of the highlights for me, as a person sort of overseeing this operation.

So this happened at the end of last semester. Again, that's the first time we're really putting this scheme into place. And near the end, my instructors-- I got this email from the instructors about one of the final recitations. And the recitation was about a system called DNS SEC, secure DNS. So it deals with security.

DNS prevents certain types of attacks on a system called DNS. That's all you need to know DNS SEC. So the first instructor described an activity they had done, where they had groups draw diagrams of their attacks on the board in parallel and then talked through each diagram, with the students explaining the problems and the approaches.

And the instructor noted that they engaged, they laughed, they collaborated, they learned more about how design figures to tell the story they wanted to tell. And I got this email, and I was so excited. Because this was not an activity that I had planned for them. This is just something they had done.

And I thought, it's happened, finally it's happened. And then I get a second email. And so a second instructor responded saying that that sounded great. And what they had done, they'd had a group of students work out how DNS would work for a particular scenario, so the original system.

And then two additional groups joined that first group, worked through how different attacks would affect that scenario. And then a fourth group acted as DNS SEC, and explain how DNS SEC would prevent such attacks. So to brand new ideas, both really good, both really different. And this is so exciting.

And then we get to instructor number three. And I've never been able to figure out a way to summarize this email. So I have to quote it for you directly.

"In my recitation, we had a marriage proposal that went bad when a romantic rival launched an attack to destroy the relationship. A second group used a different attack to induce peasants to send their taxes to a scammer. All of these attacks were then prevented by the use of DNS SEC. After each play, we did a replay with deep structure analysis that made the analogy between the play and DNS explicit. Overall, a pretty fun recitation and I was impressed with the creativity, enthusiasm, and acting talent the students displayed."

So this was amazing. This was like such-- this email thread is just such a triumph for me. I have it highlighted and saved in my email. And this has continued on since last semester. I get like one of these email threads a week now, with my instructor telling me all of the cool things that they have done.

So the last thing, which I hope has come through in this talk, it was a lot of work to get this to happen in 033. I cannot in good faith stand here tell you active learning is a breeze, you can just add it in, five minutes of prep time before the class. It's a lot of work.

But this class is so much fun now. It's fun for me to run. It's fun for instructors to teach. I don't know how many people would tell you that their 400-person class is fun to run. But I have a great time. And the amount of enjoyment that we get out of teaching 033 this way really comes through for the students.

And I think that's another component of what has helped them engage more in this class. So it has worked really well for us. I hope that you get some practical things that you can try out in your own classes. And I'm happy to take questions or to delegate questions off to Jesse and Karen.