1
00:00:05,410 --> 00:00:07,380
PROFESSOR 1: So I think
in a course like this,

2
00:00:07,380 --> 00:00:09,170
assessment is tricky.

3
00:00:09,170 --> 00:00:14,420
That these are individual
clients and projects.

4
00:00:14,420 --> 00:00:17,100
And we can try to
standardize maybe

5
00:00:17,100 --> 00:00:20,550
the design and engineering
challenge to some extent,

6
00:00:20,550 --> 00:00:23,280
but that's not
necessarily what we took

7
00:00:23,280 --> 00:00:24,940
as our primary consideration.

8
00:00:24,940 --> 00:00:27,890
We really started
with the people

9
00:00:27,890 --> 00:00:31,810
first, and tried to
match people to projects.

10
00:00:31,810 --> 00:00:34,100
And there's no doubt, there
are some projects that

11
00:00:34,100 --> 00:00:37,400
are more technically involved.

12
00:00:37,400 --> 00:00:39,650
There are some projects
where the logistics are

13
00:00:39,650 --> 00:00:42,560
more difficult--
meeting with the client

14
00:00:42,560 --> 00:00:48,190
or even working with him or
her, or even within the team.

15
00:00:48,190 --> 00:00:51,650
So it is a tricky kind of thing.

16
00:00:51,650 --> 00:00:56,860
But I think having us as
instructors, very involved

17
00:00:56,860 --> 00:00:59,990
in the class, and getting a
sense of all of the projects,

18
00:00:59,990 --> 00:01:04,160
having the mentors really
have a strong understanding

19
00:01:04,160 --> 00:01:06,330
of different
projects' progress--

20
00:01:06,330 --> 00:01:10,890
so we had one mentor
for every two teams--

21
00:01:10,890 --> 00:01:17,190
was really key to the
assessment at a high level.

22
00:01:17,190 --> 00:01:20,620
We had different pieces over
the course of the semester.

23
00:01:20,620 --> 00:01:25,910
So doing this
mid-term design review

24
00:01:25,910 --> 00:01:29,030
counts as part of
the assessment.

25
00:01:29,030 --> 00:01:31,400
Certainly the final
presentations,

26
00:01:31,400 --> 00:01:34,470
the documentation is a big part.

27
00:01:34,470 --> 00:01:39,760
But we also rely on mentor
feedback, client feedback--

28
00:01:39,760 --> 00:01:43,350
we met with all the clients
at the end of the semester.

29
00:01:43,350 --> 00:01:44,770
As well as peer feedback.

30
00:01:44,770 --> 00:01:49,240
This is a team-based
project course.

31
00:01:49,240 --> 00:01:52,540
So taking in all
these inputs is what

32
00:01:52,540 --> 00:01:55,070
we tried to do when
it came to assessment.

33
00:01:55,070 --> 00:01:58,140
To really try to understand
what students had learned

34
00:01:58,140 --> 00:02:01,580
and how they performed
in the course.

35
00:02:01,580 --> 00:02:04,340
GRACE: So for some of these
we did prepare rubric,

36
00:02:04,340 --> 00:02:07,790
or we had a vague sense of
what we were looking for.

37
00:02:07,790 --> 00:02:09,729
So for example,
for the blog posts,

38
00:02:09,729 --> 00:02:12,350
we would score it
based on not just

39
00:02:12,350 --> 00:02:14,660
how well did you
write it, and grammar,

40
00:02:14,660 --> 00:02:16,650
and how much
information is there,

41
00:02:16,650 --> 00:02:20,420
but we were looking for really
deep, personal reflection.

42
00:02:20,420 --> 00:02:24,590
And so that's what we gave
distinctiveness points for,

43
00:02:24,590 --> 00:02:25,090
I believe.

44
00:02:25,090 --> 00:02:27,060
And so, for example,
you could have

45
00:02:27,060 --> 00:02:30,610
a post just on assistive
dogs, and what types

46
00:02:30,610 --> 00:02:32,250
of assistive dogs are there.

47
00:02:32,250 --> 00:02:35,690
Which is great, it shows
that a student is learning.

48
00:02:35,690 --> 00:02:40,250
But another post-- one of
my favorite posts, actually,

49
00:02:40,250 --> 00:02:43,260
over the semester-- was
from one of my students

50
00:02:43,260 --> 00:02:45,940
who was struggling
with, should I

51
00:02:45,940 --> 00:02:49,040
go into assistive
technology as a career

52
00:02:49,040 --> 00:02:50,700
when everyone is
saying you should

53
00:02:50,700 --> 00:02:52,160
do something that's lucrative.

54
00:02:52,160 --> 00:02:53,810
But this is beneficial.

55
00:02:53,810 --> 00:02:58,520
But what if beneficial isn't
profitable, what happens then?

56
00:02:58,520 --> 00:03:00,230
And she was really
struggling with,

57
00:03:00,230 --> 00:03:02,440
am I a selfish person,
things like that.

58
00:03:02,440 --> 00:03:05,260
And so that kind of very
deep, personal growth

59
00:03:05,260 --> 00:03:10,150
is something that
I valued a lot.

60
00:03:10,150 --> 00:03:11,310
We all valued a lot.

61
00:03:14,670 --> 00:03:17,110
There's actually
very little weight

62
00:03:17,110 --> 00:03:21,050
put on the final project.

63
00:03:21,050 --> 00:03:24,150
We put a lot of weight
on how well did they

64
00:03:24,150 --> 00:03:26,390
go about the design process.

65
00:03:26,390 --> 00:03:29,980
And so, how many prototypes
did they manage to do?

66
00:03:29,980 --> 00:03:34,280
When they didn't have the
skills to do a certain project,

67
00:03:34,280 --> 00:03:36,460
were they able to go
look for resources?

68
00:03:36,460 --> 00:03:39,280
Did they go around
MIT knocking on doors,

69
00:03:39,280 --> 00:03:42,750
asking professors for help,
asking machine shop technicians

70
00:03:42,750 --> 00:03:44,470
for help?

71
00:03:44,470 --> 00:03:47,155
That was also highly valued.

72
00:03:50,680 --> 00:03:55,440
What else do you
think I'm missing out?

73
00:03:55,440 --> 00:04:02,520
We also took into account
when the clients were-- so,

74
00:04:02,520 --> 00:04:07,210
some clients would
change their mind

75
00:04:07,210 --> 00:04:10,440
about what project they
wanted the students to work on

76
00:04:10,440 --> 00:04:12,210
over the course of semester.

77
00:04:12,210 --> 00:04:14,036
Which is really difficult
for the students,

78
00:04:14,036 --> 00:04:15,910
because they would make
progress on something

79
00:04:15,910 --> 00:04:22,760
and then have to change the
project direction completely.

80
00:04:22,760 --> 00:04:25,370
And so we would try and take
that into account as well.

81
00:04:25,370 --> 00:04:28,150
Like, how were the students
able to manage that?

82
00:04:28,150 --> 00:04:30,670
And so because of these
kind of little things,

83
00:04:30,670 --> 00:04:32,410
we really didn't
put a lot of weight

84
00:04:32,410 --> 00:04:35,070
on how the project turned
out at the end of the day.

85
00:04:35,070 --> 00:04:36,530
It was more about
how much effort

86
00:04:36,530 --> 00:04:39,070
did you put into it, how much
did you grow as a person,

87
00:04:39,070 --> 00:04:42,150
how well did you manage
your clients' expectations,

88
00:04:42,150 --> 00:04:47,910
and were you able to
satisfy the client given

89
00:04:47,910 --> 00:04:49,330
all the constraints.

90
00:04:49,330 --> 00:04:52,300
So very, very organic.

91
00:04:52,300 --> 00:04:54,030
Which is why we
had to spread out

92
00:04:54,030 --> 00:04:57,199
the grading rubric across
lots of little components.

93
00:04:57,199 --> 00:04:58,740
And that's why we
had to get feedback

94
00:04:58,740 --> 00:05:01,710
from mentors, and
peers, and clients,

95
00:05:01,710 --> 00:05:05,030
and have mid-semester reviews
and final semester reviews.

96
00:05:08,210 --> 00:05:09,224
I hope I explained that.

97
00:05:09,224 --> 00:05:10,640
PROFESSOR 1: Yeah,
I think there's

98
00:05:10,640 --> 00:05:15,570
a lot of aspects to this.

99
00:05:15,570 --> 00:05:19,740
One is, we really did try
to structure the evaluation

100
00:05:19,740 --> 00:05:25,050
as much as possible in
terms of being able to try

101
00:05:25,050 --> 00:05:26,850
to grade objectively.

102
00:05:26,850 --> 00:05:30,990
And so we did have a decent
number of structured rubrics,

103
00:05:30,990 --> 00:05:33,500
that we're happy
to share, in terms

104
00:05:33,500 --> 00:05:38,780
of how we evaluated different
components of the project.

105
00:05:38,780 --> 00:05:42,860
So everything from the course,
everything from the blog posts,

106
00:05:42,860 --> 00:05:46,710
to the videos, to the
mid-semester and final semester

107
00:05:46,710 --> 00:05:54,660
panels, the documentation,
all aspects of the class.

108
00:05:54,660 --> 00:06:04,610
I think for the
individual components,

109
00:06:04,610 --> 00:06:07,540
as Grace says, really
trying to personalize

110
00:06:07,540 --> 00:06:10,930
the learning a little bit in
terms of having reflections

111
00:06:10,930 --> 00:06:14,160
or blog posts where people
could write about topics

112
00:06:14,160 --> 00:06:16,010
that interested
them, as it related

113
00:06:16,010 --> 00:06:17,751
to assistive technology.

114
00:06:17,751 --> 00:06:20,250
I think that's something that
we tried this year, as opposed

115
00:06:20,250 --> 00:06:26,280
to maybe more formed lab reports
on some of the labs that we did

116
00:06:26,280 --> 00:06:30,520
or something like that.

117
00:06:30,520 --> 00:06:33,620
The other part I think
we tried this year

118
00:06:33,620 --> 00:06:36,670
was really to take in,
as Grace was saying,

119
00:06:36,670 --> 00:06:42,740
all of these human
inputs or feedback.

120
00:06:42,740 --> 00:06:48,330
So we met weekly as a staff,
and reflected on the projects

121
00:06:48,330 --> 00:06:51,980
and the students' progress.

122
00:06:51,980 --> 00:06:57,780
We talked to clients and
had students themselves talk

123
00:06:57,780 --> 00:07:01,450
a little bit about their peers.

124
00:07:01,450 --> 00:07:07,240
And so that's, I think, what
we tried to take into account.

125
00:07:07,240 --> 00:07:09,020
To some extent we tried
to start from what

126
00:07:09,020 --> 00:07:15,720
would be ideal in terms
of understanding students'

127
00:07:15,720 --> 00:07:17,960
experiences in the class.

128
00:07:17,960 --> 00:07:21,110
And that, I think,
makes it a little bit

129
00:07:21,110 --> 00:07:25,560
tricky to evaluate or grade.

130
00:07:25,560 --> 00:07:32,960
But hopefully it was
one reasonable way,

131
00:07:32,960 --> 00:07:38,360
or one valuable way to do it.