1
00:00:07,840 --> 00:00:08,460
OK.

2
00:00:08,460 --> 00:00:09,050
Good.

3
00:00:09,050 --> 00:00:15,540
The final class in linear
algebra at MIT this Fall

4
00:00:15,540 --> 00:00:18,430
is to review the whole course.

5
00:00:18,430 --> 00:00:23,890
And, you know the best
way I know how to review

6
00:00:23,890 --> 00:00:30,410
is to take old exams and just
think through the problems.

7
00:00:30,410 --> 00:00:33,680
So it will be a three-hour
exam next Thursday.

8
00:00:37,380 --> 00:00:41,240
Nobody will be able to take an
exam before Thursday, anybody

9
00:00:41,240 --> 00:00:44,120
who needs to take it
in some different way

10
00:00:44,120 --> 00:00:47,380
after Thursday should
see me next Monday.

11
00:00:47,380 --> 00:00:49,310
I'll be in my office Monday.

12
00:00:49,310 --> 00:00:52,190
OK.

13
00:00:52,190 --> 00:00:54,330
May I just read
out some problems

14
00:00:54,330 --> 00:01:06,400
and, let me bring the board
down, and let's start.

15
00:01:06,400 --> 00:01:08,270
OK.

16
00:01:08,270 --> 00:01:12,140
Here's a question.

17
00:01:12,140 --> 00:01:18,100
This is about a 3-by-n matrix.

18
00:01:18,100 --> 00:01:21,900
And we're given --
so we're given --

19
00:01:21,900 --> 00:01:30,160
given -- A x equals 1
0 0 has no solution.

20
00:01:33,380 --> 00:01:45,010
And we're also given A x equals
0 1 0 has exactly one solution.

21
00:01:45,010 --> 00:01:45,510
OK.

22
00:01:48,090 --> 00:01:51,970
So you can probably
anticipate my first question,

23
00:01:51,970 --> 00:01:55,170
what can you tell me about m?

24
00:01:55,170 --> 00:01:59,190
It's an m-by-n matrix
of rank r, as always,

25
00:01:59,190 --> 00:02:02,910
what can you tell me
about those three numbers?

26
00:02:02,910 --> 00:02:12,160
So what can you tell me about
m, the number of rows, n,

27
00:02:12,160 --> 00:02:16,931
the number of columns,
and r, the rank?

28
00:02:16,931 --> 00:02:17,430
OK.

29
00:02:20,100 --> 00:02:23,430
See, do you want to
tell me first what m is?

30
00:02:23,430 --> 00:02:25,275
How many rows in this matrix?

31
00:02:28,610 --> 00:02:32,940
Must be three, right?

32
00:02:32,940 --> 00:02:40,610
We can't tell what n
is, but we can certainly

33
00:02:40,610 --> 00:02:43,030
tell that m is three.

34
00:02:43,030 --> 00:02:43,530
OK.

35
00:02:43,530 --> 00:02:46,830
And, what do these
things tell us?

36
00:02:46,830 --> 00:02:49,750
Let's take them one at a time.

37
00:02:49,750 --> 00:02:53,740
When I discover that some
equation has no solution,

38
00:02:53,740 --> 00:02:56,880
that there's some right-hand
side with no answer,

39
00:02:56,880 --> 00:03:01,950
what does that tell me about
the rank of the matrix?

40
00:03:06,430 --> 00:03:11,850
It's smaller m.

41
00:03:11,850 --> 00:03:13,820
Is that right?

42
00:03:13,820 --> 00:03:19,840
If there is no
solution, that tells me

43
00:03:19,840 --> 00:03:26,060
that some rows of the matrix
are combinations of other rows.

44
00:03:26,060 --> 00:03:30,850
Because if I had a
pivot in every row,

45
00:03:30,850 --> 00:03:34,440
then I would certainly be
able to solve the system.

46
00:03:34,440 --> 00:03:38,410
I would have particular
solutions and all the good

47
00:03:38,410 --> 00:03:43,170
So any time that there's a
system with no solutions,

48
00:03:43,170 --> 00:03:46,260
stuff. that tells me
that r must be below m.

49
00:03:46,260 --> 00:03:50,450
What about the fact that if,
when there is a solution,

50
00:03:50,450 --> 00:03:52,330
there's only one?

51
00:03:52,330 --> 00:03:53,320
What does that tell me?

52
00:03:55,830 --> 00:03:58,570
Well, normally there
would be one solution,

53
00:03:58,570 --> 00:04:03,070
and then we could add in
anything in the null space.

54
00:04:03,070 --> 00:04:07,110
So this is telling me the null
space only has the 0 vector in

55
00:04:07,110 --> 00:04:07,790
it.

56
00:04:07,790 --> 00:04:10,550
There's just one
solution, period,

57
00:04:10,550 --> 00:04:11,820
so what does that tell me?

58
00:04:11,820 --> 00:04:15,000
The null space has only
the zero vector in it?

59
00:04:15,000 --> 00:04:18,959
What does that tell me about
the relation of r to n?

60
00:04:18,959 --> 00:04:22,640
So this one solution
only, that means

61
00:04:22,640 --> 00:04:29,120
the null space of the matrix
must be just the zero vector,

62
00:04:29,120 --> 00:04:33,720
and what does that
tell me about r and n?

63
00:04:33,720 --> 00:04:34,410
They're equal.

64
00:04:34,410 --> 00:04:36,020
The columns are independent.

65
00:04:36,020 --> 00:04:39,730
So I've got, now, r equals
n, and r less than m,

66
00:04:39,730 --> 00:04:42,760
and now I also know m is three.

67
00:04:42,760 --> 00:04:45,580
So those are really
the facts I know.

68
00:04:45,580 --> 00:04:51,030
n=r and those numbers
are smaller than three.

69
00:04:51,030 --> 00:04:52,820
Sorry, yes, yes.

70
00:04:52,820 --> 00:04:58,030
r is smaller than m, and
n, of course, is also.

71
00:04:58,030 --> 00:05:01,830
So I guess this summarizes
what we can tell.

72
00:05:01,830 --> 00:05:05,870
In fact, why not
give me a matrix --

73
00:05:05,870 --> 00:05:09,380
because I would often ask for
an example of such a matrix --

74
00:05:09,380 --> 00:05:13,150
can you give me a matrix
A that's an example?

75
00:05:13,150 --> 00:05:16,670
That shows this possibility?

76
00:05:16,670 --> 00:05:20,780
Exactly, that
there's no solution

77
00:05:20,780 --> 00:05:26,310
with that right-hand side, but
there's exactly one solution

78
00:05:26,310 --> 00:05:28,420
with this right-hand side.

79
00:05:28,420 --> 00:05:33,400
Anybody want to suggest
a matrix that does that?

80
00:05:33,400 --> 00:05:34,560
Let's see.

81
00:05:34,560 --> 00:05:39,610
What do I -- what vector do
I want in the column space?

82
00:05:39,610 --> 00:05:42,120
I want zero, one, zero,
to be in the column space,

83
00:05:42,120 --> 00:05:45,180
because I'm able
to solve for that.

84
00:05:45,180 --> 00:05:47,930
So let's put zero, one,
zero in the column space.

85
00:05:47,930 --> 00:05:51,290
Actually, I could
stop right there.

86
00:05:51,290 --> 00:05:56,480
That would be a matrix with
m equal three, three rows,

87
00:05:56,480 --> 00:06:04,160
and n and r are both one,
rank one, one column,

88
00:06:04,160 --> 00:06:08,170
and, of course, there's
no solution to that one.

89
00:06:08,170 --> 00:06:10,290
So that's perfectly
good as it is.

90
00:06:10,290 --> 00:06:12,990
Or if you, kind of,
have a prejudice

91
00:06:12,990 --> 00:06:15,580
against matrices that
only have one column,

92
00:06:15,580 --> 00:06:17,620
I'll accept a second

93
00:06:17,620 --> 00:06:18,310
column.

94
00:06:18,310 --> 00:06:20,970
So what could I include
as a second column

95
00:06:20,970 --> 00:06:26,020
that would just be a different
answer but equally good?

96
00:06:26,020 --> 00:06:31,420
I could put this vector in the
column space, too, if I wanted.

97
00:06:31,420 --> 00:06:39,180
That would now be a case
with r=n=2, but, of course,

98
00:06:39,180 --> 00:06:43,850
three m eq- m is still
three, and this vector is not

99
00:06:43,850 --> 00:06:45,540
in the column space.

100
00:06:45,540 --> 00:06:49,470
So you're -- this is just like
prompting us to remember all

101
00:06:49,470 --> 00:06:53,660
those things, column space,
null space, all that stuff.

102
00:06:53,660 --> 00:06:56,250
Now, I probably asked
a second question

103
00:06:56,250 --> 00:06:58,200
about this type of thing.

104
00:07:01,370 --> 00:07:01,940
OK.

105
00:07:01,940 --> 00:07:04,260
Oh, I even asked, write
down an example of a

106
00:07:04,260 --> 00:07:07,080
Ah. matrix that fits
the description.

107
00:07:07,080 --> 00:07:07,580
Hm.

108
00:07:07,580 --> 00:07:13,520
I guess I haven't learned
anything in twenty-six years.

109
00:07:13,520 --> 00:07:15,340
CK.

110
00:07:15,340 --> 00:07:19,910
Cross out all statements that
are false about any matrix with

111
00:07:19,910 --> 00:07:23,480
these -- so again, these are
-- this is the preliminary sta-

112
00:07:23,480 --> 00:07:27,600
these are the facts about my
matrix, this is one example.

113
00:07:27,600 --> 00:07:29,430
But, of course, by
having an example,

114
00:07:29,430 --> 00:07:33,290
it will be easy to check some
of these facts, or non-facts.

115
00:07:33,290 --> 00:07:40,400
Let me, let me write
down some, facts.

116
00:07:40,400 --> 00:07:42,980
Some possible facts.

117
00:07:42,980 --> 00:07:45,580
So this is really true or false.

118
00:07:45,580 --> 00:07:49,510
The determinant --
this is part one,

119
00:07:49,510 --> 00:07:55,300
the determinant of A transpose
A is the same as the determinant

120
00:07:55,300 --> 00:07:57,670
of A A transpose.

121
00:07:57,670 --> 00:07:59,820
Is that true or not?

122
00:07:59,820 --> 00:08:05,230
Second one, A transpose
A, is invertible.

123
00:08:05,230 --> 00:08:06,150
Is invertible.

124
00:08:09,810 --> 00:08:17,270
Third possible fact, A A
transpose is positive definite.

125
00:08:22,070 --> 00:08:24,090
So you see how, on
an exam question,

126
00:08:24,090 --> 00:08:28,490
I try to connect the
different parts of the course.

127
00:08:28,490 --> 00:08:33,360
So, well, I mean,
the simplest way

128
00:08:33,360 --> 00:08:38,260
would be to try it with that
matrix as a good example,

129
00:08:38,260 --> 00:08:43,730
but maybe we can
answer, even directly.

130
00:08:43,730 --> 00:08:47,100
Let me take number two first.

131
00:08:47,100 --> 00:08:51,910
Because I'm -- you know, I'm
very, very fond of that matrix,

132
00:08:51,910 --> 00:08:55,460
A transpose A.

133
00:08:55,460 --> 00:09:01,170
And when is it invertible?

134
00:09:01,170 --> 00:09:03,335
When is the matrix A
transpose A, invertible?

135
00:09:08,130 --> 00:09:12,500
The great thing is that I
can tell from the rank of A

136
00:09:12,500 --> 00:09:15,790
that I don't have to
multiply out A transpose A.

137
00:09:15,790 --> 00:09:18,990
A transpose A, is invertible --

138
00:09:18,990 --> 00:09:24,550
well, if A has a null space
other than the zero vector,

139
00:09:24,550 --> 00:09:27,410
then it -- it's -- no way
it's going to be invertible.

140
00:09:27,410 --> 00:09:31,440
But the beauty is, if the null
space of A is just the zero

141
00:09:31,440 --> 00:09:34,480
vector, so the fact
-- the key fact is,

142
00:09:34,480 --> 00:09:40,030
this is invertible if
r=n, by which I mean,

143
00:09:40,030 --> 00:09:46,290
independent columns of A.

144
00:09:46,290 --> 00:09:47,230
In A.

145
00:09:47,230 --> 00:09:49,340
In the matrix A.

146
00:09:49,340 --> 00:09:53,320
If r=n -- if the matrix A
has independent columns,

147
00:09:53,320 --> 00:09:56,200
then this combination,
A transpose A,

148
00:09:56,200 --> 00:10:00,000
is square and still
that same null space,

149
00:10:00,000 --> 00:10:03,200
only the zero vector,
independent columns all good,

150
00:10:03,200 --> 00:10:07,320
and so, what's the true/false?

151
00:10:07,320 --> 00:10:14,310
Is it -- is this middle one T
or F for this, in this setup?

152
00:10:14,310 --> 00:10:17,650
Well, we discovered that --

153
00:10:17,650 --> 00:10:22,560
we discovered that -- that r
was n, from that second fact.

154
00:10:22,560 --> 00:10:24,610
So this is a true.

155
00:10:24,610 --> 00:10:26,190
That's a true.

156
00:10:26,190 --> 00:10:29,330
And, of course, A transpose
A, in this example,

157
00:10:29,330 --> 00:10:32,190
would probably be -- what
would A transpose A, be,

158
00:10:32,190 --> 00:10:32,870
for that matrix?

159
00:10:35,610 --> 00:10:38,000
Can you multiply A
transpose A, and see what

160
00:10:38,000 --> 00:10:39,380
it looks like for that matrix?

161
00:10:39,380 --> 00:10:40,600
What shape would it be?

162
00:10:43,200 --> 00:10:44,990
It will be two by two.

163
00:10:44,990 --> 00:10:47,810
And what matrix will it be?

164
00:10:47,810 --> 00:10:48,660
The identity.

165
00:10:48,660 --> 00:10:51,080
So, it checks out.

166
00:10:51,080 --> 00:10:52,925
OK, what about A A transpose?

167
00:10:55,510 --> 00:10:58,870
Well, depending
on the shape of A,

168
00:10:58,870 --> 00:11:02,770
it could be good or not so good.

169
00:11:02,770 --> 00:11:04,980
It's always symmetric,
it's always square,

170
00:11:04,980 --> 00:11:06,400
but what's the size, now?

171
00:11:06,400 --> 00:11:11,220
This is three by n,
and this is n by three,

172
00:11:11,220 --> 00:11:13,270
so the result is three by three.

173
00:11:17,550 --> 00:11:18,665
Is it positive definite?

174
00:11:21,760 --> 00:11:23,160
I don't think so.

175
00:11:23,160 --> 00:11:24,940
False.

176
00:11:24,940 --> 00:11:29,870
If I multiply that by A
transpose, A A transpose,

177
00:11:29,870 --> 00:11:31,480
what would the rank be?

178
00:11:31,480 --> 00:11:36,620
It would be the same as
the rank of A, that's --

179
00:11:36,620 --> 00:11:39,430
it would be just rank two.

180
00:11:39,430 --> 00:11:42,200
And if it's three-by-three,
and it's only rank two,

181
00:11:42,200 --> 00:11:44,270
it's certainly not
positive definite.

182
00:11:44,270 --> 00:11:47,410
So what could I say
about A A transpose,

183
00:11:47,410 --> 00:11:52,220
if I wanted to, like, say
something true about it?

184
00:11:52,220 --> 00:11:56,360
It's true that it is
positive semi-definite.

185
00:11:56,360 --> 00:12:01,550
If I made this semi-definite,
it would always be true, always.

186
00:12:01,550 --> 00:12:03,950
But if I'm looking
for positive definite,

187
00:12:03,950 --> 00:12:08,990
then I'm looking at the null
space of whatever's here,

188
00:12:08,990 --> 00:12:14,720
and, in this case,
it's got a null space.

189
00:12:14,720 --> 00:12:17,200
So A, A -- eh, shall
we just figure it out,

190
00:12:17,200 --> 00:12:17,720
here?

191
00:12:17,720 --> 00:12:22,090
A A transpose, for that
matrix, will be three-by-three.

192
00:12:22,090 --> 00:12:25,340
If I multiplied
A by A transpose,

193
00:12:25,340 --> 00:12:26,600
what would the first row be?

194
00:12:29,990 --> 00:12:31,960
All zeroes, right?

195
00:12:31,960 --> 00:12:34,970
First row of A A
transpose, could only

196
00:12:34,970 --> 00:12:40,000
be all zeroes, so it's probably
a one there and a one there,

197
00:12:40,000 --> 00:12:41,950
or something like that.

198
00:12:41,950 --> 00:12:45,860
But, I don't even
know if that's right.

199
00:12:45,860 --> 00:12:48,570
But it's all zeroes
there, so it's certainly

200
00:12:48,570 --> 00:12:49,700
not positive definite.

201
00:12:49,700 --> 00:12:53,550
Let me not put anything up
I'm not sh- don't check.

202
00:12:53,550 --> 00:12:54,930
What about this determinant?

203
00:12:54,930 --> 00:12:58,290
Oh, well, I guess --

204
00:12:58,290 --> 00:13:01,470
that's a sort of
tricky question.

205
00:13:01,470 --> 00:13:04,620
Is it true or
false in this case?

206
00:13:04,620 --> 00:13:08,810
It's false, apparently, because
A transpose A, is invertible,

207
00:13:08,810 --> 00:13:13,500
we just got a true for this
one, and we got a false,

208
00:13:13,500 --> 00:13:15,960
we got a z- we got
a non-invertible one

209
00:13:15,960 --> 00:13:16,530
for this one.

210
00:13:16,530 --> 00:13:22,060
So actually, this one
is false, number one.

211
00:13:22,060 --> 00:13:24,590
That surprises us,
actually, because it's,

212
00:13:24,590 --> 00:13:25,930
I mean, why was it tricky?

213
00:13:25,930 --> 00:13:30,120
Because what is true
about determinants?

214
00:13:30,120 --> 00:13:34,110
This would be true if
those matrices were square.

215
00:13:34,110 --> 00:13:39,530
If I have two square matrices,
A and any other matrix B,

216
00:13:39,530 --> 00:13:43,670
could be A transpose, could
be somebody else's matrix.

217
00:13:43,670 --> 00:13:47,090
Then it would be true that
the determinant of B A

218
00:13:47,090 --> 00:13:49,610
would equal the
determinant of A B.

219
00:13:49,610 --> 00:13:54,420
But if the matrices are not
square and it would actually be

220
00:13:54,420 --> 00:13:59,010
true that it would be equal
-- that this would equal

221
00:13:59,010 --> 00:14:03,940
the determinant of A times the
determinant of A transpose.

222
00:14:03,940 --> 00:14:06,610
We could even split up those
two separate determinants.

223
00:14:06,610 --> 00:14:09,080
And, of course,
those would be equal.

224
00:14:09,080 --> 00:14:11,920
But only when A is square.

225
00:14:11,920 --> 00:14:15,860
So that's just, that's a
question that rests on the,

226
00:14:15,860 --> 00:14:18,970
the falseness rests on the
fact that the matrix isn't

227
00:14:18,970 --> 00:14:20,530
square in the first place.

228
00:14:20,530 --> 00:14:22,700
OK, good.

229
00:14:22,700 --> 00:14:25,470
Let's see.

230
00:14:25,470 --> 00:14:28,860
Oh, now, even asks more.

231
00:14:28,860 --> 00:14:34,370
Prove that A transpose
y equals c --

232
00:14:34,370 --> 00:14:37,760
hah-God, it's -- this
question goes on and on.

233
00:14:40,830 --> 00:14:44,130
now I ask you about
A transpose y=c.

234
00:14:46,780 --> 00:14:48,650
So I'm asking you
about the equation --

235
00:14:48,650 --> 00:14:51,100
about the matrix A transpose.

236
00:14:51,100 --> 00:14:57,890
And I want you to prove that
it has at least one solution --

237
00:14:57,890 --> 00:15:04,160
one solution for every c,
every right-hand side c,

238
00:15:04,160 --> 00:15:13,640
and, in fact -- in fact,
infinitely many solutions

239
00:15:13,640 --> 00:15:16,630
for every c.

240
00:15:16,630 --> 00:15:17,910
OK.

241
00:15:17,910 --> 00:15:20,170
Well, none -- none
of this is difficult,

242
00:15:20,170 --> 00:15:25,470
but, it's been a little while.

243
00:15:25,470 --> 00:15:27,900
So we just have to think again.

244
00:15:27,900 --> 00:15:30,980
When I have a system of
equations -- this is --

245
00:15:30,980 --> 00:15:37,690
this matrix A transpose is now,
instead of being three by n,

246
00:15:37,690 --> 00:15:40,970
it's n by three, it's n by m.

247
00:15:40,970 --> 00:15:43,470
Of course.

248
00:15:43,470 --> 00:15:53,900
To show that a system has
at least one solution,

249
00:15:53,900 --> 00:15:56,020
when does this, when
does this system --

250
00:15:56,020 --> 00:15:57,760
when is the system
always solvable?

251
00:16:01,040 --> 00:16:06,810
When it has full row rank,
when the rows are independent.

252
00:16:06,810 --> 00:16:12,100
Here, we have n rows,
and that's the rank.

253
00:16:12,100 --> 00:16:19,450
So at least one solution,
because the number

254
00:16:19,450 --> 00:16:22,500
of rows, which is n,
for the transpose,

255
00:16:22,500 --> 00:16:24,300
is equal to r, the rank.

256
00:16:27,490 --> 00:16:30,860
This A transpose
had independent rows

257
00:16:30,860 --> 00:16:34,530
because A had independent
columns, right?

258
00:16:34,530 --> 00:16:41,480
The original A had independent
columns, when we transpose it,

259
00:16:41,480 --> 00:16:44,430
it has independent rows, so
there's at least one solution.

260
00:16:44,430 --> 00:16:46,660
But now, how do I even know
that there are infinitely

261
00:16:46,660 --> 00:16:48,050
many solutions?

262
00:16:48,050 --> 00:16:49,260
Oh, what do I --

263
00:16:49,260 --> 00:16:52,280
I want to know something
about the null space.

264
00:16:52,280 --> 00:16:57,390
What's the dimension of the
null space of A transpose?

265
00:16:57,390 --> 00:17:00,100
So the answer has got
to be the dimension

266
00:17:00,100 --> 00:17:03,850
of the null space of
A transpose, what's

267
00:17:03,850 --> 00:17:06,119
the general fact?

268
00:17:06,119 --> 00:17:10,790
If A is an m by n
matrix of rank r,

269
00:17:10,790 --> 00:17:14,260
what's the dimension
of A transpose?

270
00:17:14,260 --> 00:17:15,970
The null space of A transpose?

271
00:17:15,970 --> 00:17:20,470
Do you remember that
little fourth subspace

272
00:17:20,470 --> 00:17:24,050
that's tagging along
down in our big picture?

273
00:17:24,050 --> 00:17:26,497
It's dimension was m-r.

274
00:17:30,780 --> 00:17:32,630
And, that's bigger than zero.

275
00:17:32,630 --> 00:17:35,610
m is bigger than r.

276
00:17:35,610 --> 00:17:37,520
So there's a lot
in that null space.

277
00:17:44,520 --> 00:17:47,790
So there's always one
solution because n i- this

278
00:17:47,790 --> 00:17:49,230
is speaking about A transpose.

279
00:17:52,880 --> 00:17:55,730
So for A transpose, the roles
of m and n are reversed,

280
00:17:55,730 --> 00:17:57,540
of course, so I'm --

281
00:17:57,540 --> 00:18:01,430
keep in mind that this
board was about A transpose,

282
00:18:01,430 --> 00:18:04,880
so the roles -- so it's the
null space of a transpose,

283
00:18:04,880 --> 00:18:08,590
and there are m-r
free variables.

284
00:18:08,590 --> 00:18:13,630
OK, that's, like,
just some, review.

285
00:18:13,630 --> 00:18:17,030
Can I take another problem
that's also sort of --

286
00:18:17,030 --> 00:18:24,710
suppose the matrix A has
three columns, v1, v2, v3.

287
00:18:24,710 --> 00:18:28,880
Those are the columns
of the matrix.

288
00:18:28,880 --> 00:18:31,140
All right.

289
00:18:31,140 --> 00:18:33,500
Question A.

290
00:18:33,500 --> 00:18:36,827
Solve Ax=v1-v2+v3.

291
00:18:44,260 --> 00:18:45,670
Tell me what x is.

292
00:18:53,960 --> 00:18:57,520
Well, there, you're
seeing the most --

293
00:18:57,520 --> 00:19:04,380
the one absolutely
essential fact about matrix

294
00:19:04,380 --> 00:19:06,530
multiplication,
how does it work,

295
00:19:06,530 --> 00:19:10,760
when we do it a column at a
time, the very, very first day,

296
00:19:10,760 --> 00:19:13,800
way back in September, we
did multiplication a column

297
00:19:13,800 --> 00:19:14,590
at a time.

298
00:19:14,590 --> 00:19:16,190
So what's x?

299
00:19:16,190 --> 00:19:18,070
Just tell me?

300
00:19:18,070 --> 00:19:19,250
One minus one, one.

301
00:19:19,250 --> 00:19:19,820
Thanks.

302
00:19:19,820 --> 00:19:20,740
OK.

303
00:19:20,740 --> 00:19:22,950
Everybody's got that.

304
00:19:22,950 --> 00:19:23,550
OK?

305
00:19:23,550 --> 00:19:26,750
Then the next question is,
suppose that combination is

306
00:19:26,750 --> 00:19:28,130
zero --

307
00:19:28,130 --> 00:19:31,560
oh, yes, OK, so
question (b) says --

308
00:19:31,560 --> 00:19:42,730
part (b) says, suppose
this thing is zero.

309
00:19:42,730 --> 00:19:45,380
Suppose that's zero.

310
00:19:45,380 --> 00:19:48,520
Then the solution is not unique.

311
00:19:48,520 --> 00:19:51,730
Suppose I want true or false.

312
00:19:51,730 --> 00:19:54,440
-- and a reason.

313
00:19:54,440 --> 00:20:00,330
Suppose this
combination is zero.

314
00:20:00,330 --> 00:20:02,170
v1-v2+v3.

315
00:20:02,170 --> 00:20:06,640
Show that -- what
does that tell me?

316
00:20:06,640 --> 00:20:08,210
So it's a separate
question, maybe

317
00:20:08,210 --> 00:20:11,880
I sort of saved time
by writing it that way,

318
00:20:11,880 --> 00:20:14,620
but it's a totally
separate question.

319
00:20:14,620 --> 00:20:19,280
If I have a matrix, and I know
that column one minus column

320
00:20:19,280 --> 00:20:23,160
two plus column
three is zero, what

321
00:20:23,160 --> 00:20:33,810
does that tell me about whether
the solution is unique or not?

322
00:20:33,810 --> 00:20:36,420
Is there more than one solution?

323
00:20:36,420 --> 00:20:40,330
What's uniqueness about?

324
00:20:40,330 --> 00:20:42,420
Uniqueness is about,
is there anything

325
00:20:42,420 --> 00:20:44,720
in the null space, right?

326
00:20:44,720 --> 00:20:46,490
The solution is
unique when there's

327
00:20:46,490 --> 00:20:49,550
nobody in the null space
except the zero vector.

328
00:20:49,550 --> 00:20:57,670
And, if that's zero, then this
guy would be in the null space.

329
00:20:57,670 --> 00:21:08,020
So if this were zero, then this
x is in the null space of A.

330
00:21:08,020 --> 00:21:18,750
So solutions are never
unique, because I could always

331
00:21:18,750 --> 00:21:26,890
add that to any solution,
and Ax wouldn't change.

332
00:21:26,890 --> 00:21:29,610
So it's always that question.

333
00:21:29,610 --> 00:21:31,760
Is there somebody
in the null space?

334
00:21:31,760 --> 00:21:33,620
OK.

335
00:21:33,620 --> 00:21:37,370
Oh, now, here's a totally
different question.

336
00:21:37,370 --> 00:21:43,770
Suppose those three vectors,
v1, v2, v3, are orthonormal.

337
00:21:43,770 --> 00:21:48,340
So this isn't going to happen
for orthonormal vectors.

338
00:21:48,340 --> 00:21:51,680
OK, so part (c),
forget part (b).

339
00:21:51,680 --> 00:21:52,410
c.

340
00:21:52,410 --> 00:22:05,040
If v1, v2, v3,
are orthonormal --

341
00:22:05,040 --> 00:22:08,725
so that I would usually
have called them q1, q2, q3.

342
00:22:11,540 --> 00:22:16,420
Now, what combination --
oh, here's a nice question,

343
00:22:16,420 --> 00:22:19,020
if I say so myself --

344
00:22:19,020 --> 00:22:24,280
what combination of v1
and v2 is closest to v3?

345
00:22:24,280 --> 00:22:28,120
What point on the
plane of v1 and v2

346
00:22:28,120 --> 00:22:33,100
is the closest point to v3 if
these vectors are orthonormal?

347
00:22:33,100 --> 00:22:33,750
So let me --

348
00:22:33,750 --> 00:22:41,610
I'll start the sentence -- then
the combination something times

349
00:22:41,610 --> 00:22:51,570
v1 plus something times v2 is
the closest combination to v3?

350
00:22:51,570 --> 00:22:53,860
And what's the answer?

351
00:22:53,860 --> 00:22:57,150
What's the closest vector
on that plane to v3?

352
00:22:57,150 --> 00:23:00,030
Zeroes.

353
00:23:00,030 --> 00:23:01,480
Right.

354
00:23:01,480 --> 00:23:07,240
We just imagine the x, y,
z axes, the v1, v2, th- v3

355
00:23:07,240 --> 00:23:13,380
could be the standard
basis, the x, y, z vectors,

356
00:23:13,380 --> 00:23:19,090
and, of course, the point on
the xy plane that's closest

357
00:23:19,090 --> 00:23:24,090
to v3 on the z axis is zero.

358
00:23:24,090 --> 00:23:28,880
So if we're orthonormal,
then the projection

359
00:23:28,880 --> 00:23:34,690
of v3 onto that plane
is perpendicular,

360
00:23:34,690 --> 00:23:36,440
it hits right at zero.

361
00:23:36,440 --> 00:23:39,850
OK, so that's like a quick --

362
00:23:39,850 --> 00:23:43,890
you know, an easy question,
but still brings it out.

363
00:23:43,890 --> 00:23:45,110
OK.

364
00:23:45,110 --> 00:23:56,720
Let me see what, shall I
write down a Markov matrix,

365
00:23:56,720 --> 00:24:01,870
and I'll ask you
for its eigenvalues.

366
00:24:01,870 --> 00:24:02,680
OK.

367
00:24:02,680 --> 00:24:06,070
Here's a Markov matrix --

368
00:24:06,070 --> 00:24:14,210
this -- and, tell
me its eigenvalues.

369
00:24:14,210 --> 00:24:16,020
So here -- I'll
call the matrix A,

370
00:24:16,020 --> 00:24:19,990
and I'll call this as point
two, point four, point four,

371
00:24:19,990 --> 00:24:25,480
point four, point four, point
two, point four, point three,

372
00:24:25,480 --> 00:24:28,391
point three, point four.

373
00:24:28,391 --> 00:24:28,890
OK.

374
00:24:33,340 --> 00:24:39,020
Let's see -- it helps out to
notice that column one plus

375
00:24:39,020 --> 00:24:41,840
column two --

376
00:24:41,840 --> 00:24:46,870
what's interesting about
column one plus column two?

377
00:24:46,870 --> 00:24:50,530
It's twice as much
as column three.

378
00:24:50,530 --> 00:24:53,790
So column one plus column two
equals two times column three.

379
00:24:53,790 --> 00:24:57,300
I put that in there,
column one plus column two

380
00:24:57,300 --> 00:24:59,730
equals twice column three.

381
00:24:59,730 --> 00:25:01,430
That's observation.

382
00:25:01,430 --> 00:25:02,230
OK.

383
00:25:02,230 --> 00:25:04,500
Tell me the eigenvalues
of the matrix.

384
00:25:07,080 --> 00:25:08,455
OK, tell me one eigenvalue?

385
00:25:11,770 --> 00:25:15,230
Because the matrix is singular.

386
00:25:15,230 --> 00:25:17,570
Tell me another eigenvalue?

387
00:25:17,570 --> 00:25:20,320
One, because it's
a Markov matrix,

388
00:25:20,320 --> 00:25:25,120
the columns add to
the all ones vector,

389
00:25:25,120 --> 00:25:31,940
and that will be an
eigenvector of A transpose.

390
00:25:31,940 --> 00:25:33,435
And tell me the
third eigenvalue?

391
00:25:36,740 --> 00:25:38,770
Let's see, to make
the trace come out

392
00:25:38,770 --> 00:25:42,570
right, which is point eight,
we need minus point two.

393
00:25:45,200 --> 00:25:46,270
OK.

394
00:25:46,270 --> 00:25:52,750
And now, suppose I start
the Markov process.

395
00:25:52,750 --> 00:25:56,460
Suppose I start with u(0) --

396
00:25:56,460 --> 00:26:01,490
so I'm going to look at the
powers of A applied to u(0).

397
00:26:01,490 --> 00:26:04,840
This is uk.

398
00:26:04,840 --> 00:26:09,930
And there's my matrix, and
I'm going to let u(0) be --

399
00:26:09,930 --> 00:26:14,860
this is going to
be zero, ten, zero.

400
00:26:14,860 --> 00:26:21,060
And my question is,
what does that approach?

401
00:26:21,060 --> 00:26:24,910
If u(0) is equal to
this -- there is u(0).

402
00:26:24,910 --> 00:26:26,420
Shall I write it in?

403
00:26:26,420 --> 00:26:27,800
Maybe I'll just write in u(0).

404
00:26:30,680 --> 00:26:40,430
A to the k, starting with
ten people in state two,

405
00:26:40,430 --> 00:26:48,210
and every step follows
the Markov rule,

406
00:26:48,210 --> 00:26:52,920
what does the solution
look like after k steps?

407
00:26:52,920 --> 00:26:54,980
Let me just ask you that.

408
00:26:54,980 --> 00:26:58,500
And then, what happens
as k goes to infinity?

409
00:26:58,500 --> 00:27:00,360
This is a steady-state
question, right?

410
00:27:00,360 --> 00:27:02,770
I'm looking for
the steady state.

411
00:27:02,770 --> 00:27:06,090
Actually, the question doesn't
ask for the k step answer,

412
00:27:06,090 --> 00:27:08,380
it just jumps right
away to infinity --

413
00:27:08,380 --> 00:27:15,170
but how would I express
the solution after k steps?

414
00:27:15,170 --> 00:27:23,340
It would be some multiple of
the first eigenvalue to the k-th

415
00:27:23,340 --> 00:27:26,160
power -- times the
first eigenvector,

416
00:27:26,160 --> 00:27:30,210
plus some other multiple
of the second eigenvalue,

417
00:27:30,210 --> 00:27:34,410
times its eigenvector, and
some multiple of the third

418
00:27:34,410 --> 00:27:38,440
eigenvalue, times
its eigenvector.

419
00:27:38,440 --> 00:27:39,285
OK.

420
00:27:39,285 --> 00:27:39,785
Good.

421
00:27:42,290 --> 00:27:49,140
And these eigenvalues are
zero, one, and minus point two.

422
00:27:52,810 --> 00:27:55,290
So what happens as
k goes to infinity?

423
00:27:58,480 --> 00:28:01,550
The only thing that
survives the steady state --

424
00:28:01,550 --> 00:28:07,860
so at u infinity, this
is gone, this is gone,

425
00:28:07,860 --> 00:28:16,640
all that's left is c2x2.

426
00:28:16,640 --> 00:28:18,550
So I'd better find x2.

427
00:28:18,550 --> 00:28:20,700
I've got to find
that eigenvector

428
00:28:20,700 --> 00:28:22,710
to complete the answer.

429
00:28:22,710 --> 00:28:25,260
What's the eigenvector
that corresponds

430
00:28:25,260 --> 00:28:26,400
to lambda equal one?

431
00:28:26,400 --> 00:28:29,650
That's the key eigenvector
in any Markov process,

432
00:28:29,650 --> 00:28:32,430
is that eigenvector.

433
00:28:32,430 --> 00:28:34,550
Lambda equal one
is an eigenvalue,

434
00:28:34,550 --> 00:28:38,280
I need its eigenvector
x2, and then

435
00:28:38,280 --> 00:28:44,480
I need to know how much of it
is in the starting vector u0.

436
00:28:44,480 --> 00:28:45,240
OK.

437
00:28:45,240 --> 00:28:47,880
So, how do I find
that eigenvector?

438
00:28:47,880 --> 00:28:51,500
I guess I subtract one
from the diagonal, right?

439
00:28:51,500 --> 00:28:56,280
So I have minus point
eight, minus point eight,

440
00:28:56,280 --> 00:28:59,660
minus point six, and the
rest, of course, is just --

441
00:28:59,660 --> 00:29:04,360
still point four, point
four, point four, point four,

442
00:29:04,360 --> 00:29:07,150
point three, point
three, and hopefully,

443
00:29:07,150 --> 00:29:19,230
that's a singular matrix, so
I'm looking to solve A minus Ix

444
00:29:19,230 --> 00:29:19,980
equal zero.

445
00:29:19,980 --> 00:29:22,972
Let's see -- can anybody
spot the solution here?

446
00:29:22,972 --> 00:29:24,930
I don't know, I didn't
make it easy for myself.

447
00:29:27,900 --> 00:29:30,630
What do you think there?

448
00:29:30,630 --> 00:29:39,080
Maybe those first two
entries might be --

449
00:29:39,080 --> 00:29:41,500
oh, no, what do you think?

450
00:29:41,500 --> 00:29:44,640
Anybody see it?

451
00:29:44,640 --> 00:29:46,680
We could use elimination
if we were desperate.

452
00:29:49,210 --> 00:29:51,810
Are we that desperate?

453
00:29:51,810 --> 00:29:55,320
Anybody just call out if
you see the vector that's

454
00:29:55,320 --> 00:29:57,980
in that null space.

455
00:29:57,980 --> 00:30:00,390
Eh, there better be a
vector in that null space,

456
00:30:00,390 --> 00:30:03,220
or I'm quitting.

457
00:30:03,220 --> 00:30:11,555
Uh, ha- OK, well, I guess
we could use elimination.

458
00:30:15,200 --> 00:30:18,010
I thought maybe somebody might
see it from further away.

459
00:30:21,100 --> 00:30:23,820
Is there a chance
that these guys are --

460
00:30:23,820 --> 00:30:28,140
could it be that these two are
equal and this is whatever it

461
00:30:28,140 --> 00:30:31,500
takes, like, something
like three, three, two?

462
00:30:31,500 --> 00:30:33,870
Would that possibly work?

463
00:30:33,870 --> 00:30:37,170
I mean, that's great for this
-- no, it's not that great.

464
00:30:37,170 --> 00:30:39,810
Three, three, four --

465
00:30:39,810 --> 00:30:43,930
this is, deeper mathematics
you're watching now.

466
00:30:43,930 --> 00:30:47,460
Three, three, four, is that --

467
00:30:47,460 --> 00:30:48,000
it works!

468
00:30:48,000 --> 00:30:49,030
Don't mess with it!

469
00:30:49,030 --> 00:30:49,910
It works!

470
00:30:49,910 --> 00:30:52,420
Uh, yes.

471
00:30:52,420 --> 00:30:54,140
OK, it works, all right.

472
00:30:54,140 --> 00:31:01,710
And, yes, OK, and, so that's
x2, three, three, four,

473
00:31:01,710 --> 00:31:11,870
and, how much of that vector
is in the starting vector?

474
00:31:11,870 --> 00:31:16,420
Well, we could go through
a complicated process.

475
00:31:16,420 --> 00:31:19,430
But what's the beauty
of Markov things?

476
00:31:19,430 --> 00:31:23,460
That the total number
of the total population,

477
00:31:23,460 --> 00:31:27,870
the sum of these doesn't change.

478
00:31:27,870 --> 00:31:30,200
That the total number of
people, they're moving around,

479
00:31:30,200 --> 00:31:35,130
but they don't get born
or die or get dead.

480
00:31:35,130 --> 00:31:38,520
So there's ten of them at the
start, so there's ten of them

481
00:31:38,520 --> 00:31:41,570
there, so c2 is
actually one, yes.

482
00:31:41,570 --> 00:31:45,380
So that would be the
correct solution.

483
00:31:45,380 --> 00:31:45,880
OK.

484
00:31:45,880 --> 00:31:48,490
That would be the u infinity.

485
00:31:48,490 --> 00:31:49,120
OK.

486
00:31:49,120 --> 00:31:50,890
So I used there,
in that process,

487
00:31:50,890 --> 00:31:53,190
sort of, the main
facts about Markov

488
00:31:53,190 --> 00:31:58,160
matrices to, to get
a jump on the answer.

489
00:31:58,160 --> 00:31:59,090
OK. let's see.

490
00:31:59,090 --> 00:32:05,810
OK, here's some, kind of
quick, short questions.

491
00:32:05,810 --> 00:32:09,770
Uh, maybe I'll move over to
this board, and leave that for

492
00:32:09,770 --> 00:32:11,260
the moment.

493
00:32:11,260 --> 00:32:16,020
I'm looking for
two-by-two matrices.

494
00:32:16,020 --> 00:32:19,610
And I'll read out the property
I want, and you give me

495
00:32:19,610 --> 00:32:23,670
an example, or tell me
there isn't such a matrix.

496
00:32:23,670 --> 00:32:24,640
All right.

497
00:32:24,640 --> 00:32:25,180
Here we go.

498
00:32:25,180 --> 00:32:28,240
First -- so two-by-twos.

499
00:32:28,240 --> 00:32:36,550
First, I want the
projection onto the line

500
00:32:36,550 --> 00:32:41,820
through A equals
four minus three.

501
00:32:46,190 --> 00:32:49,600
So it's a one-dimensional
projection matrix

502
00:32:49,600 --> 00:32:50,430
I'm looking for.

503
00:32:53,520 --> 00:32:56,280
And what's the formula for it?

504
00:32:56,280 --> 00:33:00,950
What's the formula for the
projection matrix P onto a line

505
00:33:00,950 --> 00:33:03,960
through A. And then we'd just
plug in this particular A.

506
00:33:03,960 --> 00:33:08,050
Do you remember that formula?

507
00:33:08,050 --> 00:33:14,150
There's an A and an A
transpose, and normally we

508
00:33:14,150 --> 00:33:17,030
would have an A transpose
A inverse in the middle,

509
00:33:17,030 --> 00:33:20,730
but here we've just got numbers,
so we just divide by it.

510
00:33:20,730 --> 00:33:25,780
And then plug in A
and we've got it.

511
00:33:25,780 --> 00:33:26,850
So, equals.

512
00:33:26,850 --> 00:33:28,290
OK.

513
00:33:28,290 --> 00:33:30,390
You can put in the numbers.

514
00:33:30,390 --> 00:33:31,790
Trivial, right.

515
00:33:31,790 --> 00:33:32,370
OK.

516
00:33:32,370 --> 00:33:33,190
Number two.

517
00:33:38,440 --> 00:33:39,900
So this is a new problem.

518
00:33:39,900 --> 00:33:45,910
The matrix with eigenvalue zero
and three and eigenvectors --

519
00:33:45,910 --> 00:33:49,740
well, let me write these
down. eigenvalue zero,

520
00:33:49,740 --> 00:33:56,320
eigenvector one, two, eigenvalue
three, eigenvector two, one.

521
00:33:59,170 --> 00:34:01,970
I'm giving you the
eigenvalues and eigenvectors

522
00:34:01,970 --> 00:34:04,090
instead of asking for them.

523
00:34:04,090 --> 00:34:05,485
Now I'm asking for the matrix.

524
00:34:10,080 --> 00:34:11,590
What's the matrix, then?

525
00:34:11,590 --> 00:34:12,210
What's A?

526
00:34:17,310 --> 00:34:19,090
Here was a formula,
then we just put

527
00:34:19,090 --> 00:34:21,610
in some numbers, what's
the formula here,

528
00:34:21,610 --> 00:34:26,270
into which we'll just
put the given numbers?

529
00:34:26,270 --> 00:34:30,840
It's the S lambda
S inverse, right?

530
00:34:30,840 --> 00:34:35,290
So it's S, which is
this eigenvector matrix,

531
00:34:35,290 --> 00:34:41,090
it's the lambda, which
is the eigenvalue matrix,

532
00:34:41,090 --> 00:34:44,790
it's the S inverse, whatever
that turns out to be,

533
00:34:44,790 --> 00:34:46,369
let me just leave it as inverse.

534
00:34:49,489 --> 00:34:52,000
That has to be it, right?

535
00:34:52,000 --> 00:34:54,409
Because if we went in
the other direction,

536
00:34:54,409 --> 00:35:00,150
that matrix S would diagonalize
A to produce lambda.

537
00:35:00,150 --> 00:35:02,340
So it's S lambda S inverse.

538
00:35:02,340 --> 00:35:03,200
Good.

539
00:35:03,200 --> 00:35:06,150
OK, ready for number three.

540
00:35:06,150 --> 00:35:13,120
A real matrix that cannot
be factored into A --

541
00:35:13,120 --> 00:35:17,570
I'm looking for a matrix
A that never could

542
00:35:17,570 --> 00:35:24,000
equal B transpose B, for any B.

543
00:35:24,000 --> 00:35:27,840
A two-by-two matrix that could
not be factored in the form B

544
00:35:27,840 --> 00:35:30,540
transpose B.

545
00:35:30,540 --> 00:35:33,470
So all you have to do is think,
well, what does B transpose B,

546
00:35:33,470 --> 00:35:37,060
look like, and then pick
something different.

547
00:35:37,060 --> 00:35:38,530
What do you suggest?

548
00:35:42,690 --> 00:35:43,910
Let's see.

549
00:35:43,910 --> 00:35:46,850
What shall we take for
a matrix that could not

550
00:35:46,850 --> 00:35:49,430
have this form, B transpose B.

551
00:35:49,430 --> 00:35:51,770
Well, what do we know
about B transpose B?

552
00:35:51,770 --> 00:35:54,090
It's always symmetric.

553
00:35:54,090 --> 00:35:56,280
So just give me any
non-symmetric matrix,

554
00:35:56,280 --> 00:35:58,460
it couldn't possibly
have that form.

555
00:35:58,460 --> 00:35:59,160
OK.

556
00:35:59,160 --> 00:36:02,030
And let me ask the fourth
part of this question --

557
00:36:02,030 --> 00:36:07,310
a matrix that has
orthogonal eigenvectors,

558
00:36:07,310 --> 00:36:12,820
but it's not symmetric.

559
00:36:12,820 --> 00:36:15,900
What matrices have
orthogonal eigenvectors,

560
00:36:15,900 --> 00:36:17,655
but they're not
symmetric matrices?

561
00:36:20,560 --> 00:36:28,240
What other families of matrices
have orthogonal eigenvectors?

562
00:36:28,240 --> 00:36:33,130
We know symmetric matrices
do, but others, also.

563
00:36:33,130 --> 00:36:40,030
So I'm looking for
orthogonal eigenvectors,

564
00:36:40,030 --> 00:36:42,540
and, what do you suggest?

565
00:36:47,220 --> 00:36:51,460
The matrix could
be skew-symmetric.

566
00:36:51,460 --> 00:36:55,100
It could be an
orthogonal matrix.

567
00:36:55,100 --> 00:36:59,590
It could be symmetric,
but that was too easy,

568
00:36:59,590 --> 00:37:01,140
so I ruled that out.

569
00:37:01,140 --> 00:37:11,050
It could be skew-symmetric
like one minus one, like that.

570
00:37:11,050 --> 00:37:19,680
Or it could be an orthogonal
matrix like cosine sine,

571
00:37:19,680 --> 00:37:22,250
minus sine, cosine.

572
00:37:22,250 --> 00:37:28,240
All those matrices would
have complex orthogonal

573
00:37:28,240 --> 00:37:30,620
eigenvectors.

574
00:37:30,620 --> 00:37:35,540
But they would be orthogonal,
and so those examples are fine.

575
00:37:35,540 --> 00:37:36,270
OK.

576
00:37:36,270 --> 00:37:45,660
We can continue a little longer
if you would like to, with

577
00:37:45,660 --> 00:37:47,260
these --

578
00:37:47,260 --> 00:37:48,620
from this exam.

579
00:37:48,620 --> 00:37:49,990
From these exams.

580
00:37:49,990 --> 00:37:50,630
Least squares?

581
00:37:53,570 --> 00:37:56,180
OK, here's a least
squares problem in which,

582
00:37:56,180 --> 00:38:00,130
to make life quick,
I've given the answer --

583
00:38:00,130 --> 00:38:03,240
it's like Jeopardy!, right?

584
00:38:03,240 --> 00:38:06,240
I just give the answer,
and you give the question.

585
00:38:06,240 --> 00:38:07,480
OK.

586
00:38:07,480 --> 00:38:12,540
Whoops, sorry.

587
00:38:12,540 --> 00:38:17,710
Let's see, can I stay over
here for the next question?

588
00:38:17,710 --> 00:38:18,210
OK.

589
00:38:22,300 --> 00:38:22,980
least squares.

590
00:38:22,980 --> 00:38:28,370
So I'm giving you the problem,
one, one, one, zero, one, two,

591
00:38:28,370 --> 00:38:35,890
c d equals three, four, one,
and that's b, of course,

592
00:38:35,890 --> 00:38:37,290
this is Ax=b.

593
00:38:40,310 --> 00:38:43,040
And the least
squares solution --

594
00:38:43,040 --> 00:38:46,350
Maybe I put c hat
d hat to emphasize

595
00:38:46,350 --> 00:38:49,670
it's not the true solution.

596
00:38:49,670 --> 00:38:54,520
So the least square solution
-- the hats really go here --

597
00:38:54,520 --> 00:38:58,360
is eleven-thirds and minus one.

598
00:38:58,360 --> 00:39:01,470
Of course, you could have
figured that out in no time.

599
00:39:01,470 --> 00:39:05,390
So this year, I'll ask
you to do it, probably.

600
00:39:05,390 --> 00:39:09,250
But, suppose we're
given the answer,

601
00:39:09,250 --> 00:39:14,240
then let's just
remember what happened.

602
00:39:14,240 --> 00:39:16,050
OK, good question.

603
00:39:16,050 --> 00:39:21,070
What's the projection P of
this vector onto the column

604
00:39:21,070 --> 00:39:22,110
space of that matrix?

605
00:39:25,090 --> 00:39:29,350
So I'll write that
question down, one.

606
00:39:29,350 --> 00:39:30,530
What is P?

607
00:39:30,530 --> 00:39:31,490
The projection.

608
00:39:31,490 --> 00:39:41,290
The projection of b onto the
column space of A is what?

609
00:39:44,860 --> 00:39:50,030
Hopefully, that's what the
least squares problem solved.

610
00:39:50,030 --> 00:39:51,430
What is it?

611
00:39:54,430 --> 00:40:02,220
This was the best solution, it's
eleven-thirds times column one,

612
00:40:02,220 --> 00:40:07,760
plus -- or rather, minus
one times column two.

613
00:40:07,760 --> 00:40:08,260
Right?

614
00:40:08,260 --> 00:40:10,280
That's what least squares did.

615
00:40:10,280 --> 00:40:14,470
It found the combination
of the columns that

616
00:40:14,470 --> 00:40:16,350
was as close as possible to b.

617
00:40:16,350 --> 00:40:18,740
That's what least
squares was doing.

618
00:40:18,740 --> 00:40:20,450
It found the projection.

619
00:40:20,450 --> 00:40:21,960
OK?

620
00:40:21,960 --> 00:40:26,440
Secondly, draw the straight
line problem that corresponds to

621
00:40:26,440 --> 00:40:27,590
this system.

622
00:40:27,590 --> 00:40:32,060
So I guess that the straight
line fitting a straight line

623
00:40:32,060 --> 00:40:35,110
problem, we kind of recognize.

624
00:40:35,110 --> 00:40:37,410
So we recognize,
these are the heights,

625
00:40:37,410 --> 00:40:41,380
and these are the points,
and so at zero, one, two,

626
00:40:41,380 --> 00:40:46,420
the heights are three,
and at t equal to one,

627
00:40:46,420 --> 00:40:50,720
the height is four,
one, two, three, four,

628
00:40:50,720 --> 00:40:53,640
and at t equal to two,
the height is one.

629
00:40:56,180 --> 00:41:01,880
So I'm trying to fit
the best straight line

630
00:41:01,880 --> 00:41:04,470
through those points.

631
00:41:04,470 --> 00:41:04,970
God.

632
00:41:07,640 --> 00:41:10,360
I could fit a
triangle very well,

633
00:41:10,360 --> 00:41:16,130
but, I don't even know which
way the best straight line goes.

634
00:41:16,130 --> 00:41:19,340
Oh, I do know how it goes,
because there's the answer,yes.

635
00:41:19,340 --> 00:41:25,860
It has a height eleven-thirds,
and it has slope minus one,

636
00:41:25,860 --> 00:41:28,590
so it's something like that.

637
00:41:28,590 --> 00:41:29,370
Great.

638
00:41:29,370 --> 00:41:29,930
OK.

639
00:41:29,930 --> 00:41:36,680
Now, finally -- and this
completes the course --

640
00:41:36,680 --> 00:41:41,140
find a different vector
b, not all zeroes,

641
00:41:41,140 --> 00:41:45,310
for which the least square
solution would be zero.

642
00:41:45,310 --> 00:41:50,140
So I want you to
find a different B

643
00:41:50,140 --> 00:41:54,915
so that the least square
solution changes to all zeroes.

644
00:42:00,530 --> 00:42:04,360
So tell me what I'm
really looking for here.

645
00:42:04,360 --> 00:42:08,640
I'm looking for a b where the
best combination of these two

646
00:42:08,640 --> 00:42:11,780
columns is the zero combination.

647
00:42:11,780 --> 00:42:15,200
So what kind of a
vector b I looking for?

648
00:42:15,200 --> 00:42:16,780
I'm looking for
a vector b that's

649
00:42:16,780 --> 00:42:19,230
orthogonal to those columns.

650
00:42:19,230 --> 00:42:20,810
It's orthogonal
to those columns,

651
00:42:20,810 --> 00:42:22,810
it's orthogonal to
the column space,

652
00:42:22,810 --> 00:42:24,880
the best possible answer is

653
00:42:24,880 --> 00:42:25,550
zero.

654
00:42:25,550 --> 00:42:29,870
So a vector b that's orthogonal
to those columns -- let's see,

655
00:42:29,870 --> 00:42:35,830
maybe one of those minus two
of those, and one of those?

656
00:42:35,830 --> 00:42:38,460
That would be orthogonal
to those columns,

657
00:42:38,460 --> 00:42:42,720
and the best vector
would be zero, zero.

658
00:42:42,720 --> 00:42:43,470
OK.

659
00:42:43,470 --> 00:42:46,590
So that's as many questions
as I can do in an hour,

660
00:42:46,590 --> 00:42:50,490
but you get three hours,
and, let me just say,

661
00:42:50,490 --> 00:42:56,100
as I've said by e-mail, thanks
very much for your patience

662
00:42:56,100 --> 00:43:00,060
as this series of
lectures was videotaped,

663
00:43:00,060 --> 00:43:04,490
and, thanks for filling
out these forms,

664
00:43:04,490 --> 00:43:08,250
maybe just leave them on the
table up there as you go out --

665
00:43:08,250 --> 00:43:10,901
and above all, thanks
for taking the course.

666
00:43:10,901 --> 00:43:11,400
Thank you.

667
00:43:11,400 --> 00:43:12,950
Thanks.