1
00:00:00,500 --> 00:00:01,730
GILBERT STRANG: OK.

2
00:00:01,730 --> 00:00:04,810
This is positive
definite matrix day.

3
00:00:07,330 --> 00:00:12,200
Our application was the
second-order equation

4
00:00:12,200 --> 00:00:17,320
with a symmetric matrix, S.
And we solved this equation.

5
00:00:17,320 --> 00:00:20,880
Second derivative, plus
S times y, equals 0.

6
00:00:20,880 --> 00:00:23,520
And you maybe remember
how we solved it.

7
00:00:23,520 --> 00:00:27,520
We looked for an
exponential solution.

8
00:00:27,520 --> 00:00:30,990
e to the I omega t,
times a vector x.

9
00:00:30,990 --> 00:00:33,250
We substituted,
and we discovered

10
00:00:33,250 --> 00:00:39,740
x had to be an eigenvector
of S, as usual.

11
00:00:39,740 --> 00:00:47,030
And lambda, which was omega
squared, is the eigenvalue.

12
00:00:47,030 --> 00:00:51,150
But I didn't stop to
point out that if we

13
00:00:51,150 --> 00:00:53,320
want lambda to be
omega squared, we

14
00:00:53,320 --> 00:00:56,580
need to know lambda
greater or equal to 0.

15
00:00:56,580 --> 00:00:59,880
So that is the best
of the best matrices.

16
00:00:59,880 --> 00:01:05,300
Symmetric and positive definite,
or positive semidefinite,

17
00:01:05,300 --> 00:01:10,130
which means the eigenvalues
are not only real,

18
00:01:10,130 --> 00:01:12,570
they're real for
symmetric matrices.

19
00:01:12,570 --> 00:01:14,940
They're also positive.

20
00:01:14,940 --> 00:01:18,270
Positive definite matrices--
automatically symmetric,

21
00:01:18,270 --> 00:01:21,160
I'm only talking about
symmetric matrices--

22
00:01:21,160 --> 00:01:24,240
and positive eigenvalues.

23
00:01:24,240 --> 00:01:24,740
OK.

24
00:01:24,740 --> 00:01:25,520
There it is.

25
00:01:25,520 --> 00:01:27,420
Positive definite matrix.

26
00:01:27,420 --> 00:01:29,830
All the eigenvalues
are positive.

27
00:01:29,830 --> 00:01:31,490
Positive semidefinite.

28
00:01:31,490 --> 00:01:35,590
That word semi allows
lambda equal 0.

29
00:01:35,590 --> 00:01:39,120
The matrix could be singular,
but all the eigenvalues

30
00:01:39,120 --> 00:01:41,520
have to be greater
or equal to 0.

31
00:01:41,520 --> 00:01:46,870
And let me show you exactly
where those matrices come from.

32
00:01:46,870 --> 00:01:51,180
Those matrices come
from A transpose A.

33
00:01:51,180 --> 00:01:54,830
If I take any matrix A,
could be rectangular.

34
00:01:54,830 --> 00:01:59,550
And A transpose
A will be square.

35
00:01:59,550 --> 00:02:02,340
A transpose A will be symmetric.

36
00:02:02,340 --> 00:02:08,110
And it will be at least
positive semidefinite.

37
00:02:08,110 --> 00:02:09,380
Why is that?

38
00:02:09,380 --> 00:02:13,230
This is the simple step
that is worth remembering.

39
00:02:13,230 --> 00:02:17,860
What's special about A
transpose A x equal lambda x?

40
00:02:17,860 --> 00:02:19,560
The good idea?

41
00:02:19,560 --> 00:02:23,640
Multiply both sides
by x transpose.

42
00:02:23,640 --> 00:02:26,750
Take the inner product
of both sides with x.

43
00:02:26,750 --> 00:02:29,900
Then I have x transpose
times the left side,

44
00:02:29,900 --> 00:02:33,550
is x transpose times
the right side.

45
00:02:33,550 --> 00:02:36,230
I'm OK with equation 2.

46
00:02:36,230 --> 00:02:41,240
When my S is A transpose
A, that's my S. OK.

47
00:02:41,240 --> 00:02:43,120
But now I look at this.

48
00:02:43,120 --> 00:02:48,630
That is A x in a
product with itself.

49
00:02:48,630 --> 00:02:52,990
That's the length
of A x squared.

50
00:02:52,990 --> 00:02:57,150
Any time I have y
transpose y, I'm

51
00:02:57,150 --> 00:02:59,610
getting the length of y squared.

52
00:02:59,610 --> 00:03:04,240
Here y is A x, so I'm getting
the length of A x squared.

53
00:03:04,240 --> 00:03:06,640
Over here, y is
x, so I'm getting

54
00:03:06,640 --> 00:03:08,660
the length of x squared.

55
00:03:08,660 --> 00:03:14,690
And you see that number
lambda is, in this equation,

56
00:03:14,690 --> 00:03:18,620
I have a number that
can't be negative.

57
00:03:18,620 --> 00:03:22,320
A number that's
positive, for sure.

58
00:03:22,320 --> 00:03:24,760
Because x is not the 0 vector.

59
00:03:24,760 --> 00:03:29,075
So lambda is never negative.

60
00:03:29,075 --> 00:03:32,370
A x could be the 0 vector.

61
00:03:32,370 --> 00:03:36,250
If we were in a singular case,
A x could be the 0 vector.

62
00:03:36,250 --> 00:03:39,600
In that case, I would only
learn lambda equals 0,

63
00:03:39,600 --> 00:03:42,570
and I'd be in this
semidefinite case.

64
00:03:42,570 --> 00:03:48,790
So if I want to move from
semidefinite to definite,

65
00:03:48,790 --> 00:03:53,530
then I must rule out
A x equals 0 there.

66
00:03:53,530 --> 00:03:56,050
Because that's
certainly a possibility.

67
00:03:56,050 --> 00:04:00,540
If I took the 0
matrix, all 0's, as A,

68
00:04:00,540 --> 00:04:03,140
A transpose A would
be the 0 matrix.

69
00:04:03,140 --> 00:04:05,460
That would be symmetric.

70
00:04:05,460 --> 00:04:07,920
All its eigenvalues would be 0.

71
00:04:07,920 --> 00:04:10,060
Would it be positive
semidefinite?

72
00:04:10,060 --> 00:04:11,120
Yes.

73
00:04:11,120 --> 00:04:11,700
Yes.

74
00:04:11,700 --> 00:04:14,740
All its eigenvalues
would actually be 0.

75
00:04:14,740 --> 00:04:18,089
Of course, that's not a
case that we are really

76
00:04:18,089 --> 00:04:19,140
thinking about.

77
00:04:19,140 --> 00:04:21,510
More often we're
in this good case

78
00:04:21,510 --> 00:04:25,290
where all the
eigenvalues are above 0.

79
00:04:25,290 --> 00:04:25,790
OK.

80
00:04:25,790 --> 00:04:27,690
So that's the meaning.

81
00:04:27,690 --> 00:04:30,830
And now the next job.

82
00:04:30,830 --> 00:04:34,630
How do we recognize a
positive definite matrix?

83
00:04:34,630 --> 00:04:36,240
It has to be symmetric.

84
00:04:36,240 --> 00:04:37,650
That's easy to see.

85
00:04:37,650 --> 00:04:41,860
But how can we tell if its
eigenvalues are positive?

86
00:04:41,860 --> 00:04:46,360
That's not fun because computing
eigenvalues is a big job.

87
00:04:49,010 --> 00:04:52,320
For a large matrix,
we take time on that.

88
00:04:52,320 --> 00:04:55,220
We didn't know how to do
it a little while ago.

89
00:04:55,220 --> 00:04:57,040
Now there are good
ways to do it,

90
00:04:57,040 --> 00:05:01,000
but it's not for paper
and pencil, and not for I.

91
00:05:01,000 --> 00:05:04,930
So how can we tell that all
the eigenvalues are positive?

92
00:05:04,930 --> 00:05:07,460
Well, we only want
to know their sign.

93
00:05:07,460 --> 00:05:10,170
We don't have to
know what they are.

94
00:05:10,170 --> 00:05:12,820
We don't know that we need the
number, we just want to know

95
00:05:12,820 --> 00:05:14,540
is it a positive number.

96
00:05:14,540 --> 00:05:17,550
And there are
several neat tests.

97
00:05:17,550 --> 00:05:18,592
Can I show you them?

98
00:05:21,970 --> 00:05:24,210
I'm going to have five tests.

99
00:05:24,210 --> 00:05:26,040
Five equivalent tests.

100
00:05:26,040 --> 00:05:31,050
Any one of these tests is
sufficient to make the matrix S

101
00:05:31,050 --> 00:05:32,310
positive definite.

102
00:05:32,310 --> 00:05:36,720
There is a particular S there
that I'll use as a test matrix.

103
00:05:36,720 --> 00:05:39,110
So there is a
symmetric matrix S.

104
00:05:39,110 --> 00:05:41,800
And I know it is
positive definite.

105
00:05:41,800 --> 00:05:43,530
But how do I know?

106
00:05:43,530 --> 00:05:44,150
OK.

107
00:05:44,150 --> 00:05:44,660
Well.

108
00:05:44,660 --> 00:05:48,970
So can you take
five things here?

109
00:05:48,970 --> 00:05:51,560
They connect all
of linear algebra.

110
00:05:51,560 --> 00:05:53,060
It's really beautiful.

111
00:05:53,060 --> 00:05:57,260
That the eigenvalues, that's
one chapter of linear algebra.

112
00:05:57,260 --> 00:06:00,170
The pivots are another
chapter of linear algebra.

113
00:06:00,170 --> 00:06:01,920
Do you remember about pivots?

114
00:06:01,920 --> 00:06:04,020
That's when you do elimination.

115
00:06:04,020 --> 00:06:06,380
So 4 is the first pivot.

116
00:06:06,380 --> 00:06:07,660
The first pivot.

117
00:06:07,660 --> 00:06:13,200
Pivot number 1 is the 4.

118
00:06:13,200 --> 00:06:18,500
And then when I take a multiple
of that away from that,

119
00:06:18,500 --> 00:06:20,320
I get a second pivot.

120
00:06:20,320 --> 00:06:22,620
And I'd see that
that was positive.

121
00:06:22,620 --> 00:06:24,320
So what's that?

122
00:06:24,320 --> 00:06:26,720
Maybe I take 1 and
1/2 away of this.

123
00:06:29,470 --> 00:06:32,060
I multiply that by
1 and 1/2, 6, 9.

124
00:06:32,060 --> 00:06:33,840
Subtract from 6, 10.

125
00:06:33,840 --> 00:06:36,190
So I actually get
a 1 down there.

126
00:06:36,190 --> 00:06:42,760
So pivot number 2
is a 1 in that case.

127
00:06:42,760 --> 00:06:43,510
Right?

128
00:06:43,510 --> 00:06:47,900
6, 9 taken away from
6, 10 leaves me 0, 1.

129
00:06:47,900 --> 00:06:48,930
OK.

130
00:06:48,930 --> 00:06:51,040
It passed the pivot test.

131
00:06:51,040 --> 00:06:54,150
Notice I didn't compute
the eigenvalues.

132
00:06:54,150 --> 00:06:55,430
I'm just doing other tests.

133
00:06:55,430 --> 00:06:57,910
Now here's another
beautiful test.

134
00:06:57,910 --> 00:07:00,390
It involves determinants.

135
00:07:00,390 --> 00:07:02,914
Now, I have to say upper.

136
00:07:02,914 --> 00:07:05,800
Upper left.

137
00:07:05,800 --> 00:07:18,580
Upper left determinants
greater than 0.

138
00:07:18,580 --> 00:07:21,490
What do I mean by an
upper left determinant?

139
00:07:21,490 --> 00:07:23,550
I look at my matrix.

140
00:07:23,550 --> 00:07:26,690
That's a 1 by 1 determinant.

141
00:07:26,690 --> 00:07:27,660
Certainly positive.

142
00:07:27,660 --> 00:07:29,750
That determinant is 4.

143
00:07:29,750 --> 00:07:32,570
Here is a 2 by 2 determinant.

144
00:07:32,570 --> 00:07:38,080
And that determinant is 40 minus
36, so happened to be 4 again.

145
00:07:38,080 --> 00:07:40,410
So the determinant
of the matrix is 4.

146
00:07:40,410 --> 00:07:44,320
But I also needed
the ones on the way.

147
00:07:44,320 --> 00:07:46,470
I can't just find
the determinant

148
00:07:46,470 --> 00:07:48,080
of the whole matrix.

149
00:07:48,080 --> 00:07:50,760
That's the last
part of this test,

150
00:07:50,760 --> 00:07:53,570
but I have to do all the
others as I get there.

151
00:07:53,570 --> 00:07:55,370
So it passes that test.

152
00:07:55,370 --> 00:07:55,910
Check.

153
00:07:55,910 --> 00:07:56,930
It works.

154
00:07:56,930 --> 00:07:59,500
So that test is passed.

155
00:07:59,500 --> 00:08:02,660
I'm doing more work than I need
to do because one test would

156
00:08:02,660 --> 00:08:03,980
have done the job.

157
00:08:03,980 --> 00:08:06,350
Now here comes another one.

158
00:08:06,350 --> 00:08:14,150
S is A transpose A. That's
what we looked at a minute ago.

159
00:08:14,150 --> 00:08:16,820
If S has this form
A transpose A.

160
00:08:16,820 --> 00:08:20,420
Oh, what did we
convince ourselves?

161
00:08:20,420 --> 00:08:25,250
We said that this was
sure to be semidefinite.

162
00:08:25,250 --> 00:08:29,960
And I needed some condition
to avoid A x equals 0.

163
00:08:29,960 --> 00:08:32,420
There was the possibility
of A x equals 0.

164
00:08:32,420 --> 00:08:35,450
I'll just bring that down.

165
00:08:35,450 --> 00:08:39,299
You remember we started
there and ended up here.

166
00:08:39,299 --> 00:08:43,070
And if A x was 0
then lambda was 0.

167
00:08:43,070 --> 00:08:45,360
We were in the
semidefinite case.

168
00:08:45,360 --> 00:08:47,320
So I have to avoid that.

169
00:08:47,320 --> 00:08:56,205
So I have to say when A
has independent columns.

170
00:09:03,910 --> 00:09:10,000
And I think I could factor that
matrix S into A transpose A.

171
00:09:10,000 --> 00:09:11,460
I'm sure I could.

172
00:09:11,460 --> 00:09:13,630
And get independent columns.

173
00:09:13,630 --> 00:09:16,410
And it would pass test 4.

174
00:09:16,410 --> 00:09:18,450
I want to go on to test 5.

175
00:09:18,450 --> 00:09:22,070
Which really, in a way, is
the definition, the best

176
00:09:22,070 --> 00:09:24,350
definition, of
positive definite.

177
00:09:24,350 --> 00:09:31,110
So if I took number 5,
it's the energy definition.

178
00:09:31,110 --> 00:09:34,350
So can I tell you
what that means?

179
00:09:34,350 --> 00:09:38,100
I mean that x transpose Sx.

180
00:09:40,960 --> 00:09:45,040
If I take my matrix S that I'm
testing for positive definite,

181
00:09:45,040 --> 00:09:47,050
I multiply on the
right by any vector

182
00:09:47,050 --> 00:09:50,490
x, any x, and on the
left by x transpose.

183
00:09:50,490 --> 00:09:52,890
Well, I get a number.

184
00:09:52,890 --> 00:09:54,030
S is a matrix.

185
00:09:54,030 --> 00:09:55,620
Sx is a vector.

186
00:09:55,620 --> 00:09:56,890
Inner product with a vector.

187
00:09:56,890 --> 00:09:58,170
I get a number.

188
00:09:58,170 --> 00:10:03,475
And that number should
be positive for all x.

189
00:10:07,190 --> 00:10:09,270
Oh, I have to make
one exception.

190
00:10:09,270 --> 00:10:13,630
If x is the 0 vector, then
of course that answer is 0.

191
00:10:13,630 --> 00:10:19,930
All x except the 0 vector.

192
00:10:19,930 --> 00:10:21,410
OK.

193
00:10:21,410 --> 00:10:24,820
So that would be a
way to-- another test.

194
00:10:24,820 --> 00:10:29,040
And this is associated in
applications with energy.

195
00:10:29,040 --> 00:10:32,980
So I call this the
energy test, or really

196
00:10:32,980 --> 00:10:36,420
the energy definition,
of positive definite.

197
00:10:36,420 --> 00:10:38,470
x transpose Sx.

198
00:10:38,470 --> 00:10:41,240
I'd like to apply that test.

199
00:10:41,240 --> 00:10:42,690
So you'll see what does it mean.

200
00:10:42,690 --> 00:10:49,760
Now we're looking at all x
to this particular example.

201
00:10:49,760 --> 00:10:53,010
But I won't throw
away this board here.

202
00:10:53,010 --> 00:10:58,150
You see eigenvalues, pivots,
determinants, A transpose A,

203
00:10:58,150 --> 00:11:00,370
and energy.

204
00:11:00,370 --> 00:11:03,260
Really all the pieces
of linear algebra.

205
00:11:03,260 --> 00:11:06,570
A transpose A. We'll
see it more and more.

206
00:11:06,570 --> 00:11:09,600
It comes up in least squares.

207
00:11:09,600 --> 00:11:14,680
If I have a general matrix
A, it's not even square.

208
00:11:14,680 --> 00:11:16,350
It doesn't have
great properties.

209
00:11:16,350 --> 00:11:19,280
But when I compute
A transpose A,

210
00:11:19,280 --> 00:11:22,690
then I get a symmetric matrix.

211
00:11:22,690 --> 00:11:27,270
And now I know also a
positive semidefinite.

212
00:11:27,270 --> 00:11:33,420
And with a little bit more
positive definite matrix.

213
00:11:33,420 --> 00:11:34,190
OK.

214
00:11:34,190 --> 00:11:39,680
By the way, are there five
tests for semidefinite matrices?

215
00:11:39,680 --> 00:11:40,180
Yes.

216
00:11:40,180 --> 00:11:43,060
There are five similar tests.

217
00:11:43,060 --> 00:11:46,030
All eigenvalues
greater or equal to 0.

218
00:11:46,030 --> 00:11:48,660
All pivots greater
or equal to 0.

219
00:11:48,660 --> 00:11:55,470
I can go down this and just
allow that borderline case

220
00:11:55,470 --> 00:11:57,880
that brings in semidefinite.

221
00:11:57,880 --> 00:11:59,080
I won't do that.

222
00:11:59,080 --> 00:12:04,510
Let me take my matrix S.
That small, example matrix.

223
00:12:04,510 --> 00:12:06,550
And apply the energy test.

224
00:12:06,550 --> 00:12:07,940
OK.

225
00:12:07,940 --> 00:12:09,320
So I'm looking at energy.

226
00:12:12,410 --> 00:12:13,880
So I'm looking at x.

227
00:12:13,880 --> 00:12:25,490
That's x1 x2, times my matrix
4, 6, 6, 10, times x1 x2.

228
00:12:25,490 --> 00:12:26,400
That's the energy.

229
00:12:26,400 --> 00:12:29,960
That's my x transpose Sx.

230
00:12:29,960 --> 00:12:31,620
x transpose Sx.

231
00:12:31,620 --> 00:12:34,160
Is that what we
wanted to compute?

232
00:12:34,160 --> 00:12:34,860
Yes.

233
00:12:34,860 --> 00:12:37,470
x transpose Sx.

234
00:12:37,470 --> 00:12:39,030
Now, can I compute that?

235
00:12:39,030 --> 00:12:40,980
Yes.

236
00:12:40,980 --> 00:12:42,910
It's a matrix multiplication.

237
00:12:42,910 --> 00:12:44,670
Nothing magical here.

238
00:12:44,670 --> 00:12:47,030
But when I do, I'll
show you the shortcut.

239
00:12:47,030 --> 00:12:51,060
When I do that, a 4
x1 is going to appear,

240
00:12:51,060 --> 00:12:54,850
and it'll be multiplied
by that x1 over there.

241
00:12:54,850 --> 00:12:58,040
I'll get a 4 x1 squared.

242
00:12:58,040 --> 00:13:03,270
And then I'll have a 6 x2
that's multiplying that x1.

243
00:13:03,270 --> 00:13:07,110
So there's a 6 x1 x2.

244
00:13:07,110 --> 00:13:09,710
And now from this.

245
00:13:09,710 --> 00:13:11,230
That was the first component.

246
00:13:11,230 --> 00:13:14,490
And now I have 6 x1 and 10 x2.

247
00:13:14,490 --> 00:13:15,920
Multiply an x2.

248
00:13:15,920 --> 00:13:19,830
Well, that's another 6 x1 x2.

249
00:13:19,830 --> 00:13:24,230
And the 10, we'll
multiply x2 and x2.

250
00:13:24,230 --> 00:13:26,290
x2 squared.

251
00:13:26,290 --> 00:13:28,390
I did that quickly.

252
00:13:28,390 --> 00:13:32,950
But the result is
just easy to see.

253
00:13:32,950 --> 00:13:37,300
The 4, 6, 6, 10 show
up in the squares.

254
00:13:37,300 --> 00:13:40,710
The diagonal 4 and 10
show up in the squares.

255
00:13:40,710 --> 00:13:46,300
And the off diagonal
6, which doubles to 12,

256
00:13:46,300 --> 00:13:50,860
shows up in the x1
x2, the cross term.

257
00:13:50,860 --> 00:13:51,650
OK.

258
00:13:51,650 --> 00:13:57,410
Now why should that-- so that's
a number for every x1 and x2.

259
00:13:57,410 --> 00:14:01,300
Suppose x1 is 1 and x2 is 1.

260
00:14:01,300 --> 00:14:05,200
Then the number I get is
4, plus 6, plus 6, plus 10.

261
00:14:05,200 --> 00:14:07,920
That's probably 26.

262
00:14:07,920 --> 00:14:09,690
It's positive.

263
00:14:09,690 --> 00:14:13,180
What if x1 is 1--
let me try this.

264
00:14:13,180 --> 00:14:17,790
x1 is 1 and x2 is minus 1.

265
00:14:17,790 --> 00:14:20,340
Do I still get a
positive energy?

266
00:14:20,340 --> 00:14:23,550
So my vector is 1 minus 1.

267
00:14:23,550 --> 00:14:25,750
So I get 4.

268
00:14:25,750 --> 00:14:30,420
Now, because of that, I
have minus 6, and minus 6,

269
00:14:30,420 --> 00:14:34,790
and 10, from the x2 squared.

270
00:14:34,790 --> 00:14:36,710
And that's 14 minus 12.

271
00:14:36,710 --> 00:14:38,197
That's 2.

272
00:14:38,197 --> 00:14:38,780
It's positive.

273
00:14:41,330 --> 00:14:44,770
Well, I tested two vectors.

274
00:14:44,770 --> 00:14:48,800
I tested the 1, 1 vector
and the 1, minus 1 vector.

275
00:14:48,800 --> 00:14:53,300
But you have to know
that for every vector x,

276
00:14:53,300 --> 00:14:55,660
this does turn out
to be positive.

277
00:14:55,660 --> 00:14:57,910
And I can show you
that by something

278
00:14:57,910 --> 00:15:01,640
called completing the square.

279
00:15:01,640 --> 00:15:04,520
It's not what I plan to do.

280
00:15:04,520 --> 00:15:09,600
But the beauty is we now
understand this energy test.

281
00:15:09,600 --> 00:15:15,230
What it means to take x
transpose Sx, write it out,

282
00:15:15,230 --> 00:15:18,520
and ask is it always positive.

283
00:15:18,520 --> 00:15:20,290
Is it always positive?

284
00:15:20,290 --> 00:15:20,790
OK.

285
00:15:20,790 --> 00:15:24,250
So that's the fifth,
number 5, test.

286
00:15:24,250 --> 00:15:27,250
But I think of it really
as the definition.

287
00:15:27,250 --> 00:15:30,530
And it means-- can
I draw a picture?

288
00:15:30,530 --> 00:15:32,050
Here is x1.

289
00:15:32,050 --> 00:15:33,410
Here's x2.

290
00:15:33,410 --> 00:15:36,470
And now I'm going to--
this is my function.

291
00:15:39,070 --> 00:15:40,490
x transpose A x.

292
00:15:40,490 --> 00:15:42,310
My energy.

293
00:15:42,310 --> 00:15:49,110
If I graph that, I have an
x, and a y, and a function z.

294
00:15:49,110 --> 00:15:51,320
That function of x and y.

295
00:15:51,320 --> 00:15:53,450
What kind of a
graph does it have?

296
00:15:53,450 --> 00:15:57,850
When x1 and x2
are 0, it's there.

297
00:15:57,850 --> 00:16:02,420
When x1 and x2 move away
from 0, it goes positive.

298
00:16:02,420 --> 00:16:05,630
That graph is like that.

299
00:16:05,630 --> 00:16:06,670
It's sort of a bowl.

300
00:16:10,060 --> 00:16:12,180
And I have a minimum.

301
00:16:16,370 --> 00:16:20,850
One of the main application
of derivatives in calculus

302
00:16:20,850 --> 00:16:24,760
is to find the
test for a minimum,

303
00:16:24,760 --> 00:16:27,810
and decide minimum or maximum.

304
00:16:27,810 --> 00:16:29,570
Minimum or maximum.

305
00:16:29,570 --> 00:16:31,410
And you remember the
second derivative

306
00:16:31,410 --> 00:16:33,780
decides a minimum or maximum.

307
00:16:33,780 --> 00:16:36,570
Positive second
derivative, minimum.

308
00:16:36,570 --> 00:16:38,840
Negative second
derivative, maximum.

309
00:16:38,840 --> 00:16:41,840
It tells you about the
bending of the curve.

310
00:16:41,840 --> 00:16:46,840
Well, we're in two
dimensions now,

311
00:16:46,840 --> 00:16:50,090
with a function
of two variables.

312
00:16:50,090 --> 00:16:52,580
This is multivariable calculus.

313
00:16:52,580 --> 00:16:57,030
So what becomes positive
second derivative,

314
00:16:57,030 --> 00:17:00,190
becomes positive
definite matrix.

315
00:17:00,190 --> 00:17:03,306
A matrix of second derivatives.

316
00:17:03,306 --> 00:17:06,950
This is the whole
subject of optimization.

317
00:17:06,950 --> 00:17:09,839
Maximizing,
minimizing, comes here.

318
00:17:09,839 --> 00:17:10,339
OK.

319
00:17:10,339 --> 00:17:13,140
That's for another day.

320
00:17:13,140 --> 00:17:14,819
I just would like
to tell you one more

321
00:17:14,819 --> 00:17:17,490
thing about positive
definite matrices.

322
00:17:17,490 --> 00:17:21,530
I got a book in the mail
which could be quite valuable.

323
00:17:21,530 --> 00:17:24,420
It's a little
paperback, and the title

324
00:17:24,420 --> 00:17:35,140
is Frequently Asked Questions in
Interviews for Financial Math.

325
00:17:35,140 --> 00:17:36,460
Being a Quant.

326
00:17:36,460 --> 00:17:37,560
Going to Wall Street.

327
00:17:37,560 --> 00:17:39,610
Becoming rich.

328
00:17:39,610 --> 00:17:43,580
So they don't give you
all the money right away.

329
00:17:43,580 --> 00:17:47,770
They make you show that
you know something.

330
00:17:47,770 --> 00:17:52,120
And so they ask a
few math questions.

331
00:17:52,120 --> 00:17:57,630
And the first question was--
I was happy to see this.

332
00:17:57,630 --> 00:18:04,390
The first question asked,
when is this matrix

333
00:18:04,390 --> 00:18:06,670
positive definite?

334
00:18:10,860 --> 00:18:11,360
OK.

335
00:18:11,360 --> 00:18:12,560
Can you see that matrix?

336
00:18:12,560 --> 00:18:14,370
1 is on the diagonal.

337
00:18:14,370 --> 00:18:15,910
Those are correlation.

338
00:18:15,910 --> 00:18:17,640
This is a correlation matrix.

339
00:18:17,640 --> 00:18:19,780
That's why it's
important in finance.

340
00:18:19,780 --> 00:18:21,930
It might be the
three correlations

341
00:18:21,930 --> 00:18:28,140
of bonds, and stocks,
and foreign exchange.

342
00:18:28,140 --> 00:18:32,350
So each one is
correlated to itself

343
00:18:32,350 --> 00:18:35,590
with a full correlation of 1.

344
00:18:35,590 --> 00:18:38,470
But there'll be a correlation
between bonds and stocks

345
00:18:38,470 --> 00:18:40,950
going up together,
but not perfectly

346
00:18:40,950 --> 00:18:43,770
together, by some number a.

347
00:18:43,770 --> 00:18:47,810
And bonds and foreign
exchange with some number b.

348
00:18:47,810 --> 00:18:52,630
Stocks and foreign
exchange, some number c.

349
00:18:52,630 --> 00:18:55,510
So that's the matrix
of correlations.

350
00:18:55,510 --> 00:18:59,810
And the key point is,
it is positive definite.

351
00:18:59,810 --> 00:19:03,520
So the question when
you go to Wall Street

352
00:19:03,520 --> 00:19:05,570
to apply for the money.

353
00:19:05,570 --> 00:19:12,130
If you're asked what's the test
on those numbers a, b, c, to

354
00:19:12,130 --> 00:19:18,060
have a positive definite,
proper correlation matrix?

355
00:19:18,060 --> 00:19:20,920
I would suggest the
determinant test.

356
00:19:20,920 --> 00:19:24,180
The determinant test, if
I'm given a small matrix,

357
00:19:24,180 --> 00:19:25,950
I'll just do the determinants.

358
00:19:25,950 --> 00:19:27,690
So that determinant is 1.

359
00:19:27,690 --> 00:19:29,030
No problem.

360
00:19:29,030 --> 00:19:32,600
This determinant, what's
the 2 by 2 determinant?

361
00:19:32,600 --> 00:19:34,530
1 minus a squared.

362
00:19:34,530 --> 00:19:38,330
So 1 minus a squared
has to be positive.

363
00:19:38,330 --> 00:19:40,450
I'm doing the determinant test.

364
00:19:40,450 --> 00:19:43,260
And what's the 3
by 3 determinant?

365
00:19:43,260 --> 00:19:45,930
1 from the diagonal.

366
00:19:45,930 --> 00:19:49,770
And I have an acb and an acb.

367
00:19:49,770 --> 00:19:57,250
I think I have two acb's
from the three terms.

368
00:19:57,250 --> 00:20:02,230
Now, those terms
have the plus signs.

369
00:20:02,230 --> 00:20:05,020
And now I have some
with a minus sign,

370
00:20:05,020 --> 00:20:06,850
which better not be too big.

371
00:20:06,850 --> 00:20:10,290
That's the whole point on
positive definite matrices.

372
00:20:10,290 --> 00:20:15,450
The off diagonal is not allowed
to overrun the diagonal.

373
00:20:15,450 --> 00:20:19,550
The diagonal should be
the biggest numbers.

374
00:20:19,550 --> 00:20:20,140
OK.

375
00:20:20,140 --> 00:20:25,280
So I saw that a squared
had to be below 1.

376
00:20:25,280 --> 00:20:27,730
But now what's the
determinant test?

377
00:20:27,730 --> 00:20:30,910
I think this has to
be bigger than what

378
00:20:30,910 --> 00:20:33,090
I'm getting from
this direction, which

379
00:20:33,090 --> 00:20:36,260
is a b squared, and a c
squared, and an a squared.

380
00:20:36,260 --> 00:20:37,410
Oh, look at that.

381
00:20:37,410 --> 00:20:40,530
a squared, b squared,
and c squared.

382
00:20:43,260 --> 00:20:45,810
That would be the answer.

383
00:20:45,810 --> 00:20:48,700
That first test there.

384
00:20:48,700 --> 00:20:51,340
Second test there.

385
00:20:51,340 --> 00:20:55,170
Well, the easy test was
just 1, is positive.

386
00:20:55,170 --> 00:20:59,250
So really, that's what
they're looking for.

387
00:20:59,250 --> 00:21:01,710
That would be the
test, those numbers.

388
00:21:01,710 --> 00:21:08,130
So abc can't be too large
or that would begin to fail.

389
00:21:08,130 --> 00:21:09,180
Good.

390
00:21:09,180 --> 00:21:14,260
So positive definite matrices
have lots of applications.

391
00:21:14,260 --> 00:21:15,870
Here was minimum.

392
00:21:15,870 --> 00:21:19,560
Here was correlation
matrices and finance.

393
00:21:19,560 --> 00:21:21,380
Many, many other places.

394
00:21:21,380 --> 00:21:28,160
Let me just bring
down the five tests.

395
00:21:28,160 --> 00:21:31,250
Eigenvalues, pivots,
determinants, A transpose A,

396
00:21:31,250 --> 00:21:33,290
and energy.

397
00:21:33,290 --> 00:21:34,670
And I'll stop there.

398
00:21:34,670 --> 00:21:36,340
Thank you.