1
00:00:02,310 --> 00:00:05,720
GILBERT STRANG: So
this is the key video

2
00:00:05,720 --> 00:00:11,750
about solving a system of n
linear constant coefficient

3
00:00:11,750 --> 00:00:13,220
equations.

4
00:00:13,220 --> 00:00:16,560
So how do I write
those equations?

5
00:00:16,560 --> 00:00:21,630
Y is now a vector, a
vector with n components.

6
00:00:21,630 --> 00:00:25,480
Instead of one scalar,
just a single number y--

7
00:00:25,480 --> 00:00:28,760
do you want me to
put an arrow on y?

8
00:00:28,760 --> 00:00:31,600
No, I won't repeat it again.

9
00:00:31,600 --> 00:00:36,640
But that's to emphasize
that y is a vector.

10
00:00:36,640 --> 00:00:40,156
Its first derivative,
it's a first order system.

11
00:00:40,156 --> 00:00:45,650
System meaning that there can
be and will be more than one

12
00:00:45,650 --> 00:00:49,720
unknown, y1, y2, to yn.

13
00:00:49,720 --> 00:00:52,420
So how do we solve
such a system?

14
00:00:52,420 --> 00:00:57,390
Then the matrix is multiplying
that y and they equate.

15
00:00:57,390 --> 00:01:00,760
The y's are coupled
together by that matrix.

16
00:01:00,760 --> 00:01:04,099
They're coupled together,
and how do we uncouple them?

17
00:01:04,099 --> 00:01:09,070
That is the magic of
eigenvalues and eigenvectors.

18
00:01:09,070 --> 00:01:15,330
Eigenvectors are vectors
that go in their own way.

19
00:01:15,330 --> 00:01:18,150
So when you have an
eigenvector, it's

20
00:01:18,150 --> 00:01:20,470
like you have a
one by one problem

21
00:01:20,470 --> 00:01:25,840
and the a becomes
just a number, lambda.

22
00:01:25,840 --> 00:01:31,700
So for a general vector,
everything is a mixed together.

23
00:01:31,700 --> 00:01:34,580
But for an
eigenvector, everything

24
00:01:34,580 --> 00:01:38,510
stays one dimensional.

25
00:01:38,510 --> 00:01:45,450
The a changes just to a lambda
for that special direction.

26
00:01:45,450 --> 00:01:50,650
And of course, as always, we
need n of those eigenvectors

27
00:01:50,650 --> 00:01:54,370
because we want to take
the starting value.

28
00:01:54,370 --> 00:01:57,220
Just as we did for
powers, we're doing it now

29
00:01:57,220 --> 00:02:00,790
for differential equations.

30
00:02:00,790 --> 00:02:05,580
I take my starting vector, which
is probably not an eigenvector.

31
00:02:05,580 --> 00:02:08,850
I'd make it a combination
of eigenvectors.

32
00:02:08,850 --> 00:02:11,700
And I'm OK because I'm
assuming that I have

33
00:02:11,700 --> 00:02:14,950
n independent eigenvectors.

34
00:02:14,950 --> 00:02:22,140
And now I follow each
starting value c1 x1--

35
00:02:22,140 --> 00:02:23,300
what does that have?

36
00:02:23,300 --> 00:02:28,010
What happens if I'm in
the direction of x1,

37
00:02:28,010 --> 00:02:35,330
then all the messiness
of A disappears.

38
00:02:35,330 --> 00:02:40,740
It acts just like lambda
1 on that vector x1.

39
00:02:40,740 --> 00:02:41,940
Here's what you get.

40
00:02:41,940 --> 00:02:46,180
You get c1, that's
just a number, times e

41
00:02:46,180 --> 00:02:51,220
to the lambda 1t x1.

42
00:02:51,220 --> 00:02:54,440
You see there,
instead of powers,

43
00:02:54,440 --> 00:02:58,500
which we had-- that we had
lambda 1 to the kth power

44
00:02:58,500 --> 00:03:01,380
when we were doing
powers of a matrix,

45
00:03:01,380 --> 00:03:03,900
now we're solving
differential equations.

46
00:03:03,900 --> 00:03:06,620
So we get an e to the lambda 1t.

47
00:03:06,620 --> 00:03:11,330
And of course, next
by superposition,

48
00:03:11,330 --> 00:03:14,630
I can add on the solution
for that one, which

49
00:03:14,630 --> 00:03:19,530
is e to the lambda
2t x2 plus so on,

50
00:03:19,530 --> 00:03:28,270
plus cne to the lambda nt xn.

51
00:03:28,270 --> 00:03:32,250
You can see when, I could
ask, when is this stable?

52
00:03:32,250 --> 00:03:34,380
When do the solutions go to 0?

53
00:03:34,380 --> 00:03:39,580
Well, as t gets
large, this number

54
00:03:39,580 --> 00:03:44,350
will go to 0, provided
lambda 1 is negative.

55
00:03:44,350 --> 00:03:48,530
Or provided its real
part is negative.

56
00:03:48,530 --> 00:03:52,630
We can understand
everything from this piece

57
00:03:52,630 --> 00:03:54,910
by piece formula.

58
00:03:54,910 --> 00:03:57,260
Let me just do an example.

59
00:03:57,260 --> 00:04:07,290
Take a matrix A. In the powers
of a matrix-- in that video

60
00:04:07,290 --> 00:04:13,270
I took a Markov matrix-- let
me take here the equivalent

61
00:04:13,270 --> 00:04:14,690
for differential equations.

62
00:04:14,690 --> 00:04:20,829
So this will give us a
Markov differential equation.

63
00:04:20,829 --> 00:04:22,570
So let me take A now.

64
00:04:26,660 --> 00:04:30,360
The columns of a
Markov matrix add to 1

65
00:04:30,360 --> 00:04:35,830
but in the differential equation
situation, they'll add to 0.

66
00:04:35,830 --> 00:04:41,790
Like minus 1 and 1,
or like minus 2 and 2.

67
00:04:41,790 --> 00:04:53,450
So there is the eigenvalue
of 1 for our powers

68
00:04:53,450 --> 00:04:59,370
is like the eigenvalue 0
for differential equations.

69
00:04:59,370 --> 00:05:02,400
Because e to the 0t is 1.

70
00:05:02,400 --> 00:05:05,240
So anyway, let's find
the eigenvalues of that.

71
00:05:05,240 --> 00:05:08,220
The first eigenvalue is 0.

72
00:05:08,220 --> 00:05:10,160
That's what I'm interested in.

73
00:05:10,160 --> 00:05:12,830
That column adds to 0,
that column adds to 0.

74
00:05:12,830 --> 00:05:15,560
That tells me there's
an eigenvalue of 0.

75
00:05:15,560 --> 00:05:16,884
And what's its eigenvector?

76
00:05:20,120 --> 00:05:26,480
Probably 2, 1 because if
I multiply that matrix

77
00:05:26,480 --> 00:05:29,050
by that vector, I get 0.

78
00:05:29,050 --> 00:05:30,900
So lambda 1 is 0.

79
00:05:30,900 --> 00:05:37,740
And my second eigenvalue,
well the trace is minus 3

80
00:05:37,740 --> 00:05:43,520
and the lambda 1 plus
lambda 2 must give minus 3.

81
00:05:43,520 --> 00:05:48,887
And its eigenvector is-- it's
probably 1 minus 1 again.

82
00:05:52,110 --> 00:05:55,420
So I've done the
preliminary work.

83
00:05:55,420 --> 00:05:59,640
Given this matrix, we've got the
eigenvalues and eigenvectors.

84
00:05:59,640 --> 00:06:03,670
Now I take u0-- what
should we say for u0?

85
00:06:03,670 --> 00:06:07,870
U0-- y0, say y of 0 to start.

86
00:06:11,170 --> 00:06:19,310
Y of 0 as some number c1
times x1 plus c2 times x2.

87
00:06:19,310 --> 00:06:22,330
Yes, no problem, no problem.

88
00:06:22,330 --> 00:06:29,390
Whatever we have, we take this--
some combination of that vector

89
00:06:29,390 --> 00:06:32,930
and that eigenvector
will give us y of 0.

90
00:06:32,930 --> 00:06:45,750
And now the y of t is c1 e
to the 0t-- e to the lambda

91
00:06:45,750 --> 00:06:50,450
1t times x1, right?

92
00:06:50,450 --> 00:06:54,970
You see, we started with
c1x1 but after a time t,

93
00:06:54,970 --> 00:06:59,156
it's either the lambda
t and here's c2.

94
00:06:59,156 --> 00:07:06,510
E to the lambda
2 is minus 3t x2.

95
00:07:06,510 --> 00:07:13,520
That's the evolution of a Markov
process, a continuous Markov

96
00:07:13,520 --> 00:07:14,930
process.

97
00:07:14,930 --> 00:07:17,920
Compared to the
powers of a matrix,

98
00:07:17,920 --> 00:07:25,000
this is a continuous evolving
evolution of this vector.

99
00:07:25,000 --> 00:07:29,740
Now, I'm interested
in steady state.

100
00:07:29,740 --> 00:07:35,920
Steady state is what
happens as t gets large.

101
00:07:35,920 --> 00:07:40,700
As t gets large, that
number goes quickly to 0.

102
00:07:40,700 --> 00:07:46,270
In our Markov matrix example,
we had 1/2 to a power,

103
00:07:46,270 --> 00:07:47,980
and that went quickly to 0.

104
00:07:47,980 --> 00:07:54,300
Now we have the exponential with
a minus 3, that goes to zero.

105
00:07:54,300 --> 00:07:57,260
E to the 0t is the 1.

106
00:07:57,260 --> 00:08:01,960
This e to the 0t equals 1.

107
00:08:01,960 --> 00:08:05,230
So that 1 is the signal
of a steady state.

108
00:08:05,230 --> 00:08:08,400
Nothing changing, nothing
really depending on time,

109
00:08:08,400 --> 00:08:09,730
just sits there.

110
00:08:09,730 --> 00:08:15,095
So I have c1x1 is
the steady state.

111
00:08:19,820 --> 00:08:22,840
And x1 was this.

112
00:08:22,840 --> 00:08:25,640
So what am I thinking?

113
00:08:25,640 --> 00:08:28,020
I'm thinking that
no matter how you

114
00:08:28,020 --> 00:08:34,760
start, no matter what y
of 0 is, as time goes on,

115
00:08:34,760 --> 00:08:37,830
the x2 part is
going to disappear.

116
00:08:37,830 --> 00:08:43,250
And if you only have the
x1 part in that ratio 2:1.

117
00:08:43,250 --> 00:08:49,960
So again, if we had
movement between Y1 Y2

118
00:08:49,960 --> 00:08:52,830
or we have things
evolving in time,

119
00:08:52,830 --> 00:09:01,200
the steady state is--
this is the steady state.

120
00:09:06,470 --> 00:09:09,910
There is an example of
a differential equation,

121
00:09:09,910 --> 00:09:12,420
happen to have a Markov matrix.

122
00:09:12,420 --> 00:09:15,490
And with a Markov
matrix, then we

123
00:09:15,490 --> 00:09:18,900
know that we'll have
an eigenvalue of -

124
00:09:18,900 --> 00:09:25,890
in the continuous case and a
negative eigenvalue that will

125
00:09:25,890 --> 00:09:28,050
disappear as time goes forward.

126
00:09:28,050 --> 00:09:30,980
E to the minus 3t goes to 0.

127
00:09:30,980 --> 00:09:32,020
Good.

128
00:09:32,020 --> 00:09:38,510
I guess I might just add a
little bit to this video, which

129
00:09:38,510 --> 00:09:45,420
is to explain why is
0 an eigenvalue when

130
00:09:45,420 --> 00:09:51,290
whenever-- if the columns added
to 0, minus 1 plus 1 is 0.

131
00:09:51,290 --> 00:09:53,000
2 minus 2 is zero.

132
00:09:53,000 --> 00:09:55,820
That tells me 0
is an eigenvalue.

133
00:09:55,820 --> 00:10:01,540
For a Markov matrix empowers
the columns added to 1 and 1

134
00:10:01,540 --> 00:10:02,710
was an eigenvalue.

135
00:10:02,710 --> 00:10:09,410
So I guess I have now two
examples of the following fact.

136
00:10:09,410 --> 00:10:21,330
That if all columns add
to some-- what shall

137
00:10:21,330 --> 00:10:28,290
I say for the sum, s for
the sum-- then lambda

138
00:10:28,290 --> 00:10:31,980
equal s is an eigenvalue.

139
00:10:37,950 --> 00:10:43,050
That was the point from
Markov matrices, s was 1.

140
00:10:43,050 --> 00:10:46,730
Every column added to 1
and then lambda equal 1

141
00:10:46,730 --> 00:10:48,170
was an eigenvalue.

142
00:10:48,170 --> 00:10:50,580
And for this video,
every column added

143
00:10:50,580 --> 00:10:54,960
to 0 and then lambda
equal 0 was an eigenvalue.

144
00:10:54,960 --> 00:10:59,300
And also, this is another
point about eigenvalues,

145
00:10:59,300 --> 00:11:00,840
good to make.

146
00:11:00,840 --> 00:11:03,720
The eigenvalues
of a transpose are

147
00:11:03,720 --> 00:11:05,700
the same as the
eigenvalues of A.

148
00:11:05,700 --> 00:11:17,460
So I could also say if
all rows of A add to s,

149
00:11:17,460 --> 00:11:23,205
then lambda equal
s is an eigenvalue.

150
00:11:26,440 --> 00:11:31,020
I'm saying that the eigenvalues
of a matrix and the eigenvalues

151
00:11:31,020 --> 00:11:33,320
of the transpose are the same.

152
00:11:33,320 --> 00:11:37,560
And maybe you would like to
just see why that's true.

153
00:11:37,560 --> 00:11:40,320
If I want the
eigenvalues of a matrix,

154
00:11:40,320 --> 00:11:44,720
I look at the determinant of
lambda I minus A. That gives me

155
00:11:44,720 --> 00:11:51,000
eigenvalues of A. If I want
the eigenvalues of a transpose,

156
00:11:51,000 --> 00:11:57,650
I would look at this
equals 0, right?

157
00:11:57,650 --> 00:12:00,670
This equaling 0.

158
00:12:00,670 --> 00:12:03,170
That equation would
give me the eigenvalues

159
00:12:03,170 --> 00:12:06,250
of a transpose just the
way this one gives me

160
00:12:06,250 --> 00:12:07,720
the eigenvalues of A.

161
00:12:07,720 --> 00:12:11,030
But why are they the same?

162
00:12:11,030 --> 00:12:15,400
Because the determinant of
a matrix and the determinant

163
00:12:15,400 --> 00:12:18,060
of its transpose are equal.

164
00:12:18,060 --> 00:12:21,630
A matrix and its transpose
have the same determinant.

165
00:12:21,630 --> 00:12:25,723
Let me just check
that with A, B, C,

166
00:12:25,723 --> 00:12:31,860
D. And the transpose
would be A, C, B, D.

167
00:12:31,860 --> 00:12:33,950
And the determinant
in both cases

168
00:12:33,950 --> 00:12:39,520
is AD minus BC, AD minus BC.

169
00:12:39,520 --> 00:12:41,060
Transposing doesn't affect.

170
00:12:41,060 --> 00:12:46,730
So this, that is
the same as that.

171
00:12:46,730 --> 00:12:49,670
And the lambdas are the same.

172
00:12:49,670 --> 00:12:52,890
And therefore we can look
at the columns adding to s

173
00:12:52,890 --> 00:12:54,615
or the rows added to s.

174
00:12:57,200 --> 00:13:03,750
So this explains why those
two statements are both true

175
00:13:03,750 --> 00:13:09,170
together because I could look
at the rows or the columns

176
00:13:09,170 --> 00:13:11,660
and reach this conclusion.

177
00:13:11,660 --> 00:13:16,490
That if all columns add
to s-- now why is that,

178
00:13:16,490 --> 00:13:18,730
or all rows add to s?

179
00:13:18,730 --> 00:13:22,930
Let me just-- I'll just
show you the eigenvector.

180
00:13:22,930 --> 00:13:30,710
In this case, A times the
eigenvector would be all ones.

181
00:13:30,710 --> 00:13:33,700
Suppose the matrix is 4 by 4.

182
00:13:33,700 --> 00:13:37,790
If I multiply A by all ones,
when you multiply a matrix

183
00:13:37,790 --> 00:13:40,730
by a vector of
ones, then the dot

184
00:13:40,730 --> 00:13:44,180
product of this row
with that is the sum,

185
00:13:44,180 --> 00:13:47,920
is that plus that plus that
plus that, would be we s.

186
00:13:47,920 --> 00:13:52,770
This would be s because
this first row-- here is

187
00:13:52,770 --> 00:13:58,106
A-- first row of A adds to s.

188
00:13:58,106 --> 00:14:02,070
So these numbers
add to s, I get s.

189
00:14:02,070 --> 00:14:05,320
These numbers add
to s, I get s again.

190
00:14:05,320 --> 00:14:07,286
These numbers add to s.

191
00:14:07,286 --> 00:14:10,660
And these, finally
those numbers add to s.

192
00:14:10,660 --> 00:14:15,575
And I have s times 1, 1, 1, 1.

193
00:14:19,930 --> 00:14:22,450
Are you OK with this?

194
00:14:22,450 --> 00:14:25,130
When all the rows add to
s, I can tell you what

195
00:14:25,130 --> 00:14:28,696
the eigenvector is, 1, 1, 1, 1.

196
00:14:28,696 --> 00:14:34,300
And then the eigenvalue, I
can see that that's the sum s.

197
00:14:34,300 --> 00:14:39,170
So again, for special
matrices, in this case

198
00:14:39,170 --> 00:14:44,810
named after Markov, we are
able to identify important fact

199
00:14:44,810 --> 00:14:52,600
about their eigenvalue, which is
that it's that common row sum s

200
00:14:52,600 --> 00:14:58,650
equal 1 in the case of
powers and s equal 0

201
00:14:58,650 --> 00:15:05,280
in this video's case with--
let me bring down A again.

202
00:15:05,280 --> 00:15:09,630
So here, every
column added to 0.

203
00:15:09,630 --> 00:15:12,790
It didn't happen that
the rows added to 0.

204
00:15:12,790 --> 00:15:14,870
I'm not requiring that.

205
00:15:14,870 --> 00:15:18,420
I'm just saying either
way, A or A transpose

206
00:15:18,420 --> 00:15:22,500
has the same eigenvalues
and one of them is 0

207
00:15:22,500 --> 00:15:28,180
and the other is whatever
the trace tells us, that one.

208
00:15:28,180 --> 00:15:32,630
These collection of useful
fact about eigenvalues

209
00:15:32,630 --> 00:15:36,430
show up when you have
a particular matrix

210
00:15:36,430 --> 00:15:40,460
and you need to know something
about its eigenvalues.

211
00:15:40,460 --> 00:15:42,880
Good, thank you.