1
00:00:06,870 --> 00:00:09,060
PROFESSOR: Hey, we're back.

2
00:00:09,060 --> 00:00:11,870
Today we're going to do a
singular value decomposition

3
00:00:11,870 --> 00:00:13,510
question.

4
00:00:13,510 --> 00:00:16,630
The problem is really
simple to state:

5
00:00:16,630 --> 00:00:19,970
find the singular value
decomposition of this matrix

6
00:00:19,970 --> 00:00:25,150
C equals [5, 5; -1, 7].

7
00:00:25,150 --> 00:00:28,680
Hit pause, try it yourself,
I'll be back in a minute

8
00:00:28,680 --> 00:00:29,920
and we can do it together.

9
00:00:40,360 --> 00:00:44,400
All right, we're back,
now let's do it together.

10
00:00:44,400 --> 00:00:46,050
Now, I know Professor
Strang has done

11
00:00:46,050 --> 00:00:50,600
a couple of these in lecture,
but as he pointed out there,

12
00:00:50,600 --> 00:00:52,430
it's really easy
to make a mistake,

13
00:00:52,430 --> 00:00:59,460
so you can never do enough
examples of finding the SVD.

14
00:00:59,460 --> 00:01:02,690
So, what does the SVD look like?

15
00:01:02,690 --> 00:01:05,190
What do we want to end up with?

16
00:01:05,190 --> 00:01:16,620
Well, we want a decomposition
C equals U sigma V transpose.

17
00:01:16,620 --> 00:01:20,620
U and V are going to be
orthogonal matrices, that

18
00:01:20,620 --> 00:01:25,690
is, their columns
are orthonormal sets.

19
00:01:25,690 --> 00:01:28,760
Sigma is going to
be a diagonal matrix

20
00:01:28,760 --> 00:01:31,690
with non-negative entries.

21
00:01:31,690 --> 00:01:32,610
OK, good.

22
00:01:32,610 --> 00:01:34,930
So now, how do we find
this decomposition?

23
00:01:34,930 --> 00:01:39,640
Well, we need two equations, OK?

24
00:01:39,640 --> 00:01:52,070
One is C transpose C is equal
to V, sigma transpose, sigma,

25
00:01:52,070 --> 00:01:54,260
V transpose.

26
00:01:54,260 --> 00:01:57,390
And you get this just by
plugging in C transpose C

27
00:01:57,390 --> 00:02:00,940
here and noticing that U
transpose U is 1, since U

28
00:02:00,940 --> 00:02:02,752
is an orthogonal matrix.

29
00:02:02,752 --> 00:02:04,200
Okay.

30
00:02:04,200 --> 00:02:08,620
And the second equation is just
noticing that V transpose is V

31
00:02:08,620 --> 00:02:11,640
inverse, and moving it to the
other side of the equation,

32
00:02:11,640 --> 00:02:15,930
which is C*V equals U*sigma.

33
00:02:15,930 --> 00:02:18,400
OK, so these are
the two equations

34
00:02:18,400 --> 00:02:24,470
we need to use to find
V, sigma, and U. OK,

35
00:02:24,470 --> 00:02:28,040
so let's start
with the first one.

36
00:02:28,040 --> 00:02:33,070
Let's compute C transpose
C. So C transpose C

37
00:02:33,070 --> 00:02:47,220
is that-- Well, if
you compute, we'll

38
00:02:47,220 --> 00:03:00,300
get a 26, an 18, an
18, and a 74, great.

39
00:03:00,300 --> 00:03:03,100
Now, what you notice
about this equation

40
00:03:03,100 --> 00:03:06,680
is this is just a
diagonalization of C transpose

41
00:03:06,680 --> 00:03:10,460
C. So we need to find
the eigenvalues-- those

42
00:03:10,460 --> 00:03:13,780
will be the entries
of sigma transpose

43
00:03:13,780 --> 00:03:16,160
sigma-- and the
eigenvectors which will be

44
00:03:16,160 --> 00:03:19,910
the columns of a V. Okay, good.

45
00:03:22,490 --> 00:03:24,810
So how do we find those?

46
00:03:24,810 --> 00:03:30,690
Well, we look at the determinant
of C transpose C minus lambda

47
00:03:30,690 --> 00:03:35,730
times the identity,
which is the determinant

48
00:03:35,730 --> 00:03:48,710
of 26 minus lambda, 18, 18,
and 74-- 74 minus lambda,

49
00:03:48,710 --> 00:03:49,320
thank you.

50
00:03:55,520 --> 00:03:59,860
Good, OK, and what
is that polynomial?

51
00:03:59,860 --> 00:04:06,460
Well, we get a lambda squared,
now the 26 plus 74 is 100,

52
00:04:06,460 --> 00:04:09,130
so minus 100*lambda.

53
00:04:09,130 --> 00:04:14,310
And I'll let you do 26 times 74
minus 18 squared on your own,

54
00:04:14,310 --> 00:04:22,100
but you'll see you get 1,600,
and this easily factors

55
00:04:22,100 --> 00:04:25,080
as lambda minus 20
times lambda minus 80.

56
00:04:27,880 --> 00:04:31,550
So the eigenvalues
are 20 and 80.

57
00:04:31,550 --> 00:04:33,150
Now what are the eigenvectors?

58
00:04:33,150 --> 00:04:39,850
Well, you take C transpose C
minus 20 times the identity,

59
00:04:39,850 --> 00:04:47,486
and you get 6, 18, 18 and 54.

60
00:04:50,400 --> 00:04:54,550
To find the eigenvector
with eigenvalue 20,

61
00:04:54,550 --> 00:04:58,940
we need to find a vector in
the null space of this matrix.

62
00:04:58,940 --> 00:05:02,650
Note that the second
column is three times

63
00:05:02,650 --> 00:05:08,430
the first column, so our
first vector, v_1, we can just

64
00:05:08,430 --> 00:05:16,220
take that to be, well, we
could take it to be [-3, 1],

65
00:05:16,220 --> 00:05:21,040
but we'd like it to be
a unit vector, right?

66
00:05:21,040 --> 00:05:23,960
Remember the columns of
v should be unit vectors

67
00:05:23,960 --> 00:05:25,600
because they're orthonormal.

68
00:05:25,600 --> 00:05:28,699
So 3 squared plus
1 squared is 10,

69
00:05:28,699 --> 00:05:30,490
we have to divide by
the square root of 10.

70
00:05:34,260 --> 00:05:43,580
OK, similarly, we do C transpose
C minus 80 times the identity,

71
00:05:43,580 --> 00:05:56,400
we'll get -54, 18; 18,
and -6, and similarly

72
00:05:56,400 --> 00:06:05,000
we can find that v_2 will
be 1 over square root of 10,

73
00:06:05,000 --> 00:06:08,360
3 over the square root of 10.

74
00:06:08,360 --> 00:06:11,550
Great, OK, so what
information do we have now?

75
00:06:11,550 --> 00:06:15,070
we have our V matrix, which
is just made up of these two

76
00:06:15,070 --> 00:06:18,720
columns, and we actually
have our sigma matrix too,

77
00:06:18,720 --> 00:06:23,670
because the squares of the
diagonal entries of sigma

78
00:06:23,670 --> 00:06:26,250
are 20 and 80.

79
00:06:26,250 --> 00:06:33,130
Good, so let's write those
down, write down what we have.

80
00:06:33,130 --> 00:06:40,640
So we have V-- I just
add these vectors

81
00:06:40,640 --> 00:06:43,120
and make them the
columns of my matrix.

82
00:06:43,120 --> 00:06:48,770
Square root of 10, 1
over square root of 10;

83
00:06:48,770 --> 00:06:55,570
1 over square root of 10,
3 over square root of 10.

84
00:06:55,570 --> 00:07:02,490
And sigma, this is just the
square roots of 20 and 80,

85
00:07:02,490 --> 00:07:08,085
which is just 2 root 5 and
4 root 5 along the diagonal.

86
00:07:11,200 --> 00:07:14,500
Squeezing it in here, I hope
you all can see these two.

87
00:07:17,104 --> 00:07:21,830
Good, so these are two of the
three parts of my singular

88
00:07:21,830 --> 00:07:24,940
value decomposition.

89
00:07:24,940 --> 00:07:27,190
The last thing I
need to find is u,

90
00:07:27,190 --> 00:07:31,600
and for that I need to use this
second equation right here.

91
00:07:31,600 --> 00:07:37,470
So you need to multiply
C times V, okay so So

92
00:07:37,470 --> 00:07:47,550
c is [5, 5; -1, 7],
let's multiply it by V,

93
00:07:47,550 --> 00:07:54,380
1 over root 10, 3 over
square root of 10.

94
00:07:54,380 --> 00:07:56,960
What do we get?

95
00:07:56,960 --> 00:08:01,920
Well, I'll let you
work out the details,

96
00:08:01,920 --> 00:08:05,000
but it's not hard here.

97
00:08:05,000 --> 00:08:09,650
You get -10 over root 10, which
is just negative square root

98
00:08:09,650 --> 00:08:12,180
of 10 here.

99
00:08:12,180 --> 00:08:32,940
Then I just get 2 square
root of 10, and then I get--

100
00:08:32,940 --> 00:08:44,760
1 is 2 square root of 10 and--

101
00:08:44,760 --> 00:08:49,730
I think I made an error here.

102
00:08:49,730 --> 00:08:55,350
Give me a second to look
through my computation again.

103
00:08:55,350 --> 00:08:58,300
AUDIENCE: [INAUDIBLE]

104
00:08:58,300 --> 00:09:03,410
PROFESSOR: The (2, 1) entry
should be-- oh, yes, thank you.

105
00:09:03,410 --> 00:09:05,660
The (2, 1) entry should
be the square root of 10.

106
00:09:05,660 --> 00:09:20,980
Good, yes, that's what I was
hoping, yes, because we get--

107
00:09:20,980 --> 00:09:24,380
Yes, I did it in
the wrong order,

108
00:09:24,380 --> 00:09:28,460
right, so your recitation
instructor should

109
00:09:28,460 --> 00:09:33,970
know how to multiply matrices,
great, yes, thank you.

110
00:09:33,970 --> 00:09:37,580
You multiply this first, then
this, then this, and then this,

111
00:09:37,580 --> 00:09:44,030
and if you do it correctly
you will get this matrix here.

112
00:09:44,030 --> 00:09:45,510
Good, great.

113
00:09:45,510 --> 00:09:49,980
So now I'd like this
to be my U matrix,

114
00:09:49,980 --> 00:09:55,990
but it's actually U times sigma,
so I need to make these entries

115
00:09:55,990 --> 00:09:57,570
unit length.

116
00:09:57,570 --> 00:10:08,260
OK, so I get -1 over root 2, 1
over root 2, 1 over root 2, 1

117
00:10:08,260 --> 00:10:12,120
over root 2, times
my sigma matrix

118
00:10:12,120 --> 00:10:17,740
here, which is, remember,
2 square root of 5,

119
00:10:17,740 --> 00:10:22,130
4 square root of 5,
and these constants

120
00:10:22,130 --> 00:10:25,390
are just what I needed to
divide these columns by in order

121
00:10:25,390 --> 00:10:27,840
to make them unit vectors.

122
00:10:27,840 --> 00:10:33,630
So now, here's my U matrix,
1 over square root of 2,

123
00:10:33,630 --> 00:10:38,250
-1 over square root of 2;
1 over square root of 2,

124
00:10:38,250 --> 00:10:41,160
1 over square root of 2, good.

125
00:10:41,160 --> 00:10:46,710
So now I have all three
matrices, U, V, and sigma

126
00:10:46,710 --> 00:10:49,340
and despite some little
errors here and there,

127
00:10:49,340 --> 00:10:51,950
I think this is actually right.

128
00:10:51,950 --> 00:10:54,340
You should go check it
yourself, because if you're

129
00:10:54,340 --> 00:10:59,370
at all like me, you've screwed
up several times by now.

130
00:10:59,370 --> 00:11:02,190
But anyway, this is
a good illustration

131
00:11:02,190 --> 00:11:05,120
of how to find the singular
value decomposition.

132
00:11:05,120 --> 00:11:07,880
Recall that you're
looking for U sigma V

133
00:11:07,880 --> 00:11:11,990
transpose where u and v
are orthogonal matrices,

134
00:11:11,990 --> 00:11:16,410
and sigma is diagonal
with non-negative entries.

135
00:11:16,410 --> 00:11:21,290
And you find it using
these two equations,

136
00:11:21,290 --> 00:11:26,540
you compute C transpose C,
that's V sigma transpose sigma

137
00:11:26,540 --> 00:11:32,340
times V transpose, and you
also have C*V is U*sigma.

138
00:11:32,340 --> 00:11:35,100
I hope this was a
helpful illustration.