1
00:00:06,454 --> 00:00:07,870
ANA RITA PIRES:
In lecture, you've

2
00:00:07,870 --> 00:00:10,260
learned about Gram-Schmidt
orthogonalization,

3
00:00:10,260 --> 00:00:12,600
and that's what today's
problem is about.

4
00:00:12,600 --> 00:00:17,790
We have a matrix A, and its
columns are a, b, and c.

5
00:00:17,790 --> 00:00:21,000
And I want you to find
orthonormal vectors

6
00:00:21,000 --> 00:00:24,360
q_1, q_2, and q_3 from
those three columns.

7
00:00:24,360 --> 00:00:28,890
Then I want you to write A
as a-- it's QR decomposition

8
00:00:28,890 --> 00:00:32,240
where Q is an orthogonal matrix,
and R is an upper triangular

9
00:00:32,240 --> 00:00:33,270
matrix.

10
00:00:33,270 --> 00:00:36,730
Remember, an orthogonal matrix
is a matrix whose columns

11
00:00:36,730 --> 00:00:39,480
are orthonormal vectors.

12
00:00:39,480 --> 00:00:41,169
Work on it for a
little while, hit

13
00:00:41,169 --> 00:00:42,960
pause, and when you're
ready I'll come back

14
00:00:42,960 --> 00:00:44,001
and we'll do it together.

15
00:00:52,170 --> 00:00:55,160
Did you manage to
solve that all right?

16
00:00:55,160 --> 00:00:58,490
Well let's start
solving it together.

17
00:00:58,490 --> 00:01:00,850
So Gram-Schmidt
orthogonalization,

18
00:01:00,850 --> 00:01:04,250
as you should
remember from lecture,

19
00:01:04,250 --> 00:01:06,210
consists of the following.

20
00:01:06,210 --> 00:01:10,540
At each step, you find
your orthonormal vector

21
00:01:10,540 --> 00:01:15,000
by taking the vector that
you started with, a, b, or c

22
00:01:15,000 --> 00:01:18,420
in this case, and
making it orthonormal

23
00:01:18,420 --> 00:01:22,430
to the previous ones.

24
00:01:22,430 --> 00:01:25,780
Let's actually do it.

25
00:01:25,780 --> 00:01:27,920
We want to find q_1.

26
00:01:27,920 --> 00:01:30,780
Well, to find q_1,
start with a, and make

27
00:01:30,780 --> 00:01:32,890
it orthonormal to
the previous one.

28
00:01:32,890 --> 00:01:35,020
There's no previous one,
so that's very easy.

29
00:01:35,020 --> 00:01:37,390
The direction of a
is fine and you just

30
00:01:37,390 --> 00:01:40,950
need to ensure that your
vector has length 1.

31
00:01:40,950 --> 00:01:44,790
Well, a is the vector [1, 0, 0].

32
00:01:44,790 --> 00:01:46,520
So you should divide
it by its length,

33
00:01:46,520 --> 00:01:54,240
but its length is 1, so
this is simply [1, 0, 0].

34
00:01:54,240 --> 00:01:55,280
q_1 is done.

35
00:01:55,280 --> 00:01:56,360
Now let's do q_2.

36
00:01:59,000 --> 00:02:03,560
So with q_2, I will
start with my vector b.

37
00:02:03,560 --> 00:02:05,520
And then I want to
make it, well first

38
00:02:05,520 --> 00:02:09,210
of all, orthogonal to what I
already have, which is q_1.

39
00:02:09,210 --> 00:02:14,070
For that, I'm going to subtract
off from b the projection of b

40
00:02:14,070 --> 00:02:16,290
onto q_1.

41
00:02:16,290 --> 00:02:21,970
Minus b dot q_1 times q_1.

42
00:02:21,970 --> 00:02:25,010
Usually, when you're doing the
projection of a vector onto

43
00:02:25,010 --> 00:02:27,630
another vector, you have
to divide it by the length

44
00:02:27,630 --> 00:02:29,600
of, in this case, q1_.

45
00:02:29,600 --> 00:02:34,700
But because q_1 has length
1, you don't need to do that.

46
00:02:34,700 --> 00:02:36,260
So what will it be here?

47
00:02:36,260 --> 00:02:46,770
Well b dot q_1 is going
to be-- b is [2, 0, 3],

48
00:02:46,770 --> 00:02:53,360
minus, b dot q_1 will
be 2, and [1, 0, 0].

49
00:02:53,360 --> 00:02:59,870
So this will be [0, 0, 3].

50
00:02:59,870 --> 00:03:02,320
This vector is
orthogonal to this one,

51
00:03:02,320 --> 00:03:04,440
and you can check by
doing your dot product.

52
00:03:04,440 --> 00:03:07,250
It should be 0, and it is.

53
00:03:07,250 --> 00:03:11,780
We need it also to be length
1, because we want these two

54
00:03:11,780 --> 00:03:13,920
vectors to be orthonormal.

55
00:03:13,920 --> 00:03:15,810
So this is not actually q_2.

56
00:03:15,810 --> 00:03:20,200
Let's call this one
q_2 prime, and set q_2

57
00:03:20,200 --> 00:03:25,520
equals to q_2 prime divided by
its length, which in this case

58
00:03:25,520 --> 00:03:26,755
is 3.

59
00:03:29,580 --> 00:03:32,890
[0, 0, 1].

60
00:03:32,890 --> 00:03:35,140
That's my vector q_2.

61
00:03:35,140 --> 00:03:40,960
Let's go on to the
third one, q_3.

62
00:03:40,960 --> 00:03:45,220
Well again, I start
with my third vector, c.

63
00:03:45,220 --> 00:03:49,370
And then I want to subtract
the projection of c

64
00:03:49,370 --> 00:03:52,620
onto q_1 and onto q_2,
and that will give me

65
00:03:52,620 --> 00:03:56,830
a q_3 that is orthogonal
to both q_1 and q_2.

66
00:03:56,830 --> 00:04:09,270
c minus c dot q_1 times q_1
minus c dot q_2 dot q_2.

67
00:04:09,270 --> 00:04:17,910
This will be c, [4, 5, 6]
minus, q_1 was [1 0 0],

68
00:04:17,910 --> 00:04:28,590
so 4, times [1, 0, 0],
minus-- q2-- 6 [0, 0, 1].

69
00:04:28,590 --> 00:04:34,930
So this vector
will be [0, 5,  0].

70
00:04:34,930 --> 00:04:39,990
And once again, this one is
orthogonal to q_1 and q_2,

71
00:04:39,990 --> 00:04:42,270
but it is not norm 1 yet.

72
00:04:42,270 --> 00:04:47,350
So q_3-- I'll call
that one q_3 prime,

73
00:04:47,350 --> 00:04:51,630
and I'll set q_3 equal
to q_3 prime divided

74
00:04:51,630 --> 00:04:55,770
by its length which is 5.

75
00:04:55,770 --> 00:04:59,480
q_3 is the vector [0, 1, 0].

76
00:05:04,310 --> 00:05:05,720
One thing that I
want you to note

77
00:05:05,720 --> 00:05:10,410
is that my vectors q_1, q_2,
q_3 are very nice in this case.

78
00:05:10,410 --> 00:05:13,230
Usually, when you perform
Gram-Schmidt orthogonalization,

79
00:05:13,230 --> 00:05:14,930
you end up with
lots of square roots

80
00:05:14,930 --> 00:05:17,100
because you're
dividing by the length.

81
00:05:17,100 --> 00:05:22,190
In this case, we have
everything is integers,

82
00:05:22,190 --> 00:05:25,550
which is, well, very lucky.

83
00:05:25,550 --> 00:05:28,440
Next part of the problem
is we want to write the QR

84
00:05:28,440 --> 00:05:37,210
decomposition of the matrix A. A
equals Q*R. Well, the matrix A,

85
00:05:37,210 --> 00:05:38,660
you already know what it is.

86
00:05:38,660 --> 00:05:50,450
It is the matrix
[1, 2, 4; 0, 0,  5; 0, 3, 6].

87
00:05:50,450 --> 00:05:54,620
And Q you want to be
an orthogonal matrix.

88
00:05:54,620 --> 00:05:56,680
Like I said before,
an orthogonal matrix

89
00:05:56,680 --> 00:06:00,130
has orthonormal vectors
for its columns.

90
00:06:00,130 --> 00:06:02,590
And we already
have such a matrix.

91
00:06:02,590 --> 00:06:06,160
It's the matrix that has q_1,
q_2, and q_3 as its column

92
00:06:06,160 --> 00:06:07,640
vectors.

93
00:06:07,640 --> 00:06:15,250
1, 0, 0; 0, 0, 1; and 0, 1, 0.

94
00:06:15,250 --> 00:06:18,400
Now, we need an upper
triangular matrix

95
00:06:18,400 --> 00:06:21,450
that makes this equality true.

96
00:06:21,450 --> 00:06:23,120
Take a moment to
look at your matrix

97
00:06:23,120 --> 00:06:27,020
Q. It is simply a
permutation matrix,

98
00:06:27,020 --> 00:06:29,290
so it's very easy to come
up with a matrix that

99
00:06:29,290 --> 00:06:30,850
should fit here.

100
00:06:30,850 --> 00:06:34,220
What this permutation
matrix does is it

101
00:06:34,220 --> 00:06:39,310
exchanges rows two and three
from my matrix R to give you A.

102
00:06:39,310 --> 00:06:40,970
So you know what R must be.

103
00:06:40,970 --> 00:06:47,910
It must be 1, 2, 4; 0, 3, 6--
that's the third row of A--

104
00:06:47,910 --> 00:06:52,420
and then 0, 0, 5, which
is the second row of A.

105
00:06:52,420 --> 00:06:56,420
And indeed, R is
upper triangular.

106
00:06:56,420 --> 00:07:00,810
This is your QR decomposition of
the matrix A. Q is orthogonal,

107
00:07:00,810 --> 00:07:04,059
R is upper triangular.

108
00:07:04,059 --> 00:07:06,100
But let's see where these
numbers in the matrix R

109
00:07:06,100 --> 00:07:08,360
are coming from.

110
00:07:08,360 --> 00:07:15,990
Let me label these columns for
you, a, b, c and q_1, q_2, q_3.

111
00:07:15,990 --> 00:07:18,385
And then I have my matrix R.

112
00:07:18,385 --> 00:07:21,900
You know from the way that
matrix multiplication works

113
00:07:21,900 --> 00:07:27,800
that A is going
to be this matrix

114
00:07:27,800 --> 00:07:30,906
Q times the first
column of R. So you

115
00:07:30,906 --> 00:07:34,370
can regard that as these
numbers in the first column of R

116
00:07:34,370 --> 00:07:37,400
are giving you the linear
coefficients in which you

117
00:07:37,400 --> 00:07:40,730
need to take these
vectors to add up

118
00:07:40,730 --> 00:07:43,130
to A. Let me write that down.

119
00:07:43,130 --> 00:07:50,550
A is going to be 1 times q_1
plus 0 times q_2 plus 0 times

120
00:07:50,550 --> 00:07:51,980
q_3.

121
00:07:51,980 --> 00:07:53,380
Let's do it for b.

122
00:07:53,380 --> 00:07:57,740
The second column of this matrix
will be Q times this column.

123
00:07:57,740 --> 00:08:08,140
So it will be 2 times q_1 plus
3 times q_2 plus 0 times q_3.

124
00:08:08,140 --> 00:08:14,500
And finally, for c I will have
c is equal to this matrix times

125
00:08:14,500 --> 00:08:23,910
this vector, 4*q_1
plus 6*q_2 plus 5*q_3.

126
00:08:23,910 --> 00:08:26,800
Now let's go back and see where
these numbers are showing up.

127
00:08:30,100 --> 00:08:34,320
I wanted to have A
equals 1 times q_1.

128
00:08:34,320 --> 00:08:35,340
Well that's very easy.

129
00:08:35,340 --> 00:08:39,039
It comes from here,
a equals 1 times q_1.

130
00:08:42,830 --> 00:08:44,320
Let's try the second one.

131
00:08:44,320 --> 00:08:49,640
b equals 2*q_1 plus 3*q_2.

132
00:08:49,640 --> 00:08:52,650
Well q_2, let's see.

133
00:08:52,650 --> 00:08:54,960
q_2 prime is equal
to 3 times q_2,

134
00:08:54,960 --> 00:08:57,430
so let me write
this here to help.

135
00:08:57,430 --> 00:08:59,900
3*q_2.

136
00:08:59,900 --> 00:09:05,460
Now let me remind you that
b dot q_1 was equal to 2.

137
00:09:05,460 --> 00:09:06,600
Now look at this equation.

138
00:09:06,600 --> 00:09:12,690
You we have b is equal
to 2*q_1 plus 3*q_2,

139
00:09:12,690 --> 00:09:14,570
which is what we wanted.

140
00:09:14,570 --> 00:09:15,680
Let's check q_3.

141
00:09:18,720 --> 00:09:25,450
q_3 prime is equal to 5*q_3 so
let me write that here, 5*q_3.

142
00:09:25,450 --> 00:09:32,545
And now I have c is equal
to-- this number was 4,

143
00:09:32,545 --> 00:09:40,320
and this number was 6-- c is
equal to 4*q_1 plus 6*q_2 plus

144
00:09:40,320 --> 00:09:45,250
5*q_3, which indeed,
is what we wanted.

145
00:09:45,250 --> 00:09:47,280
So this is where these
numbers from the matrix R

146
00:09:47,280 --> 00:09:49,780
are coming from.

147
00:09:49,780 --> 00:09:52,380
And that finishes this problem.

148
00:09:52,380 --> 00:09:56,920
I hope you have a better grasp
of the QR decomposition now.

149
00:09:56,920 --> 00:09:57,420
Bye.

150
00:09:57,420 --> 00:09:59,414
See you next time.