1
00:00:00,182 --> 00:00:01,890
PROFESSOR: Let's
continue our examination

2
00:00:01,890 --> 00:00:06,050
of GCD's and linear combinations
and Euclidean algorithm

3
00:00:06,050 --> 00:00:09,150
by examining what's often
called the extended Euclidean

4
00:00:09,150 --> 00:00:09,660
algorithm.

5
00:00:09,660 --> 00:00:11,150
It's a good name for it.

6
00:00:11,150 --> 00:00:14,100
Its ancient name, dating
back to ancient India,

7
00:00:14,100 --> 00:00:15,780
is the pulverizer.

8
00:00:15,780 --> 00:00:18,640
And we will see what
that does in a moment.

9
00:00:18,640 --> 00:00:20,270
So the theorem that
is the culmination

10
00:00:20,270 --> 00:00:25,480
that we're aiming for is
that the GCD of two numbers

11
00:00:25,480 --> 00:00:27,810
is an integer linear
combination of the two numbers.

12
00:00:27,810 --> 00:00:31,840
That is, the GCD of a and
b is simply sa plus tb,

13
00:00:31,840 --> 00:00:33,720
where s and t are integers.

14
00:00:33,720 --> 00:00:36,330
And what the pulverizer
enables us to do is given a

15
00:00:36,330 --> 00:00:38,390
and b we can find s and t.

16
00:00:38,390 --> 00:00:42,170
In fact, we can find s and
t virtually as efficiently

17
00:00:42,170 --> 00:00:43,410
as the Euclidean algorithm.

18
00:00:43,410 --> 00:00:46,170
It's just by performing
the Euclidean algorithm

19
00:00:46,170 --> 00:00:49,200
and keeping a track of some
additional side information

20
00:00:49,200 --> 00:00:51,130
as it progresses.

21
00:00:51,130 --> 00:00:53,040
Now a corollary of this
fact is that we now

22
00:00:53,040 --> 00:00:55,470
know that the-- if we
want to characterize

23
00:00:55,470 --> 00:00:57,400
the linear combinations
of a and b,

24
00:00:57,400 --> 00:01:02,060
they're precisely the multiples
of the GCD of a and b.

25
00:01:02,060 --> 00:01:05,740
That's because we know that
every factor of both a and b

26
00:01:05,740 --> 00:01:09,440
divides any linear
combination of a and b.

27
00:01:09,440 --> 00:01:12,240
And therefore, the GCD,
which is a factor of a and b,

28
00:01:12,240 --> 00:01:15,660
divides any linear combination.

29
00:01:15,660 --> 00:01:19,460
So that means that any linear
combination is a multiple

30
00:01:19,460 --> 00:01:20,871
of the GCD of a and b.

31
00:01:20,871 --> 00:01:22,620
Of course, once we
know the GCD of a and b

32
00:01:22,620 --> 00:01:25,150
is itself a linear
combination, it

33
00:01:25,150 --> 00:01:28,170
means that you've got all
of the linear combinations

34
00:01:28,170 --> 00:01:30,960
by taking multiples of the GCD.

35
00:01:33,930 --> 00:01:36,030
How do we get s and t?

36
00:01:36,030 --> 00:01:38,440
Well, the basic
idea is that we're

37
00:01:38,440 --> 00:01:41,480
going to find these coefficients
by applying the Euclidean

38
00:01:41,480 --> 00:01:42,260
algorithm.

39
00:01:42,260 --> 00:01:47,234
And as we go, we're going to
be calculating coefficients.

40
00:01:47,234 --> 00:01:48,900
And eventually, when
we're all finished,

41
00:01:48,900 --> 00:01:50,650
we'll wind up with the s and t.

42
00:01:50,650 --> 00:01:53,990
Specifically, let's remember the
Euclidean algorithm starts off

43
00:01:53,990 --> 00:01:56,645
with a and b.

44
00:01:56,645 --> 00:01:59,370
And then it has two registers,
or numbers, x and y,

45
00:01:59,370 --> 00:02:00,740
that it keeps updating.

46
00:02:00,740 --> 00:02:04,260
And the invariant is that
the GCD of the x and y,

47
00:02:04,260 --> 00:02:07,030
that are being continually
updated by the Euclidean

48
00:02:07,030 --> 00:02:08,759
algorithm, stays the same.

49
00:02:08,759 --> 00:02:11,360
It's always the GCD of a and b.

50
00:02:11,360 --> 00:02:13,690
So what we're
going to do is just

51
00:02:13,690 --> 00:02:18,110
keep track of coefficients, call
them c, d, e, and f, such that

52
00:02:18,110 --> 00:02:20,620
the x, whatever
we're up to, we know

53
00:02:20,620 --> 00:02:23,550
how to express as a linear
combination of a and b.

54
00:02:23,550 --> 00:02:26,100
And the y, whatever
y we're up to,

55
00:02:26,100 --> 00:02:29,870
we can also express as a
linear combination of a and b.

56
00:02:29,870 --> 00:02:31,390
So we're going to
be keeping track

57
00:02:31,390 --> 00:02:33,860
of these four
coefficients, c, d, e,

58
00:02:33,860 --> 00:02:35,680
and f that have this property.

59
00:02:35,680 --> 00:02:37,920
This property is going
to be another invariant

60
00:02:37,920 --> 00:02:42,900
of our extended Euclidean
algorithm, or pulverizer.

61
00:02:42,900 --> 00:02:46,300
Well, how do we get initial
values for a c, d, e and f?

62
00:02:46,300 --> 00:02:47,060
Well, that's easy.

63
00:02:47,060 --> 00:02:49,060
At the start, x is a.

64
00:02:49,060 --> 00:02:51,330
And so, c is 1.

65
00:02:51,330 --> 00:02:53,040
And d is 0.

66
00:02:53,040 --> 00:02:56,530
Because a is 1a plus 0b.

67
00:02:56,530 --> 00:02:59,270
Similarly, y is 0a plus 1b.

68
00:02:59,270 --> 00:03:02,774
So we know what these
values of c, d, e, and f

69
00:03:02,774 --> 00:03:04,190
are at the start
of the algorithm.

70
00:03:04,190 --> 00:03:05,920
The question is how
do we update them?

71
00:03:05,920 --> 00:03:08,600
Well, how does a Euclid work?

72
00:03:08,600 --> 00:03:11,950
Well, remember, at the
next step, the value of x

73
00:03:11,950 --> 00:03:13,310
is the old value of y.

74
00:03:13,310 --> 00:03:17,120
So if I had the old
value of y as ea plus fb,

75
00:03:17,120 --> 00:03:20,380
then I clearly
have the next value

76
00:03:20,380 --> 00:03:23,850
of x as the same
linear combination

77
00:03:23,850 --> 00:03:27,240
that y had previously.

78
00:03:27,240 --> 00:03:28,907
What about y next?

79
00:03:28,907 --> 00:03:30,490
Well, at the next
step, the value of y

80
00:03:30,490 --> 00:03:33,520
is simply the
remainder of x and y.

81
00:03:33,520 --> 00:03:35,920
Well, the remainder
of x and y, remember,

82
00:03:35,920 --> 00:03:38,890
is just x minus the
quotient times y

83
00:03:38,890 --> 00:03:42,670
where the quotient is the
quotient of x divided by y.

84
00:03:42,670 --> 00:03:47,520
So this is equal to the
remainder of x and y.

85
00:03:47,520 --> 00:03:52,060
And that means that
since I also have

86
00:03:52,060 --> 00:03:54,740
x expressed as a
linear combination,

87
00:03:54,740 --> 00:03:58,140
this x minus qy is simply
this linear combination

88
00:03:58,140 --> 00:04:02,080
for x minus the quotient number
times the linear combination

89
00:04:02,080 --> 00:04:02,740
for y.

90
00:04:02,740 --> 00:04:04,810
Well, the difference of
two linear combinations

91
00:04:04,810 --> 00:04:06,180
is a linear combination.

92
00:04:06,180 --> 00:04:09,050
So just combining
coefficients what I discovered

93
00:04:09,050 --> 00:04:13,135
is that the way to express y
next as a linear combination

94
00:04:13,135 --> 00:04:17,320
of a and b is just to combine
the previous coefficients, c,

95
00:04:17,320 --> 00:04:21,600
d, e, and f with the
quotient in this way.

96
00:04:21,600 --> 00:04:23,785
And that's all there is to it.

97
00:04:23,785 --> 00:04:25,910
Well, let's work out an
example to see how it goes.

98
00:04:25,910 --> 00:04:30,664
Suppose that a is
899 and b is 493.

99
00:04:30,664 --> 00:04:32,705
These were numbers that
we had previously applied

100
00:04:32,705 --> 00:04:34,710
the Euclidean algorithm to.

101
00:04:34,710 --> 00:04:37,340
So now, what we're
doing is observing--

102
00:04:37,340 --> 00:04:39,500
I'm going to begin by
calculating the remainder.

103
00:04:39,500 --> 00:04:41,876
But this time, when
calculating the remainder,

104
00:04:41,876 --> 00:04:43,250
let's keep track
of the quotient.

105
00:04:43,250 --> 00:04:44,980
So I'm going to
find the remainder

106
00:04:44,980 --> 00:04:47,960
of 899 divided by 493.

107
00:04:47,960 --> 00:04:49,110
It's 406.

108
00:04:49,110 --> 00:04:50,630
And the quotient is 1.

109
00:04:50,630 --> 00:04:54,650
That is 899 is 1
times 493 plus 406.

110
00:04:54,650 --> 00:04:55,660
What does that tell me?

111
00:04:55,660 --> 00:05:01,510
Well, 406 then is-- remember
899 is a and 493 is b.

112
00:05:01,510 --> 00:05:04,630
I'm discovering that the
first remainder, 406, is

113
00:05:04,630 --> 00:05:07,960
1 times a plus minus 1 times b.

114
00:05:07,960 --> 00:05:09,500
So now, I have that
first remainder

115
00:05:09,500 --> 00:05:13,490
expressed as the desired
linear combination of a and b.

116
00:05:13,490 --> 00:05:14,790
Well, what's next?

117
00:05:14,790 --> 00:05:19,600
Well, now that I've
got 406 and 493,

118
00:05:19,600 --> 00:05:21,470
I'm supposed to
take the remainder

119
00:05:21,470 --> 00:05:23,620
of 493 divided by 406.

120
00:05:23,620 --> 00:05:25,100
Well, that's 87.

121
00:05:25,100 --> 00:05:29,640
In fact, 493 has a quotient
1 times 406 plus 87.

122
00:05:29,640 --> 00:05:33,850
So that tells me that 87 is
this number minus that number.

123
00:05:33,850 --> 00:05:40,570
87 is 493 minus 406.

124
00:05:40,570 --> 00:05:42,480
Well, remember, 493 is b.

125
00:05:42,480 --> 00:05:46,010
So 87 is 1 times b
minus 1 times 406.

126
00:05:46,010 --> 00:05:47,400
But wait, look up here.

127
00:05:47,400 --> 00:05:50,713
406, I know how to express it
as a linear combination of a

128
00:05:50,713 --> 00:05:51,370
and b.

129
00:05:51,370 --> 00:05:56,110
So let's replace the
406 by 1a plus minus 1b.

130
00:05:56,110 --> 00:05:59,340
And what I'm going to
wind up with-- remember,

131
00:05:59,340 --> 00:06:03,770
it's a minus minus, so
I wind up contributing

132
00:06:03,770 --> 00:06:05,690
an a and an extra b.

133
00:06:05,690 --> 00:06:09,870
And I wind up with
a minus a plus 2b.

134
00:06:09,870 --> 00:06:10,940
Said that wrong.

135
00:06:10,940 --> 00:06:12,040
The a is getting negated.

136
00:06:12,040 --> 00:06:13,750
But you can check my algebra.

137
00:06:13,750 --> 00:06:17,230
So there we are with the linear
combination that expresses

138
00:06:17,230 --> 00:06:18,890
the next remainder, 87.

139
00:06:18,890 --> 00:06:20,260
All right, let's continue.

140
00:06:20,260 --> 00:06:21,990
After this, what
we're supposed to do

141
00:06:21,990 --> 00:06:29,800
is find the quotient of 406
by 87 and the remainder.

142
00:06:29,800 --> 00:06:32,770
So when you divide 406 by
87 you get a quotient of 4

143
00:06:32,770 --> 00:06:36,300
and a remainder of 58,
which means the remainder 58

144
00:06:36,300 --> 00:06:39,810
is 406 minus 4 times 887.

145
00:06:39,810 --> 00:06:44,450
But now, looking above, I
have the coefficients of 406

146
00:06:44,450 --> 00:06:45,510
for a and b.

147
00:06:45,510 --> 00:06:50,106
And I have the coefficients
for 87 for a and b here.

148
00:06:50,106 --> 00:06:52,230
And so, I have to multiply
those by 4 and add them.

149
00:06:52,230 --> 00:06:56,040
I wind up that the way to
express 58 in terms of a and b

150
00:06:56,040 --> 00:06:59,670
is 5a plus minus 9b.

151
00:06:59,670 --> 00:07:03,130
And next, I'm supposed
to find the remainder

152
00:07:03,130 --> 00:07:05,660
of 87 divided by 58.

153
00:07:05,660 --> 00:07:06,670
The quotient's 1.

154
00:07:06,670 --> 00:07:08,900
The remainder is 29.

155
00:07:08,900 --> 00:07:14,440
And that means that 29 is 1
times 87 minus 1 times 58.

156
00:07:14,440 --> 00:07:18,930
Looking back, I see how to
express 87 in terms of a and b

157
00:07:18,930 --> 00:07:20,690
and 58 in terms of a and b.

158
00:07:20,690 --> 00:07:23,400
I can just combine
those expressions

159
00:07:23,400 --> 00:07:29,500
to wind up with 29 is minus
6 times a plus 11 times b.

160
00:07:29,500 --> 00:07:34,520
Next, I have to take the
quotient of 58 divided by 29.

161
00:07:34,520 --> 00:07:37,100
Well, the quotient is 2,
but the cool thing now

162
00:07:37,100 --> 00:07:38,730
is the remainder is 0.

163
00:07:38,730 --> 00:07:41,950
That's the stopping condition
for the Euclidean algorithm.

164
00:07:41,950 --> 00:07:45,290
It means that the answer is 29.

165
00:07:45,290 --> 00:07:46,610
There's no remainder anymore.

166
00:07:46,610 --> 00:07:50,460
So the GCD of 29 and 0 is 29.

167
00:07:50,460 --> 00:07:53,130
The final GCD, then,
we finished is 29.

168
00:07:53,130 --> 00:07:54,370
But look what we got.

169
00:07:54,370 --> 00:07:58,320
In the last step I
had expressed that GCD

170
00:07:58,320 --> 00:08:01,380
as a linear
combination of a and b.

171
00:08:01,380 --> 00:08:02,990
And that's the pulverizer.

172
00:08:02,990 --> 00:08:06,540
I've just figured out that
possible values for s and t

173
00:08:06,540 --> 00:08:09,410
are minus 6 and 11.

174
00:08:09,410 --> 00:08:11,870
And this is a perfectly
general procedure

175
00:08:11,870 --> 00:08:15,820
that will always give you
coefficients s and t that

176
00:08:15,820 --> 00:08:21,052
express the GCD of a and
b in terms of a and b.

177
00:08:21,052 --> 00:08:23,600
Now, sometimes it's
technically convenient

178
00:08:23,600 --> 00:08:28,099
be able to control which of
the coefficients are positive

179
00:08:28,099 --> 00:08:28,890
and which negative.

180
00:08:28,890 --> 00:08:30,520
Clearly, if you're
going to combine

181
00:08:30,520 --> 00:08:33,150
a and b that are
both positive numbers

182
00:08:33,150 --> 00:08:40,789
and wind up with
a smaller number

183
00:08:40,789 --> 00:08:43,500
by adding multiples of them,
one of those coefficients

184
00:08:43,500 --> 00:08:44,740
has to be negative.

185
00:08:44,740 --> 00:08:48,320
So in this case, we had the
coefficient of 89 was minus 6.

186
00:08:48,320 --> 00:08:52,290
And the coefficient of b was 11.

187
00:08:52,290 --> 00:08:54,020
And suppose that
I wanted, though,

188
00:08:54,020 --> 00:08:58,000
the first coefficient of a
to be the positive number

189
00:08:58,000 --> 00:09:00,170
and the other one
to be negative.

190
00:09:00,170 --> 00:09:01,170
How can I do that?

191
00:09:01,170 --> 00:09:03,960
Well, there's a pretty trivial
little trick for doing that.

192
00:09:03,960 --> 00:09:06,520
It's ingenious, but it's
immediately verifiable.

193
00:09:06,520 --> 00:09:09,780
How do I get a positive
coefficient for 899?

194
00:09:09,780 --> 00:09:12,740
Well, there's a general way
to get new coefficients.

195
00:09:12,740 --> 00:09:16,940
If you look at minus
6 899 plus 11 493,

196
00:09:16,940 --> 00:09:21,680
if I add any multiple of
493 to the first coordinate,

197
00:09:21,680 --> 00:09:24,820
and I subtract the
same multiple of 899

198
00:09:24,820 --> 00:09:27,130
from the second
coordinate, all I'm doing

199
00:09:27,130 --> 00:09:33,080
is adding 493 times k times
899 to the first term.

200
00:09:33,080 --> 00:09:37,760
And I'm subtracting
493 times 899 times

201
00:09:37,760 --> 00:09:38,690
k for the second term.

202
00:09:38,690 --> 00:09:39,880
They cancel out.

203
00:09:39,880 --> 00:09:42,490
So this linear
combination is going

204
00:09:42,490 --> 00:09:44,580
to be the same as that one.

205
00:09:44,580 --> 00:09:46,050
It's going to be the same GCD.

206
00:09:46,050 --> 00:09:49,260
But now, by adding in any
multiple-- by the way,

207
00:09:49,260 --> 00:09:53,390
k could be positive
or negative-- of 493,

208
00:09:53,390 --> 00:09:57,530
I can make the first coefficient
as big or as small as I like.

209
00:09:57,530 --> 00:09:59,970
In particular, if I
want it to be positive,

210
00:09:59,970 --> 00:10:02,220
might as well take the
smallest value of k that works,

211
00:10:02,220 --> 00:10:03,220
which is 1.

212
00:10:03,220 --> 00:10:07,210
So if I let k be 1, I discover
that I add 493 to minus 6.

213
00:10:07,210 --> 00:10:11,780
I get 487.

214
00:10:11,780 --> 00:10:15,740
And I subtract 899 from
111 and I get minus 888.

215
00:10:15,740 --> 00:10:19,590
And there we are with another
expression for-- this time

216
00:10:19,590 --> 00:10:23,340
s is 487 and t is minus 888.

217
00:10:23,340 --> 00:10:26,500
And the second one is negative
and the first one is positive.

218
00:10:26,500 --> 00:10:29,030
It's going to turn out that
this little trick will enable

219
00:10:29,030 --> 00:10:32,860
us, in the next
video, to come up

220
00:10:32,860 --> 00:10:35,920
with a general solution to the
Die Hard bucket problem, which

221
00:10:35,920 --> 00:10:36,830
is fun.

222
00:10:36,830 --> 00:10:40,510
But let's finish up
the current story.

223
00:10:40,510 --> 00:10:43,690
And the remark is that
the pulverizer is really

224
00:10:43,690 --> 00:10:45,950
another very efficient
algorithm, exactly

225
00:10:45,950 --> 00:10:48,330
the way the Euclidean
algorithm is efficient.

226
00:10:48,330 --> 00:10:51,470
It's basically got the
same number of transitions

227
00:10:51,470 --> 00:10:54,850
when you update the pair
xy to get a new pair, y

228
00:10:54,850 --> 00:10:56,390
remainder of x divided by y.

229
00:10:56,390 --> 00:11:03,400
So it's taking twice log to
the base 2 b transitions.

230
00:11:03,400 --> 00:11:04,970
So it's exponentially efficient.

231
00:11:04,970 --> 00:11:09,922
It's working in the length
and binary of the number b.

232
00:11:09,922 --> 00:11:12,380
Of course, there's a few more
additions and multiplications

233
00:11:12,380 --> 00:11:16,840
per transition for
the extended GCD,

234
00:11:16,840 --> 00:11:19,520
or the pulverizer, than the
ordinary Euclidean algorithm.

235
00:11:19,520 --> 00:11:20,410
So big deal.

236
00:11:20,410 --> 00:11:25,045
It means that the number of
total arithmetic operations of

237
00:11:25,045 --> 00:11:29,530
adds and multiplies is
proportional to the log

238
00:11:29,530 --> 00:11:31,425
to the base 2 of b.

239
00:11:31,425 --> 00:11:32,590
I said here 6.

240
00:11:32,590 --> 00:11:34,060
I think it's actually like 10.

241
00:11:34,060 --> 00:11:37,350
But the main thing is it's
a small constant times

242
00:11:37,350 --> 00:11:39,140
the log to the base 2 of b.

243
00:11:39,140 --> 00:11:42,630
The pulverizer is a very
efficient algorithm as well as

244
00:11:42,630 --> 00:11:43,956
the Euclidean algorithm.

245
00:11:43,956 --> 00:11:45,870
And those are going
to be crucial facts

246
00:11:45,870 --> 00:11:47,750
that we'll build on.