1
00:00:00,890 --> 00:00:03,320
PROFESSOR: Random walks
provide probabilistic models

2
00:00:03,320 --> 00:00:05,399
for a bunch of settings.

3
00:00:05,399 --> 00:00:06,940
In fact, we've seen
a couple already,

4
00:00:06,940 --> 00:00:09,570
so let's examine what
they are in general.

5
00:00:09,570 --> 00:00:13,650
So the set up for a random walk
is that you have a digraph,

6
00:00:13,650 --> 00:00:16,430
and we can also often think
and talk about the digraph

7
00:00:16,430 --> 00:00:20,160
as though it was a state diagram
for a machine with state,

8
00:00:20,160 --> 00:00:23,980
so here's a three-state
digraph-- blue, orange,

9
00:00:23,980 --> 00:00:27,740
and green-- and the part
that becomes probabilistic

10
00:00:27,740 --> 00:00:31,520
is that we think of the
process of which edge

11
00:00:31,520 --> 00:00:34,020
to follow when you're
at a given state

12
00:00:34,020 --> 00:00:35,675
is made probabilistically.

13
00:00:35,675 --> 00:00:37,050
And the only rules
are that we're

14
00:00:37,050 --> 00:00:41,750
going to assign probabilities
to the edges in a way

15
00:00:41,750 --> 00:00:43,690
like this where, for
example, what I'm

16
00:00:43,690 --> 00:00:46,930
telling you is there's a 1/3
probability that I'll follow

17
00:00:46,930 --> 00:00:50,980
the edge from O to O, and a 2/3
probability that I'll follow

18
00:00:50,980 --> 00:00:53,590
the edge from O to
green, and the rule

19
00:00:53,590 --> 00:00:55,580
is simply that the sum
of the probabilities

20
00:00:55,580 --> 00:00:58,320
on the outgoing edges
has to sum to 1.

21
00:00:58,320 --> 00:01:01,060
So let's fill in the rest
of the graph in a legal way.

22
00:01:01,060 --> 00:01:03,840
So here we have B
with a 1/2 probability

23
00:01:03,840 --> 00:01:08,560
of going from B to B, a 1/4 from
B to O, and a quarter from B

24
00:01:08,560 --> 00:01:13,290
to G. And the green state
is certain in one step

25
00:01:13,290 --> 00:01:15,590
if you're at green
to go to blue next.

26
00:01:15,590 --> 00:01:16,980
There's only one edge out.

27
00:01:16,980 --> 00:01:20,510
It has probability 1.

28
00:01:20,510 --> 00:01:23,260
Now, gambler's ruin can
be seen as an example

29
00:01:23,260 --> 00:01:24,510
of this kind of a random walk.

30
00:01:24,510 --> 00:01:26,730
The states where the amount
of money that you had,

31
00:01:26,730 --> 00:01:29,990
ranging from 0 when you're
bankrupt to T when you've

32
00:01:29,990 --> 00:01:33,080
reached your target and N
is the start state, which

33
00:01:33,080 --> 00:01:38,020
is your initial stake, the
green edges are weighted

34
00:01:38,020 --> 00:01:41,110
with a probability
P of winning a bet,

35
00:01:41,110 --> 00:01:45,440
so we have transitions
from K to K plus 1 for K

36
00:01:45,440 --> 00:01:48,930
less than T, with a
weight probability P

37
00:01:48,930 --> 00:01:51,840
and, likewise, the red edges are
weighted with the probability

38
00:01:51,840 --> 00:01:55,340
of losing a bet,
Q, or 1 minus P.

39
00:01:55,340 --> 00:01:59,270
So there is a digraph,
or a state machine,

40
00:01:59,270 --> 00:02:03,480
that describes the
gambler's ruin problem

41
00:02:03,480 --> 00:02:06,124
as a probabilistic
walk on a graph.

42
00:02:06,124 --> 00:02:07,790
And the typical kind
of question that we

43
00:02:07,790 --> 00:02:11,060
would ask about a
random walk on a graph

44
00:02:11,060 --> 00:02:13,730
would be, what's the
probability of reaching T,

45
00:02:13,730 --> 00:02:18,430
the target, before reaching
0, bankrupt, given that you're

46
00:02:18,430 --> 00:02:23,020
starting at some state And.

47
00:02:23,020 --> 00:02:27,910
So in walks come up in a bunch
of quite different settings.

48
00:02:27,910 --> 00:02:30,500
For example, in
physics Brownian motion

49
00:02:30,500 --> 00:02:34,400
is the random motion
of a particle that's

50
00:02:34,400 --> 00:02:39,350
being buffeted by
atomic forces, and its

51
00:02:39,350 --> 00:02:41,490
modeled by saying
that this particle can

52
00:02:41,490 --> 00:02:43,740
move in any
direction in 3D space

53
00:02:43,740 --> 00:02:46,850
and chosen uniformly at random.

54
00:02:46,850 --> 00:02:49,670
And the theory of
Brownian motion--

55
00:02:49,670 --> 00:02:52,350
it had been observed first
without being understood.

56
00:02:52,350 --> 00:02:54,930
Einstein was the first one
to come up with a random walk

57
00:02:54,930 --> 00:02:58,650
model and corresponding theorems
about the behavior of particles

58
00:02:58,650 --> 00:03:00,000
under Brownian motion.

59
00:03:00,000 --> 00:03:02,441
In fact, that was one of the
main components of his Nobel

60
00:03:02,441 --> 00:03:02,940
Prize.

61
00:03:02,940 --> 00:03:05,510
He wasn't given a Nobel
Prize for relativity

62
00:03:05,510 --> 00:03:10,920
at the time, because it had
not yet been firmly proven,

63
00:03:10,920 --> 00:03:13,970
although it was
widely celebrated.

64
00:03:13,970 --> 00:03:15,690
Another case is in finance.

65
00:03:15,690 --> 00:03:19,480
We've already seen how gambler's
ruin reflects, or seems

66
00:03:19,480 --> 00:03:25,220
to reflect, the biased
random oscillation of stock

67
00:03:25,220 --> 00:03:29,570
prices over time,
and we will see

68
00:03:29,570 --> 00:03:33,000
at the end of this set
of videos an application

69
00:03:33,000 --> 00:03:36,060
of random walks on
a graph to model

70
00:03:36,060 --> 00:03:46,050
web search and clustering of a
focus on vertices in a digraph.

71
00:03:46,050 --> 00:03:47,829
So the general
kinds of questions

72
00:03:47,829 --> 00:03:50,370
that come up when you're talking
about random walks on graphs

73
00:03:50,370 --> 00:03:54,190
are illustrated by this
simple three-state example

74
00:03:54,190 --> 00:03:55,890
with blue, orange, and green.

75
00:03:55,890 --> 00:03:58,690
We might ask, for
example, starting

76
00:03:58,690 --> 00:04:01,710
at state B, what's the
probability of reaching

77
00:04:01,710 --> 00:04:04,030
state O in seven steps?

78
00:04:04,030 --> 00:04:05,780
And that would be easy
enough to calculate

79
00:04:05,780 --> 00:04:08,750
in this small example, but it
would be a typical question.

80
00:04:08,750 --> 00:04:10,720
A more interesting
general question would be,

81
00:04:10,720 --> 00:04:12,720
what's the average
number of steps that it

82
00:04:12,720 --> 00:04:14,980
takes to get from B to O?

83
00:04:14,980 --> 00:04:17,790
I mean, you could with
probability of 1/4,

84
00:04:17,790 --> 00:04:22,230
you go there in one step,
but with probability an 1/8,

85
00:04:22,230 --> 00:04:25,610
you go there in three
steps, and so on.

86
00:04:25,610 --> 00:04:29,240
You can calculate again
explicitly and easily enough

87
00:04:29,240 --> 00:04:31,190
what the average number
of steps from B to O

88
00:04:31,190 --> 00:04:33,660
is in this simple
example, and we'll shortly

89
00:04:33,660 --> 00:04:36,260
remark about general ways
to solve that problem.

90
00:04:36,260 --> 00:04:38,670
And finally, you can ask a
gambler's ruin type question,

91
00:04:38,670 --> 00:04:40,750
what's the probability
of starting

92
00:04:40,750 --> 00:04:43,630
at B of getting to G before O?

93
00:04:43,630 --> 00:04:46,540
Well, in this
trivial example, you

94
00:04:46,540 --> 00:04:48,460
can just read off the answer.

95
00:04:48,460 --> 00:04:56,800
You are going to get to G
before O with 50-50 probability,

96
00:04:56,800 --> 00:04:59,910
because from B you have to
go one place or the other

97
00:04:59,910 --> 00:05:02,330
with equal probability.

98
00:05:02,330 --> 00:05:04,079
But in general,
this becomes a more

99
00:05:04,079 --> 00:05:05,620
interesting and
complicated question,

100
00:05:05,620 --> 00:05:10,560
which you can solve by methods
that we're about to lay out.

101
00:05:10,560 --> 00:05:13,170
Let me just remind you
that we've already seen

102
00:05:13,170 --> 00:05:16,190
an interesting and illustrative
example of random walk

103
00:05:16,190 --> 00:05:19,580
on a graph when we were
looking at coin tosses.

104
00:05:19,580 --> 00:05:23,650
The problem, for example,
of if I toss a fair coin

105
00:05:23,650 --> 00:05:26,890
and I wait for three consecutive
tosses that form the pattern

106
00:05:26,890 --> 00:05:31,010
HTH or the pattern
TTH, and I want

107
00:05:31,010 --> 00:05:34,700
to know what's the probability
of winning because HTH comes

108
00:05:34,700 --> 00:05:40,210
before TTH, I can model
that with a-- we said

109
00:05:40,210 --> 00:05:43,950
with an infinite tree
diagram, using our tree

110
00:05:43,950 --> 00:05:46,040
method for forming
probability spaces,

111
00:05:46,040 --> 00:05:49,230
but the tree was very
recursively defined.

112
00:05:49,230 --> 00:05:52,780
Sub-trees were isomorphic
to the original tree, which

113
00:05:52,780 --> 00:05:56,110
allowed us, in fact, to come
up with a finite description

114
00:05:56,110 --> 00:06:00,410
of the infinite tree that
amounted to a finite state

115
00:06:00,410 --> 00:06:01,590
machine or finite graph.

116
00:06:01,590 --> 00:06:05,240
So let's look at that
example in more detail.

117
00:06:05,240 --> 00:06:07,580
If I'm trying to model
the coin flipping thing,

118
00:06:07,580 --> 00:06:10,230
we start off in a state
where the previous two

119
00:06:10,230 --> 00:06:11,180
flips don't exist.

120
00:06:11,180 --> 00:06:13,120
I haven't flipped anything yet.

121
00:06:13,120 --> 00:06:15,910
The state's going to record
the values of the previous two

122
00:06:15,910 --> 00:06:18,640
flips, and with no
prior flips, there's

123
00:06:18,640 --> 00:06:21,620
a 50-50 chance that the first
flip will be H, in which case

124
00:06:21,620 --> 00:06:24,910
I'm in the state with just
an H and nothing preceding,

125
00:06:24,910 --> 00:06:28,196
or there's a 50-50 chance
I flip a T, in which case

126
00:06:28,196 --> 00:06:29,570
I'm in the state
in which there's

127
00:06:29,570 --> 00:06:31,890
been a T and nothing previous.

128
00:06:31,890 --> 00:06:33,850
But I can already
say something then

129
00:06:33,850 --> 00:06:38,720
about the probability of
tossing HTH before TTH,

130
00:06:38,720 --> 00:06:40,250
namely the probability
of winning.

131
00:06:40,250 --> 00:06:42,392
The probability of
winning is, of course,

132
00:06:42,392 --> 00:06:44,850
the probability of winning,
given that I start at the start

133
00:06:44,850 --> 00:06:48,110
state with no prior
flips, but the probability

134
00:06:48,110 --> 00:06:51,290
that I win starting
here is simply

135
00:06:51,290 --> 00:06:57,920
the probability that I win
starting at the state nothing H

136
00:06:57,920 --> 00:07:00,540
and where the probability
that I win at the state

137
00:07:00,540 --> 00:07:04,550
started nothing T, with the two
probabilities weighted equally

138
00:07:04,550 --> 00:07:08,030
since this is a fair coin,
and there's a 50-50 chance

139
00:07:08,030 --> 00:07:08,780
of going each way.

140
00:07:08,780 --> 00:07:10,490
That is the
probability of winning

141
00:07:10,490 --> 00:07:14,900
given no prior tosses is 1/2
the probability of winning

142
00:07:14,900 --> 00:07:17,490
if the first toss
is an H plus 1/2

143
00:07:17,490 --> 00:07:19,670
the probability of winning
if the first toss is

144
00:07:19,670 --> 00:07:23,090
a T. This is just an
application of the Law of Total

145
00:07:23,090 --> 00:07:24,770
Probability.

146
00:07:24,770 --> 00:07:28,030
So continuing in this way, let's
expand more of the digraph.

147
00:07:28,030 --> 00:07:31,935
So suppose that I have tossed a
head and then after that I toss

148
00:07:31,935 --> 00:07:36,230
a head, and I go to
state HH or I toss a T,

149
00:07:36,230 --> 00:07:37,510
and I go to state HT.

150
00:07:37,510 --> 00:07:41,070
So here I'm just recording
the previous two flips,

151
00:07:41,070 --> 00:07:42,710
with the most recent
one on the right.

152
00:07:45,730 --> 00:07:48,460
This structure of
the state diagram

153
00:07:48,460 --> 00:07:49,960
tells us that if I
want to know what

154
00:07:49,960 --> 00:07:53,690
the probability of winning,
given that I flipped exactly

155
00:07:53,690 --> 00:07:56,470
one head at the start,
the probability is simply

156
00:07:56,470 --> 00:07:58,320
by, again, total probability.

157
00:07:58,320 --> 00:08:01,620
The probability of winning
from HH weighted by 1/2

158
00:08:01,620 --> 00:08:05,050
and the probability of winning
from HT weighted by 1/2,

159
00:08:05,050 --> 00:08:08,940
and I wind up again with a
simple linear equation that

160
00:08:08,940 --> 00:08:12,090
connects the probability
of winning in one state

161
00:08:12,090 --> 00:08:16,590
with the probability of winning
in the states that it goes to.

162
00:08:16,590 --> 00:08:18,880
Let's continue and
do another example.

163
00:08:18,880 --> 00:08:21,750
So, likewise, if I
expand what happens

164
00:08:21,750 --> 00:08:25,830
after I flip a T or an H after
having flipped the first head,

165
00:08:25,830 --> 00:08:28,450
I get a corresponding
equation that the probability

166
00:08:28,450 --> 00:08:33,580
of winning after a single
tail is the same as 1/2

167
00:08:33,580 --> 00:08:35,799
the probability of winning
with a tail followed

168
00:08:35,799 --> 00:08:38,409
by an H or a tail
followed by a tail.

169
00:08:38,409 --> 00:08:40,470
This is a more interesting
part of the diagram,

170
00:08:40,470 --> 00:08:43,710
where suppose that my past
two flips have been two H's.

171
00:08:43,710 --> 00:08:46,650
Well, if I flip an
H again, then I'm

172
00:08:46,650 --> 00:08:50,560
back in state where the previous
two flips where H's, or if I

173
00:08:50,560 --> 00:08:53,580
flip a T, then I'm
in this state HT,

174
00:08:53,580 --> 00:08:58,030
where the previous flips were
an H and a T, in that order.

175
00:08:58,030 --> 00:08:59,920
And that tells me,
if I want to know

176
00:08:59,920 --> 00:09:02,450
about the probability
of winning given HH,

177
00:09:02,450 --> 00:09:04,880
now it's the
probability of winning

178
00:09:04,880 --> 00:09:09,440
given HH plus the probability
of winning given HT.

179
00:09:09,440 --> 00:09:12,990
And, again, it's a linear
equation connecting up

180
00:09:12,990 --> 00:09:14,770
the probability of
winning in one state

181
00:09:14,770 --> 00:09:17,700
with the probability of winning
in other states, possibly

182
00:09:17,700 --> 00:09:20,860
itself, but there's
no circularity here.

183
00:09:20,860 --> 00:09:23,540
It's just a system
of linear equations.

184
00:09:23,540 --> 00:09:25,610
Well, there's what the
whole diagram looks like.

185
00:09:25,610 --> 00:09:28,560
In particular, once
you flipped HT,

186
00:09:28,560 --> 00:09:33,010
if you then flip an H you've won
because you got to HTH first,

187
00:09:33,010 --> 00:09:35,020
and you stay in the
win state forever

188
00:09:35,020 --> 00:09:38,830
or, alternatively, once you
flip TT, if you flip an H

189
00:09:38,830 --> 00:09:41,870
you've lost because
TT has come up first.

190
00:09:41,870 --> 00:09:44,730
If you flip the T again,
you stay in state TT.

191
00:09:44,730 --> 00:09:46,490
And what we can say
is the probability

192
00:09:46,490 --> 00:09:48,690
of winning if we're
in the win state is 1,

193
00:09:48,690 --> 00:09:51,970
and the probability of winning
if you're in a lose state is 0.

194
00:09:51,970 --> 00:09:54,620
And overall, I simply
have this system

195
00:09:54,620 --> 00:09:57,270
of linear equations for
the probability of winning

196
00:09:57,270 --> 00:09:59,330
in one state given
other states, and I

197
00:09:59,330 --> 00:10:02,250
can solve these linear
equations to find

198
00:10:02,250 --> 00:10:04,640
the probability of winning
in the start state, which

199
00:10:04,640 --> 00:10:07,580
is simply the
probability of winning.

200
00:10:07,580 --> 00:10:11,380
So looking back at our
questions for random walks,

201
00:10:11,380 --> 00:10:13,380
where we ask whether the
probability of reaching

202
00:10:13,380 --> 00:10:16,550
O in seven steps starting at B,
what's the probability of that?

203
00:10:16,550 --> 00:10:18,900
What's the average number
of steps to go from B to O?

204
00:10:18,900 --> 00:10:20,480
What's the probability
of reaching G

205
00:10:20,480 --> 00:10:22,290
before O, starting at B?

206
00:10:22,290 --> 00:10:26,880
In every case, these questions
can be formulated simply

207
00:10:26,880 --> 00:10:29,090
as solving systems
of linear equations

208
00:10:29,090 --> 00:10:31,450
whose structure
directly reflects

209
00:10:31,450 --> 00:10:34,470
the structure of the digraph.