1
00:00:01,270 --> 00:00:03,910
In this last video,
we illustrate

2
00:00:03,910 --> 00:00:06,210
how to use the techniques
we have recently

3
00:00:06,210 --> 00:00:08,880
learned in order to
answer some questions

4
00:00:08,880 --> 00:00:12,250
about the following
classical problem--

5
00:00:12,250 --> 00:00:16,410
consider a gambler
putting a bet of $1

6
00:00:16,410 --> 00:00:20,380
in a game that has a pay off
of one dollar if she wins.

7
00:00:20,380 --> 00:00:23,320
We assume that this
is a fair game,

8
00:00:23,320 --> 00:00:27,320
so the probability of winning
$1 on each play of the game

9
00:00:27,320 --> 00:00:28,250
is one-half.

10
00:00:28,250 --> 00:00:32,910
And so the probability of
losing the bet is also one-half.

11
00:00:32,910 --> 00:00:36,150
Suppose that she
starts with i dollars

12
00:00:36,150 --> 00:00:38,930
and continues to play
the game until either she

13
00:00:38,930 --> 00:00:44,380
reaches a goal of n dollars, or
she has no money left, whatever

14
00:00:44,380 --> 00:00:46,950
comes first.

15
00:00:46,950 --> 00:00:49,600
Let us consider a
first question, which

16
00:00:49,600 --> 00:00:54,070
is the following-- what is the
probability that she ends up

17
00:00:54,070 --> 00:00:58,840
with having her
goal of n dollars?

18
00:00:58,840 --> 00:01:02,830
Now, how to go about
solving this problem?

19
00:01:02,830 --> 00:01:05,080
Can we think of a Markov
chain representation

20
00:01:05,080 --> 00:01:07,200
for this problem?

21
00:01:07,200 --> 00:01:09,620
But in that case, what
would be good choices

22
00:01:09,620 --> 00:01:12,629
for the definition
of the states?

23
00:01:12,629 --> 00:01:13,170
Let us think.

24
00:01:13,170 --> 00:01:16,730
At any point in time, the
only relevant information

25
00:01:16,730 --> 00:01:21,150
is the amount of money
the gambler has available.

26
00:01:21,150 --> 00:01:25,970
How she got to that amount
in the past is irrelevant.

27
00:01:25,970 --> 00:01:31,030
And if this amount is
neither zero nor n,

28
00:01:31,030 --> 00:01:32,729
then she will play again.

29
00:01:32,729 --> 00:01:35,400
And the next state
will be a number

30
00:01:35,400 --> 00:01:38,750
which will be increased
or decreased by one unit,

31
00:01:38,750 --> 00:01:42,300
depending on winning
or losing the next bet.

32
00:01:42,300 --> 00:01:46,470
So we can represent the
possible evolution of this game

33
00:01:46,470 --> 00:01:49,930
with the following
probability transition graph.

34
00:01:49,930 --> 00:01:52,979
So we have n plus 1 states.

35
00:01:52,979 --> 00:01:56,160
This is the state where
she loses all her money.

36
00:01:56,160 --> 00:02:00,050
This is the state where
she has i amount of money

37
00:02:00,050 --> 00:02:02,020
before the next betting.

38
00:02:02,020 --> 00:02:05,810
And here, this is the state
n where she reaches her goal

39
00:02:05,810 --> 00:02:07,320
and she can leave.

40
00:02:07,320 --> 00:02:09,840
In terms of the
transition probability,

41
00:02:09,840 --> 00:02:13,650
assuming that you are at a
given time in that state,

42
00:02:13,650 --> 00:02:17,210
that means that you have
i money in your pocket,

43
00:02:17,210 --> 00:02:19,660
and you play the next bet.

44
00:02:19,660 --> 00:02:23,470
With a probability one-half,
you will gain or win.

45
00:02:23,470 --> 00:02:26,600
And so your amount
of money is i plus 1.

46
00:02:26,600 --> 00:02:30,290
Or you lose and your
money is i minus 1.

47
00:02:30,290 --> 00:02:35,340
And you keep repeating until
you reach either n or zero.

48
00:02:35,340 --> 00:02:37,360
And then you stop.

49
00:02:37,360 --> 00:02:42,900
So this is a Markov chain, and
that state 0 and that state n

50
00:02:42,900 --> 00:02:44,579
are absorbing states.

51
00:02:44,579 --> 00:02:48,790
Once you reach them,
you stay there forever.

52
00:02:48,790 --> 00:02:52,360
So what this question is
asking is the probability a

53
00:02:52,360 --> 00:02:57,100
of i of-- starting from i, what
is the probability that you

54
00:02:57,100 --> 00:03:00,390
will end up in that
absorbing state?

55
00:03:00,390 --> 00:03:03,420
And we have done this
calculation previously.

56
00:03:03,420 --> 00:03:07,760
So let us repeat the
technique very briefly.

57
00:03:07,760 --> 00:03:11,630
Clearly here, if you
start with 0 dollars,

58
00:03:11,630 --> 00:03:12,860
you will never reach that.

59
00:03:12,860 --> 00:03:14,670
So it's going to be 0.

60
00:03:14,670 --> 00:03:18,170
On the other hand, if you
start with your desired goal,

61
00:03:18,170 --> 00:03:19,240
you don't play anymore.

62
00:03:19,240 --> 00:03:22,110
So your probability is 1.

63
00:03:22,110 --> 00:03:23,640
Now of course,
what is of interest

64
00:03:23,640 --> 00:03:27,260
is if i is strictly
between 0 and n.

65
00:03:27,260 --> 00:03:30,760
And now the question is how
to calculate that probability.

66
00:03:30,760 --> 00:03:32,550
And we have seen
that the way to do

67
00:03:32,550 --> 00:03:35,320
that is to look
at the situation,

68
00:03:35,320 --> 00:03:39,300
and say let's assume that
you are in that state i.

69
00:03:39,300 --> 00:03:41,500
And what happens next?

70
00:03:41,500 --> 00:03:46,280
Well with a probability 0.5,
you will move to that state.

71
00:03:46,280 --> 00:03:50,760
And now you are in that level
with i plus 1 amount of money.

72
00:03:50,760 --> 00:03:54,410
And what is the probability
that, given that you're here,

73
00:03:54,410 --> 00:03:56,079
you're going to end up in n?

74
00:03:56,079 --> 00:03:58,740
It's going to be a i plus one.

75
00:03:58,740 --> 00:04:01,470
So it's a i plus one.

76
00:04:01,470 --> 00:04:04,410
Plus the other
alternative is that you

77
00:04:04,410 --> 00:04:07,570
are going to lose money
and end up in that state.

78
00:04:07,570 --> 00:04:11,780
And there, the remaining
probability to reach the time n

79
00:04:11,780 --> 00:04:14,860
is a i minus 1.

80
00:04:14,860 --> 00:04:17,870
So you have this
kind of equation.

81
00:04:17,870 --> 00:04:21,329
This is valid for all
i between 0 and n.

82
00:04:21,329 --> 00:04:26,280
And this is a system of
equations that you can solve.

83
00:04:26,280 --> 00:04:28,010
It's not very
difficult to solve.

84
00:04:28,010 --> 00:04:29,910
Actually, you can
look in the textbook.

85
00:04:29,910 --> 00:04:31,760
There will be some
trick to do that.

86
00:04:31,760 --> 00:04:33,420
There are many, many
ways to do that.

87
00:04:33,420 --> 00:04:36,700
We're not going to spend
our time going into details,

88
00:04:36,700 --> 00:04:39,130
but essentially if
you solve that system,

89
00:04:39,130 --> 00:04:42,760
you will see that the
answer will be that a of i

90
00:04:42,760 --> 00:04:46,000
is i over n.

91
00:04:46,000 --> 00:04:49,350
So if you start with
i amount of money,

92
00:04:49,350 --> 00:04:52,640
the probability that you're
going to reach your goal here

93
00:04:52,640 --> 00:04:55,100
is i over n.

94
00:04:55,100 --> 00:04:58,659
So here clearly, if
you're extremely greedy,

95
00:04:58,659 --> 00:05:02,260
and you have a very,
very, very high goal,

96
00:05:02,260 --> 00:05:05,000
that means n is
very, very large--

97
00:05:05,000 --> 00:05:08,600
so large that compared
to your initial amount i,

98
00:05:08,600 --> 00:05:10,647
n can be considered
to be infinity.

99
00:05:10,647 --> 00:05:13,230
Then the probability that you're
going to reach your high goal

100
00:05:13,230 --> 00:05:18,000
will go to 0, where
n goes to infinity.

101
00:05:18,000 --> 00:05:20,060
So again, if you are
extremely greedy,

102
00:05:20,060 --> 00:05:23,450
no matter how much your fixed
amount of initial money is,

103
00:05:23,450 --> 00:05:26,640
the probability that you
will stop the game reaching

104
00:05:26,640 --> 00:05:31,220
your goal is going to
be increasingly small.

105
00:05:31,220 --> 00:05:34,720
And since the other state
is 1 minus this one,

106
00:05:34,720 --> 00:05:37,460
the priority that you're
going to get ruined

107
00:05:37,460 --> 00:05:40,710
is going to be closer to 1.

108
00:05:40,710 --> 00:05:43,850
All right, so we have
that answer here.

109
00:05:43,850 --> 00:05:46,970
What about the next question?

110
00:05:46,970 --> 00:05:48,920
Next question is
the following-- what

111
00:05:48,920 --> 00:05:52,440
would be the expected
wealth at the end?

112
00:05:52,440 --> 00:05:55,190
Again, this is a
Markov chain where

113
00:05:55,190 --> 00:05:56,900
there are two absorbing states.

114
00:05:56,900 --> 00:05:58,960
All the others are transient.

115
00:05:58,960 --> 00:06:01,020
You're guaranteed
with probability 1

116
00:06:01,020 --> 00:06:04,330
that you will reach
either 0 or n.

117
00:06:04,330 --> 00:06:07,910
So it's a valid question to
know once you reach either 0

118
00:06:07,910 --> 00:06:11,330
or n, what is the expected
wealth at the end?

119
00:06:11,330 --> 00:06:14,530
Well, if you arrive
here, it's going to be 0.

120
00:06:14,530 --> 00:06:16,790
And if you arrive here,
it's going to be n.

121
00:06:16,790 --> 00:06:19,450
So the expected
value of that wealth

122
00:06:19,450 --> 00:06:23,320
will be 0 times the probability
of ending in that, plus n times

123
00:06:23,320 --> 00:06:25,120
the probability
of getting there.

124
00:06:25,120 --> 00:06:28,210
So it's going to be that
expression-- 0 times

125
00:06:28,210 --> 00:06:33,040
1 minus a of i,
plus n times a of i.

126
00:06:33,040 --> 00:06:39,970
And here what we then get is
n times i over n, which is i.

127
00:06:39,970 --> 00:06:41,130
Which is quite interesting.

128
00:06:41,130 --> 00:06:45,590
This is exactly how you started.

129
00:06:45,590 --> 00:06:49,570
So in some sense, in expectation
there is no free lunch here.

130
00:06:49,570 --> 00:06:52,500
The next question is--
how long does the gambler

131
00:06:52,500 --> 00:06:54,460
expect to stay in the game?

132
00:06:54,460 --> 00:06:57,850
We know that eventually,
he will either reach 0

133
00:06:57,850 --> 00:06:59,830
or n with probability 1.

134
00:06:59,830 --> 00:07:03,160
The question is-- how
long is it going to take?

135
00:07:03,160 --> 00:07:05,830
Again, we have seen
in a previous video

136
00:07:05,830 --> 00:07:09,670
that this is essentially
calculating the expectation

137
00:07:09,670 --> 00:07:11,130
to absorption.

138
00:07:11,130 --> 00:07:13,030
And we know how to do that.

139
00:07:13,030 --> 00:07:15,550
So let's recap what we had said.

140
00:07:15,550 --> 00:07:19,770
If you define mu of i to be
the expected number of plays

141
00:07:19,770 --> 00:07:22,540
starting from i,
what do you have?

142
00:07:22,540 --> 00:07:25,880
Well, for i equal
to 0 or i equals n,

143
00:07:25,880 --> 00:07:28,010
either way-- if you
start from here,

144
00:07:28,010 --> 00:07:30,390
or you start from here--
the expected number of plays

145
00:07:30,390 --> 00:07:31,440
is 0, right?

146
00:07:31,440 --> 00:07:33,520
Because you're done.

147
00:07:33,520 --> 00:07:37,930
And otherwise, you use the same
kind of derivation that we had.

148
00:07:37,930 --> 00:07:42,470
If you start at i
between 1 and n,

149
00:07:42,470 --> 00:07:48,480
then you will see that mu of i,
after one transition, plus 1,

150
00:07:48,480 --> 00:07:53,570
you will either be in state
i plus 1-- in that case,

151
00:07:53,570 --> 00:07:57,200
this expectation will
be mu i plus 1 --

152
00:07:57,200 --> 00:08:00,450
or you will be in
state i minus 1.

153
00:08:00,450 --> 00:08:05,680
In that case, the
expectation is mu i minus 1.

154
00:08:05,680 --> 00:08:08,470
So this is an equation
that you have,

155
00:08:08,470 --> 00:08:10,370
which is almost the
same as this one,

156
00:08:10,370 --> 00:08:12,830
except that you
have a plus 1 in it.

157
00:08:12,830 --> 00:08:17,140
And as we had discussed
before, in general

158
00:08:17,140 --> 00:08:19,620
this is the kind of
formula that you have.

159
00:08:19,620 --> 00:08:21,680
Now you can solve the system.

160
00:08:21,680 --> 00:08:22,890
I will let you do that.

161
00:08:22,890 --> 00:08:24,300
There are many ways to do this.

162
00:08:24,300 --> 00:08:26,700
But the solution that
you're going to have

163
00:08:26,700 --> 00:08:34,610
is that mu i will be equals
to i times n minus i.

164
00:08:34,610 --> 00:08:35,520
This is the result.

165
00:08:38,390 --> 00:08:40,530
Finally what would
be the case if you

166
00:08:40,530 --> 00:08:45,510
didn't have a fair game-- for
example, unfair to the gambler

167
00:08:45,510 --> 00:08:48,660
or unfair to the casino?

168
00:08:48,660 --> 00:08:52,550
What we mean here is
that the probability p

169
00:08:52,550 --> 00:08:55,930
is different from 0.5.

170
00:08:55,930 --> 00:09:02,160
So here, instead of 0.5,
you have p everywhere.

171
00:09:02,160 --> 00:09:04,570
And here, of course,
you have 1 minus p

172
00:09:04,570 --> 00:09:08,400
everywhere on this side.

173
00:09:08,400 --> 00:09:10,090
So you have a
probability p of winning,

174
00:09:10,090 --> 00:09:14,180
and probability 1 minus
p of losing each bet.

175
00:09:14,180 --> 00:09:17,520
So you might ask the
same question-- well,

176
00:09:17,520 --> 00:09:21,470
for the probability a of
i, you still have 0 here.

177
00:09:21,470 --> 00:09:23,380
You still have 1 here.

178
00:09:23,380 --> 00:09:25,180
The formula that you
would write here,

179
00:09:25,180 --> 00:09:27,330
instead of writing
it this way, it

180
00:09:27,330 --> 00:09:30,550
would be-- you start from
here with a probability p.

181
00:09:30,550 --> 00:09:31,410
You end up here.

182
00:09:31,410 --> 00:09:35,350
And with a probability of 1
minus p, you end up there.

183
00:09:35,350 --> 00:09:38,360
And the expression that
you get for a of i--

184
00:09:38,360 --> 00:09:43,970
if you define r to be 1
minus p over p-- you will see

185
00:09:43,970 --> 00:09:49,820
that a of i is going to be
1 minus r to the power of i

186
00:09:49,820 --> 00:09:55,060
over 1 minus r to the power n.

187
00:09:55,060 --> 00:09:59,510
And what would be the expected
amount of time she will play?

188
00:09:59,510 --> 00:10:03,500
Instead of that equation,
if you solve it,

189
00:10:03,500 --> 00:10:07,830
you would have mu of i
equals r plus 1 divided

190
00:10:07,830 --> 00:10:16,840
by r minus 1 times i minus
n times 1 minus r to the i

191
00:10:16,840 --> 00:10:21,330
divided by 1 minus
r to the power n.

192
00:10:21,330 --> 00:10:25,730
Because you would have here
again p, and 1 minus p here.

193
00:10:28,890 --> 00:10:33,490
And you can see that when p is
strictly less than one-half--

194
00:10:33,490 --> 00:10:36,110
in other words, it's even
worse for this gambler--

195
00:10:36,110 --> 00:10:40,860
then a of i-- which is
the probability of getting

196
00:10:40,860 --> 00:10:45,600
to this favorable
state-- will also go to 0

197
00:10:45,600 --> 00:10:48,290
when n goes to infinity.

198
00:10:48,290 --> 00:10:52,370
And in the case where p is
strictly greater than 0.5--

199
00:10:52,370 --> 00:10:56,930
that means that she has some
favored odd on her favor--

200
00:10:56,930 --> 00:11:01,880
then in that case, this number r
to the power n will go to zero.

201
00:11:01,880 --> 00:11:05,970
And 1 minus r of i will
represent the probability

202
00:11:05,970 --> 00:11:08,730
that she would become
infinitely rich.

203
00:11:08,730 --> 00:11:12,880
In other words, being very
greedy and n going to infinity.

204
00:11:12,880 --> 00:11:16,840
This will go to 0 and 1
minus r to the power of i

205
00:11:16,840 --> 00:11:20,520
is the probability that she
would get infinitely rich.