1
00:00:01,090 --> 00:00:04,510
In this lecture, we introduce
Markov chains, a general class

2
00:00:04,510 --> 00:00:06,910
of random processes
with many applications

3
00:00:06,910 --> 00:00:09,890
dealing with the evolution
of dynamical systems.

4
00:00:09,890 --> 00:00:12,730
They have been used in
physics, chemistry, information

5
00:00:12,730 --> 00:00:15,900
sciences, queuing theory,
internet applications,

6
00:00:15,900 --> 00:00:20,910
statistics, finance, games,
music, genetics, baseball,

7
00:00:20,910 --> 00:00:22,730
history, you name it.

8
00:00:22,730 --> 00:00:26,920
So what make these processes
so powerful and practical?

9
00:00:26,920 --> 00:00:29,660
Well, as opposed to the
Bernoulli and Poisson

10
00:00:29,660 --> 00:00:32,530
processes, which are
memoryless in the sense

11
00:00:32,530 --> 00:00:36,040
that the future does
not depend on the past,

12
00:00:36,040 --> 00:00:39,600
Markov chains are
more elaborate as they

13
00:00:39,600 --> 00:00:42,380
allow the representation
of situations where

14
00:00:42,380 --> 00:00:48,340
the future depends on the
past and, to some extent,

15
00:00:48,340 --> 00:00:53,370
could be predicted
from the past.

16
00:00:53,370 --> 00:00:56,470
More precisely, we are
going to consider models

17
00:00:56,470 --> 00:01:00,610
where the influence of
the past on the future

18
00:01:00,610 --> 00:01:06,110
is summarized by the notion
of a state, which evolves

19
00:01:06,110 --> 00:01:09,680
over time according to some
probability distribution.

20
00:01:09,680 --> 00:01:14,020
That's the link between
the past and the future.

21
00:01:14,020 --> 00:01:17,410
We will restrict ourselves
to discrete time Markov

22
00:01:17,410 --> 00:01:20,270
chains in which
the state changes

23
00:01:20,270 --> 00:01:23,050
at certain discrete time steps.

24
00:01:23,050 --> 00:01:26,660
The state at time t
plus 1, which is here,

25
00:01:26,660 --> 00:01:29,850
is a function of
the state at time t,

26
00:01:29,850 --> 00:01:32,900
and there is some
noise, or randomness.

27
00:01:32,900 --> 00:01:37,060
As another view, this is what
we will cover in this lecture.

28
00:01:37,060 --> 00:01:39,970
We will first introduce
the basic concepts

29
00:01:39,970 --> 00:01:44,310
using the example of a checkout
counter at the supermarket.

30
00:01:44,310 --> 00:01:47,450
We will then abstract
from the example

31
00:01:47,450 --> 00:01:50,530
and give some
general definitions.

32
00:01:50,530 --> 00:01:53,430
Afterwards, we will look
at various questions,

33
00:01:53,430 --> 00:01:56,900
such as predicting what could
happen in the future given

34
00:01:56,900 --> 00:01:59,650
the current state
of our systems.

35
00:01:59,650 --> 00:02:02,800
We will end this lecture
by giving some key

36
00:02:02,800 --> 00:02:06,470
structural properties
of Markov processes.

37
00:02:06,470 --> 00:02:08,389
So let us start.