Session 28: Models vs. Data 1

• Download English-US transcript (PDF)

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high quality educational resources for free. To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

WILLIAM GREEN: So today we're going to talk about Bayesian prior estimation and prior estimation in general. So the last time we were writing down the expressions for the probability of observing a mean measurement if you know what the model is. So let's try to do that again. So suppose I have a model that predicts some observable and it depends on some knobs, and it depends on some parameters.

And suppose because I have great powers of faith that I believe this model is 100% correct with every core of my being. And also because I have tremendous confidence in all the people who built my apparatus, and the knobs that I turn actually correspond to the real values, and I have tremendous confidence in all the literature that reports the parameter values. And so I'm absolutely certain that this is the truth. So we'll start from a position of absolute certainty, and then we'll degrade into doubt as the collector goes on.

So let's start from the position of someone who has absolute faith that this is the truth. Is true. So I have a model, I really believe this model. So for example, I believe that the kilogram weight in the SI Institute in Paris weighs exactly one kilogram. I believe that with every core of my being. I'm completely confident that model is correct. So there are some things I'm really confident on. That's one. And maybe guys have some things you really believe, too. So let's go with things we really believe.

So I plan to conduct some experiments that measure this observable and are related to this model. And so I'm going to do 10 repeats of measuring y. So I'm going to get to the kilogram blob that's in Paris, and I'm going to stick it on my really expensive scale that I really believe is great, and I'm going to measure its weight. And then I'm going to put it back, and then put it back, and put it back, and put it back. And I'm going to get another really great scale that I really believe is great, and I'm going to measure it there, too. So I've got a lot of repeats of measuring the weight of this kilogram, and I believe it's really a kilogram.

But the stupid measurements don't say a kilogram. They say, you know, 1.0003, 0.99995, all kinds of numbers not equal to one kilogram. So now I'm going to try to figure out what the probability is that it would have measured some particular value y. So what is the probability that my experimental is between some value, say, y and y plus dy. So that's a question for you. So what's the probability? Sorry, what?

[INAUDIBLE]

OK, so we think that the probability that y is in this interval given that the model is true. And I know the theta values perfectly, and I know the x values perfectly, is equal to some integral of what? The bounds integral, probably y to y plus dy, believe that? What's the integrand? Sorry, what?

AUDIENCE: The probability [INAUDIBLE] AC function of y.

WILLIAM GREEN: Right, so what is it?

AUDIENCE: [INAUDIBLE]

You wrote it down last time, I think. So this [INAUDIBLE] is large? Standard normal, right? So it should be one over sigma root of 2 pi. Does that sound OK? I mean here. It's probably the same, it's fine.

Yep.

AUDIENCE: What does that notation mean, if your model is true?

WILLIAM GREEN: So this means, given that the model is true, and I know these data values are exactly certain numbers, and the x values are actually certain numbers, what's the probability that I would make a measurement whose average would fall in this interval? So this line means given that this is true, what's the probability of that? OK, is this right? Is this surprising? This is OK?

So this is what I mean. So we say that our probability distribution converges to a Gaussian distribution, this is what we expect. So we expect it to have been large enough for this to be true. Yeah? This is very important. This is like the whole course, actually. This is the whole section, is this one equation. So I just wanted to make sure you really get what this says. And if you don't like the integral, you can make dy really small, and then it's just this times dy. OK?

Actually, this notation's like [INAUDIBLE] I think I should do this way. I should do this. This is a number. Let's get rid of the integral. Let's make dy really small. I'll make it [INAUDIBLE]. That all right? So this is the probability density that we would observe, this is the experimental value y that we observe from the mean, and this is the little with of our tiny little interval. Is that all right? Yes?

AUDIENCE: So is sigma the [INAUDIBLE] on there?

WILLIAM GREEN: Ah, what is sigma? That's a great question. We didn't write down what sigma was. What is sigma?

AUDIENCE: Standard deviation?

WILLIAM GREEN: It's not the standard deviation exactly. Standard deviation of the mean, right? So there's two sigmas. We have the sigma of y, of the measurements, and that's equal to average value of y squared minus. So we just figure that for how many experiments we do, we just compute the average of y squared, the average y, subtract them. That's the variance. And then sigma that I used in that equation there is 1 over n times sigma y. And we call this the variation of the mean, it's the uncertainty in the mean value of y.

And the central limits theorem said that as long as n gets really large, we expect that this should converge to this. And we talked last time about how when n get bigger, these averages don't really change when it gets big. They're just the average. But this number declined as n gets big because of this one over n formula.

And to understand that, suppose I measure the weight, and I measure, it should be around one kilogram. But in fact my measurements are all over here. Lots of measurements. So they have a variance something like this. But if I make a plot of-- as I run, I compute the running average. So when I run the first two points, I get some average value here. After I run 27 points more, the average value is here. After I run 1,000 repeats, the average value is here. It's getting pretty close to this, and the uncertainty in this number's getting smaller and smaller as I'm doing better and better averages. The average more and more repeats. Does that make sense? OK.

So from this key equation, I can derive a lot of things. And it depends what you want to do. So one thing people do a lot is it was called model validation. And what does this mean? It means I have a model, I believe it's true. I have some parameters, I believe they're true. But there are some foolish skeptics out there who don't have the faith that I do. And they think that my model's baloney, or my parameter values are wrong, or something. And so to prove I'm right, I'm going to make some experiments. And I'm going to show that I make a plot that looks like the experiment and model agree. Some of you might have done this in your life, yes? Everybody might make a parity plot or something. You've seen these things before.

Now, this is like a confidence builder. You're trying to get the skeptics out there to believe that there's some evidence to back up your faith that this model is perfect. And what you really want to know is like, if the measurement that I measure, the average for my 10,000 repeated measurements, I expect that this quantity should be pretty big somehow in some way. By then quantitatively saying what that means exactly what's a good fit, what's a bad fit, this is actually kind of a difficult question, and we'll come back to this one. But that's a very common use of this equation is to try to do validation.

Now because it's kind of complicated, most people don't actually do it. So instead what they do is they just plot some data points, and they plot your model curve. And as long as they look good, then you're done. So that's the normal way that it's done in the literature currently. But of course, that's completely unquantitative. It doesn't really say whether the model and the data really agree, it just means they look sort of like each other. So that's like a human qualitative thing.

Now, if the purpose of validation is just to convince humans, then you've done the purpose. Now, if your purpose is to try to quantitatively say something, then you really have to get into this equation, which usually is not done but would be the right thing to do for validation.

Now the alternative view is disproving a model. But I just say that there's several ways this can happen. You can try to disprove a model, but you might also show that the theta values are incorrect. Or you might show that the experiment is wrong. These are all possibilities, reasons why the model and the data might not agree with each other.

So this equation, it only holds if the model is really true, if the parameter values are all perfectly correct, if we know exactly what all the knob values are perfectly. If any of those things are not true, then you should have some discrepancy, and there should be a way to show it. And really what you're showing is that you'd observed some y that is very unlikely to be observed. So probably observing that y is very extremely unlikely if all these other things were true. So if all these things are true, and you compute this value, and this value's very tiny, then it makes you think that it's unlikely that you would have observed that. And therefore, you might try to use that as an argument to say that something must be wrong. The model's wrong, parameters are wrong, the knobs are wrong, something's wrong. My y values are wrong. It could be any of those things.

So this is often the most exciting papers to publish. You publish a paper, you take some model that a lot of people believe. You tell them they're full of baloney, it's completely wrong. My great experiment shows you are completely wrong. And so you'll see a lot of these in Nature. I should warn you, a lot of those get retracted later, a very high retraction rate in Nature. Because they want to publish papers like that that show that the common view is incorrect, and sometimes it's true. But oftentimes the common view is actually correct, and there's something wrong with the experiment, or the interpretation, or how they computed this equation, or whatever. And so actually it turns out the common view is perfectly fine, and it's just that the foolish authors went off on a tangent. And then they have to six months later publish a retraction, by the way, sorry, paper was completely wrong. And so you see a lot of that.

So that's a second kind of thing. And we'll talk more about that a little bit later, too. And then another thing is I'll relax my assumptions. So I'll say, well, I'm sure that the model is true, and I'm sure that my knob settings are perfect, and I know what they are. But I'm not really sure about all the parameters. And therefore I want to use the experiment to try to refine parameter values.

So I'm trying to take my y's that I measure and somehow infer something about the thetas. And this is a very common thing to do. So in my group we've tried to measure the rate coefficient for a reaction. We believe there is value of that theta, and in fact, we probably have an estimate of what it is. But we're not sure of the exact number, and we'd like to do an experiment to refine the number and get it more accurately determined. So that's another useful thing to do.

And this leads into two somewhat different points of view about this. One you've probably done already called least squares fitting. That's one view. And the other is this Bayesian view that I'll tell you about next. So there's sort of A and B. There's one that I'll call Bayesian, and one I'll call least squares. They're sort of related to each other, but not exactly the same conceptually. So I'll try to explain that.

So the Bayesian view is probabilistic, so it's actually pretty straightforward to write down. Remember that we wrote that the probability of A and B is equal to the probability of A times the probability of B given A, and it's also equal to the probability of B times the probability of A given B. And what we have here is one of these conditional probabilities, if the thetas have a certain value, this is a certain probability.

So I should be able to use that formula somehow. So I can write down that the probability of measuring y given theta is equal to the probability of y times the probability of theta given y divided by the probability of theta. So I just took this formula, and I plugged in y's and thetas instead of A's and B's. So I said, these two are equal to each other. Rearranged it so then I can rewrite this.

This is the way we have it in here, probably of measuring y given theta. Let's flip it around. So probability of theta given that we measured y is equal to the probability of theta times the probability of observing y if theta was true divided by the probability of y. Terrible handwriting there. That's just algebra.

So this is what we want to know. We want to know, what's the probability distribution of the parameter values y? Because some of them are uncertain. Now, before we started the experiment, we had some idea of what the ranges were for all the primary values. Like I'm trying to measure a rate coefficient. I know from experience with other similar reactions, from a quantum chemistry calculation, from some indirect evidence, from some other more complicated experiment, I have some idea that this rate coefficient has to be in a certain range.

Now, it could be pretty uncertain. It might be five orders of magnitude uncertain. But I know it's not less than 0. I know it can't be faster than the diffusion limit, how fast things can come together. So for sure I know some range, and oftentimes I know a much narrower range than that. So I have some information about these parameter values before I even start. Some of the parameters of the model I know perfectly, or pretty well. So you know maybe there's a Planck's constant, or the heat of formation of one of my chemicals or something like that shows up in the numbers, and I might know that parameter really pretty accurately. Whereas the particular rate coefficient I care about is the thing I really don't know very well.

So some of these have tight probability distributions ahead of time, and some of them have loose ones. And this thing has a name. It's called the prior. And it's our prior information before we did the experiment. And this one, after we've done the experiment, we're going to change it. So we're going to say previously people thought that the parameters all lied in these certain ranges. And now I'm going to get a tighter range, because I have some additional experimental information. So this is called the posterior. This means before, it means after. So this is what I know about parameter values before and after the experiment.

This is the formula that I have over there. It's a probability that if the thetas had a certain value, probably would have observed what I saw. Yeah?

AUDIENCE: Which one refers to which?

WILLIAM GREEN: Sorry, this is the prior, this is the posterior. And those of you who are paying attention to notation realize I'm not doing this very nicely. Because these are continuous variables, and I'm writing capital PRs, and they should be not be capital PRs, it should be probably density functions instead. So let's rewrite it nicely.

So the probability of theta given y, probability density is equal to the probability density of theta initially times the probability of y given theta, [INAUDIBLE] density divided by the probability density of-- all right? And what I just basically did was this is the correct equation the previous other one was all multiplied by d theta dy, it shouldn't be done that way. So this is OK?

Now this is the prior information I have about the parameter values. I know that they have to fall into some ranges. And really all I'm doing is I'm correcting that information. I'm improving the information to tighten the distribution. So initially I know that my rate constant, here's my rate constant. I know that it's got to be greater than zero, and I don't think it's really down there at zero anyway. I think it's somewhere in here. I really don't know much. And I really don't think it's all up at the diffusion limit, and no way it's higher than the diffusion limit. So that's my initial information that I have about the probability distribution of K. So it's the rate coefficient I want to know, and I know it's bigger than zero, and I know it's less than infinity. And actually I know there's some physical limit, it can't be higher than something or other.

And you can do this for any problem, right? I give you any parameter, you should be able to tell me something about it. You might be uncertain by 20 orders of magnitude, but at least you have an error bar some width. It can't be anything, right? A lot of parameters have to be positive, for example. You know that. And you usually know something. you might not think you know anything, but you do, you actually do know before you start.

So you actually know, this is the P of theta to start with. And after I've done the experiment, hopefully you're going to know more information about it. I might know that this quantity here is going to be like a Gaussian distribution. It might have a kind of goofball dependence on theta. I should comment that. Notice how theta appears inside F. So theta's up in the exponent. It's sort of inside a Gaussian, but it's like processed by F and so the observable might have a pretty goofball dependence on this rate coefficient. So this thing could be some weird thing.

But for sure, when I change theta so this changes a lot, it's going to make a pretty big difference. Because up inside the exponent of a Gaussian, so it's going to drop off a lot somewhere. So I should get something that looks something like this maybe for my experiment. So this one is P of K initially, the prior. This one is P of yk. And what this equation says is I want to multiply those two together. And so I'm going to multiply this times this, and I'm going to get some new thing that's something like that when I multiply this time that. Is that OK?

And so that's my new numerator of this equation. Now this denominator doesn't make too much sense. This says, what's the probability that I measured the mean I measured, given nothing? So this is sort of like the prior probability that I would have measured it or something. I don't know what this is. So instead what people do, is they say, forget this. But instead, let's multiply this by a constant that's going to normalize it to make it probability density so that it integrates to one.

So that's the way Bayes' theorem is used. This is called Bayesian analysis. And so what it's telling you is how to take your experimental information as expressed in this formula and use all your previous information about the parameters, put them all together, now we have a cumulative information about everything. So we have some parameters that came into our problem into my experiment, but from previous work, I also knew something about those parameters. Now I put it all together and I get a new value of probability distribution of those parameters. And if my expert was really good, it would make this really tight [WHOOSHING SOUND]. And then when I multiply these two together, it's going to make this really sharp, and we have a really good value of k. So that's like the ideal case if I have a really great, well-designed experiment executed perfectly with great precision, then I can do this.

More generally, when I don't think about it, I get some distribution like this. I still learn something compared to what I had before, but it might not be much. So now I can end up with some distribution that's a little tighter than before. So is this OK so far? All right, now this is super simple. I didn't have to solve anything, all I had to do was multiply two distributions together.

So in some respects, this is what you should always do. All you do is you take your experiment, you multiply the probability distribution corresponds to your experiment times the prior, and you get some posterior, and that's why new information about the distribution. And if I have a distribution like this, suppose this is my new distribution here, I can still get it central value, that's my mean value, k. I can get an estimate of the range of k. So I end up with a k plus or minus dk maybe, from just looking at the plot. In fact, I never even have to evaluate what this constant is in order to do this. I can just go look at the plot, see where the peak is, figure out the width, and I can report now because in my experiment, k plus or minus dk is more precisely determined than it was before.

Now, a practical challenge with this is that theta is usually a lot of parameters. And I only drew the plot here in one dimension, but really it's a multi-dimensional plot. So really what looked like, suppose I had two parameters. I had my k I care about, and I have some other parameter, theta 2, that also it shows up in my model. And say, before I started, I knew theta 2 fell in this kind of range, and I knew k fell in this kind of range.

So really before I started, if I think what it looks like, I really had sort of a blobby rectangular contour plot, where I think it's more likely that the k value and the theta 2 value are somewhere in this range. And the most likely one is maybe somewhere in the middle there. But I really didn't know much. So it could be anywhere in this whole blob.

Now, when I do the experiment, the experimental value depends of both k and theta 2. And commonly what'll happen is that the distribution from the experiment-- need color chalk here. Let's get rid of these guys. So this is my probably distribution, there's my prior. If I do the experiment, maybe I'll have something like this. That the experiment says that the guys have to be somewhere in the contour plot like this. Because I can get pretty good fits of the data with different values of k as long as I compensate with the value theta 2.

Now I multiply these two dimensional functions. The original is a blob function, and this is a stretched out blob. And I multiply a stretched out blob times a fat blob, I get some stretched out blob that looks something like the intersection of these guys. And so I end up with some kind of blob like that. I'll draw it really thick. So this is my posterior, some kind of blob like this.

So now I know a little bit more about these two parameters than I did before I started because of my experiment. Is this OK? I really can't say I know what the real value of k is, or the value of theta 2. But I know that combinations of k and theta 2 that are sort of in this range, all of them will give me pretty good fits to my data, and also be consistent with all the previous information I have about those parameter values. Is that all right?

Now, I drew it with two parameters. In a lot of models we have, we have five parameters, six parameters, seven parameters, nine parameters, 14 parameters. We have a lot of parameters. And so then we try to make this plot, even how to display the plot is going to be a little problematic. But it's there, right? And somehow, we still narrowed down the hypervolume in the parameter space from whatever it was to begin with to now we know something a little bit better. We have a narrower range of the parameters that would be consistent with all the information available, including my new experiment.

And then the next guy does his experiment, and he does an experiment that shows that these guys have to be somewhere in this range in order to be consistent with his experiment. And so now I can narrow down the range to be something like that. And the next person does their experiment, and they get something else, and something else, and something else. And eventually by 2050, we have a pretty nice determination of the parameter values.

So that's the advance of science, as drawn in chalk by Professor Green at the board. So this is a very important way to think about it, is what you're doing when you do experiments, is you're generally restricting the range of parameter space that's still consistent with everything. And when we mean consistent, we mean that the probability that you would have observed what you did observe is reasonably high. We'll still have to come back to quantitatively figure out what reasonably high means.

Now, when you did this before when you were kids, nobody mentioned the word Bayes, or Bayesian, or conditional probabilities, right? So they just said, oh, just do a least squares fit. How many of you did that before? So somebody told me before, forget this stuff, we're going to never even mention this stuff. We're just going to do a least squares fit.

Now, where did the least squares fit idea came from? It came from looking at this formula and saying, you know, this is the deviations between the experiment and the model prediction, and I weigh them somehow, and I have the square. And that's the thing I want to make small. If I have a high probability that what I observed really happened, or the probably I'm going to observe this, it's got to be that these guys have to be reasonably close to each other. They're really different. And it's going to be very small, because it's inside an exponential. And if those guys are really different, and the squared thing is really large, then the probability is incredibly small that I would have observed that.

So we think that this thing should be small. And in fact, if I want to get the very best fit I can get, which means like the probability was the highest of what I observed in the real observation or something, then if I'm free to adjust one of these thetas, I can adjust the theta, try to make this thing like equal to zero, or small as I can. So that's where the concept of least squares comes from.

Now, when you're doing least squares, you almost always have multiple parameters, and therefore you're going to have to have multiple data. And they can't just be a repeat of one number. Can't be your data, it's not sufficient to determine the parameters. So normally when you do an experiment, you have to change the knobs. We have to make measurements in a couple different conditions. Like for example, kinetics. You often want the Arrhenius A factor and the EA. And so I got to run the experiment in more than one temperature or I'm never going to be able to figure that out. So I have to change the temperature in my reactor. Make some measurements at one temperature, and make some measurements at a different temperature.

And for almost everything in life that you want to measure, you're going to have to do this. You vary the concentration of your enzyme if you want to see how the enzyme kinetics depends on something. you can't just keep running exactly the same condition over and over. You'll get that number really precise, but it's not enough information to really fill out the parameters in your model. So you're going to have to run several different experiments with different knob settings.

Also, normally we don't just measure one quantity, one observable in each experiment. We usually try to measure as many things as we can. So we actually have several observables at each knob setting, and we have several knob settings, so we have quite a lot of data. And each one of those is repeated a whole bunch of times so that we're confident that we can use this Gaussian formula.

And so what we really have is the pi-th observable measured at the l-th knob position. Well, I'm sorry, l's not good either, it's used in your notes for something else. M, there you go. The m-th knob position. Now, normally you have several knobs, so that's a factor. And we have a lot of observables we can make at each position.

So this thing is a measurement. And we repeated this multiple times so I can get the average. And we're also going to have a corresponding sigma I M, which is the variance of the mean. So it's variance, that's going to be divided by the square root of the number of repeats for that particular experiment and that particular observable.

So this is your incoming data set, and you also have your model which predicts y model, it predicts the observable i for the sequel to fi of xk theta. So if you have certain knob settings, like certain temperature, and you have your parameter values, then you can calculate what the model thinks should be the observable value, and then you can actually measure it and measure its variance. so that's the normal situation. And now you want to figure out, are there some values of the theta that make the model and the data agree? And that's the least squares fitting thing.

So what we can define as a new quantity, weight of the residual vector EJ, which is defined to be jk, to be consistent with Joe Scott's notes.

AUDIENCE: Is k the same as m?

WILLIAM GREEN: M is in oppositions, I'll tell you what k is in a second. m, sorry. Too many indices. OK, so this is the residual between the model position. And now-- oh man, I'm sorry, [INAUDIBLE]. K is an index over i and m. So k is just going to list all the data you got. Some of the data came from the same knob settings, some came from different knob settings. Yeah?

AUDIENCE: So is x the m the y model i [INAUDIBLE]?

WILLIAM GREEN: Thank you, yes. y model i, I guess this is now k. And so k is one of these indices that carry-- you can bind two indices together and put them together just like you did in your PD problems. All right. Now, I wrote down this sigma. But actually if you're measuring multiple things at the same experiment, you should expect them to be correlated. So really what we should worry about is the c, the covariance matrix, that we defined last time. So you should also compute that thing.

And so what you should expect is the probability density that we would measure any particular residuals if the model is true. And if we have these certain parameters, theta, this should be equal to 2 pi negative k over 2 the determinant of c negative 1.2 exponential of negative 1/2, epsilon transpose c inverse epsilon.

So this is the multi measurement version of the same equation here. So this is the quantity that we think should be small if we have good parameter values and we did a good experiment. Actually, even when we did bad experiments, still should be small if we have good parameter values. And that's because the c's, if we did a bad experiment, we'll have a high variance or something, then we should see the c's will give us weightings that will reflect that. Yeah?

AUDIENCE: [INAUDIBLE]

WILLIAM GREEN: Is that--

AUDIENCE: So you have the next [INAUDIBLE] K [INAUDIBLE].

WILLIAM GREEN: Oh I'm sorry, this is the capital K, this is the number of data points. So little k is equal to 1 to capital K.

AUDIENCE: So does capital K count for both experiments?

WILLIAM GREEN: It's the number of distinct data values after you've already averaged over repeats. So you do m experiments, at each experiment you measure capital I observables. So it's like m times I, so K. If you measured everything in every experiment, it's equal to I times m.

Now there's two ways that people approach this in literature. The fancy way is you say, you know, this covariance matrix comes in in a pretty important way into this probability distribution function. And so maybe I need to worry a lot about whether I really know the covariance matrix. And my uncertainty in the mean drops pretty fast as I do averaging, but I'm not so confident that my answer in the covariance matrix was small.

So what people do sometimes is they'll try to vary both c and theta, and try to get a best fit where they're varying c. But then they have additional constraints on c that c has to satisfy the equations you gave last time about how you calculate the covariance matrix from data. And so I was saying, well, I want this c to personify these equations pretty well, but true covariance of the world of the system is not the same as what I actually measure by just measuring, say, five repeats of an experiment. And so I might want to vary the c.

You try to vary the c, turns out to be kind of complicated math, so not many people do it. Even though conceptually it makes some sense, you should worry about the fact you're not really sure about the covariance. So what a lot of people do is they say, let's just use the c that's computed from the formulas I gave you last time experimentally. So just say, let's just take c experimental, put them in here. And now this is a constant. And now the only thing that varies in this problem is thetas which come into the epsilons. Because the epsilons depend on theta.

And so in that case, I can just try to maximize this probability. And what that happens to do is to minimize this quantity in the exponent. And so all I need to do is say, for example, theta best is equal to arg [INAUDIBLE] theta epsilon of theta c epsilon. And so this is the least squares fitting problem that you guys have probably done before. And probably what you did was you assumed I had perfectly uncorrelated data, and all my errors were the same. And so c, which is the identity matrix, and I took it out. Probably did that before? Yeah, OK. That's pretty dangerous to do, I'd say.

What people do a lot, which is a little bit less dangerous, is at least say, well, you know, when I measure the concentration of species x by GC, I have an error bar of plus or minus 5%. And when I measure the temperature with my thermocouple, I have an error bar of plus or minus 2 degrees. And so the variances of these guys should be a lot different, temperature and GC signal. And therefore I definitely need to weight my deviation somehow. And it's really what you do is you keep the diagonal entries of this. That's often done. And we just forget the fact that they might be covariant. But if you've done the experiments, you actually do have enough information to compute this thing anyway, so you might as well just use the experimental value.

So this is the least squares thing. And let's think, what the heck is this doing? We're saying, all of a sudden we grabbed all the parameters in the model, which might include things like the molecular weight of hydrogen or something. And we can find the very best values that would make our data match the experiment as best as possible. And in some sense, that's great, we know the best values and parameters for our experiment. But of course, if we vary the molecular weight of hydrogen, it's going to screw up somebody else's experiment. Because somebody else did some other experiment that depended on the molecular weight of hydrogen, and they had to get some other value to match their experiment.

So in these parameter set, anything I do to vary those parameters, I got to watch out that maybe some of those parameters are involved with somebody else's model and [INAUDIBLE] some other experiments. And I'm not really free to vary them all freely. So this is the idea from the Bayesian of having the prior intimation is so you know some of the ranges on these thetas already, and some of them you might know really sharp distributions, like the molecular weight of hydrogen. You might know that to a lot of decimal places.

And so when people do this, normally you don't vary all of the thetas. Usually what you do is you select a set of thetas that you feel free to vary because they're so uncertain, and other thetas that you think, oh, I better not touch them. Because if I adjust them, I may go to crazy values that are inconsistent with somebody else's experiment. So a lot of times like the molecular weights, you would not touch them. You would just say, I got to just stick to the recommended values and the tables. I'm not free to vary the molecular weight of hydrogen, even though if I did it would make my data match my experiment better. Makes my model and the experiment match more precisely.

So deciding which parameters to vary in this is a really crucial thing. And that's a lot of the art of doing this has to do with that issue. Also, you don't have to keep the thetas in the form you have them. You could do a transformation. So you could change to, say, W's that's equal to, say, some matrix times the thetas, and I could express the equation in terms of the W's. So I could transform my original representational parameters as some other parameters.

And often times, your experiment might be really good at determining some of these W's, even if it might be incapable of determining any of the thetas. So you often might know some linear combination of parameters, or maybe not linear combinations, some non-linear combination of parameters might actually be determinable very well from your experiment, even though you can't determine things separately.

And this gets into the idea of dimensionless numbers. So your experiment might depend on some dimensionless number very sensitively. And you can be quite confident from your external data what the value of that dimensionless number must be. But if you look inside the dimensionless number, it depends on a lot of different things. And you might not have any information about them separately. All you know is about your experiment just tells you the value of that one parameter very accurately.

So this is another big part of the art of doing the model versus data is setting up your model in terms of parameters that you really can determine, and getting out all the ones you can't determine and fixing them. So we're really going to generally change do this kind of thing. But we're going to say that some thetas are fixed, and also we might change to a different representation, change to W's instead. Yeah?

AUDIENCE: Can you explain where this transform-- I don't really know what's up with--

WILLIAM GREEN: Yeah, let's get an example. Suppose I was doing a reactor that had A equilibrium of B. And I was really interested in kf, the forward rate for A going to B. I'm a kineticist, I love to know A goes to B. However, if I setup the experiment wrong, it might be that this readction ran all the way to equilibrium. And what I see in the products is actually just the equilibrium ratio of A to B.

So what I'm measuring might be something that's dependent really on kf over kr, and that might be the quantity I can really determine. Because that's equilibrium constant. If I didn't think about it, I could just try to have the model fitting procedure, just optimize to find the very best value of kf. And in that situation, it might have a lot of trouble, because it might be quite indeterminant what the kf is, because really all that matters is the ratio.

Also I think about this some more, suppose I run at short times, and I measure the time dependence. What I'm really measuring is kf plus kr. Do you remember we did the analysis of A goes to B, one of the early homework problems? The time cost was actually kf plus kr, not kf separately. And so if I measure the exponential decay time constant, I'm really determining kf plus kr, I might be able to determine that very well. Actually, in my lab, I can do a great job for this. I have an instrument that can measure the time constant of the exponential of k really precisely, but it's determining the sum. It's not determining either one of them separately. And I might have to do a separate experiment, say a thermo-experiment to get the ratio. And then from the two I can put them together and get the two values distinctly.

So this will be an example of this would be a W. My W is kf plus kr, the matrix would be 1001, something like that. 1-1, something like that. Where I add these two guys, kf, kr. These are my two parameters, 1 plus 1. And I can determine W1 now very accurately. sorry, this is m, this is W.

So now in terms of W, this has two parameters now, W1 and W2. I can't determine W2 from my experiment, but I can determine W1 really well. So then when I do the least squares fitting, I should vary W1. I can fix it it for my experimental data, and just leave W2 fixed at some value. I can't do anything about it. That all right?

Now, do you get the difference in these two points of view? This is like, two completely different ways to look at the problem. You can think about it as, these parameters are free for me to vary, and I just have to be careful to select the ones I'm really free to vary. And that's the least squares fitting point of view. Or I could say, I'm not really determining anything in particular, all I'm doing is taking the whole range of uncertainty that we have about parameters, and by my experiment, I narrowed it down in the Bayesian view. So it's the two different points of view.

To do this one, I need to make sure I have enough data to determine something. So I have to have enough determine some parameter, at least one, otherwise there's no point in doing this. This one I can do even if I can't determine anything, because I could still narrow down the range of parameters. But this might be harder to report in a table. Because all I have at the end is a new probably density function of multiple parameters. All right? OK, we're done. See you guys on Friday.

Free Downloads

Video

• iTunes U (MP4 - 119MB)
• Internet Archive (MP4 - 119MB)

Caption

• English-US (SRT)