1
00:00:08,410 --> 00:00:15,410
This will be Professor Cohen's kind of last
complete lecture, although, on the last day

2
00:00:15,809 --> 00:00:22,390
of classes the plan is we're going to take
some of the time just to talk over what we've

3
00:00:22,390 --> 00:00:24,930
done and review things.

4
00:00:24,930 --> 00:00:31,560
And it will be really oriented towards general
aspects of systems engineering.

5
00:00:31,560 --> 00:00:33,090
And, Aaron, I will turn it over to you.

6
00:00:33,090 --> 00:00:36,760
Thank you.

7
00:00:36,760 --> 00:00:43,760
Today, I'm going to lecture primarily on systems
engineering but management as it relates to

8
00:00:45,520 --> 00:00:46,489
systems engineering.

9
00:00:46,489 --> 00:00:50,030
I have several checks and balances.

10
00:00:50,030 --> 00:00:55,199
And, of course, today I've got people from
the Draper Laboratories and people from the

11
00:00:55,199 --> 00:01:01,780
predecessor to Draper Laboratories, MIT Instrumentation
Laboratory are going to check me to see if

12
00:01:01,780 --> 00:01:04,349
I say the right thing.

13
00:01:04,349 --> 00:01:10,340
First of all, I would like to say to you as
a class that I read your last submission reports

14
00:01:10,340 --> 00:01:13,470
and really thought they were outstanding.

15
00:01:13,470 --> 00:01:19,099
They show a great deal of interest, a great
deal of understanding and a great deal of

16
00:01:19,099 --> 00:01:19,349
initiative.

17
00:01:19,130 --> 00:01:19,789
And so, I really compliment you.

18
00:01:19,789 --> 00:01:26,789
And I'm sure your final reports will be better,
but they really were very, very, very good.

19
00:01:27,780 --> 00:01:34,780
You've heard a lot of people talk previously,
starting with, in my way of thinking, Dale

20
00:01:36,569 --> 00:01:41,910
Myers who was sort of the person who started
the program in Washington at the time.

21
00:01:41,910 --> 00:01:47,440
You didn't hear from another very important
man, George Miller, who was really Dale's

22
00:01:47,440 --> 00:01:50,720
boss at the time.

23
00:01:50,720 --> 00:01:54,348
But George is still alive, is still doing
very well.

24
00:01:54,348 --> 00:01:55,789
And you didn't hear from a lot of other people.

25
00:01:55,789 --> 00:02:02,789
You did hear from Chris Kraft which sort of
set the tone, you might say, for a very interesting

26
00:02:03,360 --> 00:02:09,220
and understanding man who really knows what
he's doing and says what he thinks.

27
00:02:09,220 --> 00:02:13,950
But there are several other people that I
would like to talk about that had a great

28
00:02:13,950 --> 00:02:18,550
bearing on my career in terms of systems engineering.

29
00:02:18,550 --> 00:02:23,030
There were two people, both who have passed
away.

30
00:02:23,030 --> 00:02:25,530
One is Joe Shea.

31
00:02:25,530 --> 00:02:29,340
And Joe was a very remarkable engineer.

32
00:02:29,340 --> 00:02:36,340
He was program manager of the Apollo Program
at the Johnson Space Center, then left and

33
00:02:38,270 --> 00:02:44,870
became Vice President of Raytheon here and
then became a professor at MIT in the Aero-Astro

34
00:02:44,870 --> 00:02:45,120
Department.

35
00:02:45,000 --> 00:02:49,890
And I thought he maybe taught the forerunner
to this course.

36
00:02:49,890 --> 00:02:52,100
He taught systems engineering.

37
00:02:52,100 --> 00:02:59,100
And Joe Shea was actually the best systems
engineer I've ever met, so I am going to talk

38
00:02:59,190 --> 00:03:03,340
a little bit and give you some understanding
of my interaction with Joe.

39
00:03:03,340 --> 00:03:07,640
The best job I ever had, you might say, really,
is I've had a lot of different jobs, but my

40
00:03:07,640 --> 00:03:14,120
first job was really working with the MIT
Instrumentation Laboratory where I met a great

41
00:03:14,120 --> 00:03:15,200
number of people.

42
00:03:15,200 --> 00:03:19,860
I'm not going to mention everybody because
it would fill up the time.

43
00:03:19,860 --> 00:03:25,660
They taught me systems engineering, but had
it not been for the MIT Instrumentation Laboratory

44
00:03:25,660 --> 00:03:29,010
we wouldn't have gone to the Moon on the schedule
we went on.

45
00:03:29,010 --> 00:03:31,780
They really did a fantastic job, all these
people.

46
00:03:31,780 --> 00:03:34,670
I mean they were just you might say giants.

47
00:03:34,670 --> 00:03:39,290
There were great people at Rockwell, at North
American Aviation such as George Jeffs and

48
00:03:39,290 --> 00:03:46,290
great people at Grumman such as Joe Gavin
and other places, but I have to say that the

49
00:03:47,090 --> 00:03:53,230
MIT Instrumentation Lab really carried the
ball on understanding how to do guidance,

50
00:03:53,230 --> 00:03:54,880
navigation and control.

51
00:03:54,880 --> 00:03:59,940
And if you read some of the comments by George
Low, who is the other person I am going to

52
00:03:59,940 --> 00:04:01,900
talk about, George Low is another giant.

53
00:04:01,900 --> 00:04:07,090
He says the most complicated system on the
Apollo spacecraft was the guidance navigation

54
00:04:07,090 --> 00:04:08,390
system, if you read what he said.

55
00:04:08,390 --> 00:04:15,310
And could you really hit a target 240,000
miles away and bring it back?

56
00:04:15,310 --> 00:04:18,738
But let me get back to Joe Shea.

57
00:04:18,738 --> 00:04:22,029
I said he was a great systems engineer.

58
00:04:22,029 --> 00:04:25,910
And Joe Shea taught me a couple of things.

59
00:04:25,910 --> 00:04:29,790
One, he taught me not to be afraid to make
a decision.

60
00:04:29,790 --> 00:04:36,430
He said the fact that you are in charge, the
fact that you have looked at all the information,

61
00:04:36,430 --> 00:04:41,260
the fact that you know the details, you are
the best one to make a decision.

62
00:04:41,260 --> 00:04:48,260
And if you could make a timely decision and
your battering average is a little over 60%

63
00:04:48,889 --> 00:04:50,850
you're doing very well.

64
00:04:50,850 --> 00:04:56,350
But the fact you made it timely and the fact
that you understand it, you can make a change.

65
00:04:56,350 --> 00:04:58,970
You can change if you are wrong.

66
00:04:58,970 --> 00:05:01,710
And that always stuck with me, not to be afraid
to make a decision.

67
00:05:01,710 --> 00:05:08,070
Because many times as you go through your
role in systems engineering you're not going

68
00:05:08,070 --> 00:05:11,419
to have all the information to make a decision
on a timely basis.

69
00:05:11,419 --> 00:05:17,810
It is just not going to be there, especially
if you do a program that pertains to advanced

70
00:05:17,810 --> 00:05:21,590
technology, advanced research and development.

71
00:05:21,590 --> 00:05:23,419
That is one thing that is important.

72
00:05:23,419 --> 00:05:27,190
Another thing that Joe Shea brought to the
table.

73
00:05:27,190 --> 00:05:32,960
During the Apollo Program, before we had Joe
Shea, we would sit around a conference table

74
00:05:32,960 --> 00:05:39,210
in the early `60s trying to figure out what
systems engineering was, but we couldn't really

75
00:05:39,210 --> 00:05:41,650
explain what we had to do.

76
00:05:41,650 --> 00:05:44,010
We didn't know what we really had to do.

77
00:05:44,010 --> 00:05:51,010
But when Joe Shea came, he brought with him
the insight of what you needed to do to do

78
00:05:54,080 --> 00:05:55,010
systems engineering.

79
00:05:55,010 --> 00:06:01,320
Now, this is the Apollo stack as you see in
on the pad you might say.

80
00:06:01,320 --> 00:06:03,090
The launch vehicles.

81
00:06:03,090 --> 00:06:04,500
The launch vehicle.

82
00:06:04,500 --> 00:06:06,949
The first stage, second stage.

83
00:06:06,949 --> 00:06:13,949
Then you get to the lunar module, the command
and service module, launch escape tower.

84
00:06:15,060 --> 00:06:17,330
That is what it looks like on the pad.

85
00:06:17,330 --> 00:06:23,300
Well, Joe Shea gave me probably the toughest
job and one of the most important jobs that

86
00:06:23,300 --> 00:06:26,030
I had in my career.

87
00:06:26,030 --> 00:06:33,030
He said, Aaron, you are going to be in charge
of resolving all the ICDs, the interface control

88
00:06:33,220 --> 00:06:33,720
documents.

89
00:06:33,720 --> 00:06:39,290
That is all the interfaces between all the
stages, between the command and service module

90
00:06:39,290 --> 00:06:44,130
and the lunar module, between the command
and service module, lunar module and the stack,

91
00:06:44,130 --> 00:06:47,790
between the command and service module, lunar
module and the launch complex.

92
00:06:47,790 --> 00:06:52,330
There were about a thousand ICDs involved
in that.

93
00:06:52,330 --> 00:06:59,330
And that was a formidable task because an
interface control document you cannot design

94
00:06:59,770 --> 00:07:04,009
without the interfaces being defined, and
you cannot define the interfaces without the

95
00:07:04,009 --> 00:07:05,020
design.

96
00:07:05,020 --> 00:07:08,150
So it is really a chicken and the egg type
of thing.

97
00:07:08,150 --> 00:07:11,550
And it turns out that was a turning point
in my career because we did do that.

98
00:07:11,550 --> 00:07:16,490
In fact, a little sideline, a little interesting
story.

99
00:07:16,490 --> 00:07:20,070
He anointed me to do this job but didn't just
leave me there.

100
00:07:20,070 --> 00:07:24,800
He took me to the contractors, he took me
to the MIT Instrumentation Lab, he took me

101
00:07:24,800 --> 00:07:29,820
to Rockwell, he took me to Grumman and he
took me to the Marshall Space Flight Center

102
00:07:29,820 --> 00:07:32,370
and Kennedy Space Center.

103
00:07:32,370 --> 00:07:36,210
And the funny part about it, he took me in
to see von Braun, which I didn't know about.

104
00:07:36,210 --> 00:07:38,620
I didn't know von Braun.

105
00:07:38,620 --> 00:07:41,750
And he said, Wernher, this is Aaron Cohen.

106
00:07:41,750 --> 00:07:44,699
We were in his big office at Marshall.

107
00:07:44,699 --> 00:07:48,680
He said he is going to resolve all the ICDs,
all the interface control documents.

108
00:07:48,680 --> 00:07:50,770
And Wernher von Braun said that is wonderful.

109
00:07:50,770 --> 00:07:52,100
What is an ICD?

110
00:07:52,100 --> 00:07:58,600
So I knew I had a problem there right away
that the head charge of Marshall didn't know

111
00:07:58,600 --> 00:07:59,650
what an ICD was.

112
00:07:59,650 --> 00:08:02,699
But that was a little bit of a sideline on
it.

113
00:08:02,699 --> 00:08:08,580
I did have the job to actually resolve all
the interface control documents.

114
00:08:08,580 --> 00:08:14,270
And that turned out to be one of my most interesting,
most challenging jobs as a young engineer.

115
00:08:14,270 --> 00:08:19,639
It really taught me systems engineering, because
one of the important parts of system engineering

116
00:08:19,639 --> 00:08:23,310
is understanding where the interfaces are
and how you get the interfaces resolved.

117
00:08:23,310 --> 00:08:29,380
And you could imagine trying to get Grumman
at that time at North American Aviation and

118
00:08:29,380 --> 00:08:33,539
MIT Instrumental Labs to agree on an interface
document.

119
00:08:33,539 --> 00:08:37,130
I mean it wasn't the easiest thing in the
world to do.

120
00:08:37,130 --> 00:08:37,380
Yes.
Did you have a question?

121
00:08:39,440 --> 00:08:43,259
I was just wondering, aside from technically
making sure that the interfaces are working

122
00:08:43,259 --> 00:08:47,509
and everything, what is involved in resolving
an ICD?

123
00:08:47,509 --> 00:08:49,410
That's a good question.

124
00:08:49,410 --> 00:08:52,429
Well, first of all, I am going to talk a little
bit more about ICDs in general.

125
00:08:52,429 --> 00:08:55,300
ICDs you can define in several categories.

126
00:08:55,300 --> 00:09:01,800
One is mechanical or physical ICDs, bolt hold
patterns, just how things bolt together.

127
00:09:01,800 --> 00:09:07,569
The other, you might say, is electrical ICDs,
what kind of electrical signals are sent and

128
00:09:07,569 --> 00:09:09,490
electrical interfaces.

129
00:09:09,490 --> 00:09:12,449
And the other are functional ICDs.

130
00:09:12,449 --> 00:09:19,449
One of the big ICDs we had to resolve was
a term called Guidance Reference Release.

131
00:09:19,920 --> 00:09:24,790
Marshal had an inertial measurement unit in
their instrument unit, and we had to have

132
00:09:24,790 --> 00:09:30,610
that synchronized with our system or with
the system in the command and service module.

133
00:09:30,610 --> 00:09:34,110
What is involved in it is you sit down and
you talk.

134
00:09:34,110 --> 00:09:36,819
You have a draft ICD.

135
00:09:36,819 --> 00:09:37,959
And we had these big meetings.

136
00:09:37,959 --> 00:09:41,309
We had them at the Kennedy Space Center.

137
00:09:41,309 --> 00:09:44,490
I don't know if you've ever been to the Kennedy
Space Center, but we actually fill the firing

138
00:09:44,490 --> 00:09:46,730
room with people.

139
00:09:46,730 --> 00:09:52,269
And each group had a certain set of draft
ICDs, and they would sit down in working groups

140
00:09:52,269 --> 00:09:59,269
and decide what had to be done to get the
interface compatible in terms of the three

141
00:09:59,279 --> 00:10:05,689
phases Physical, electrical and functional.

142
00:10:05,689 --> 00:10:06,769
And you just work out the details.

143
00:10:06,769 --> 00:10:11,019
Aaron, probably now in the 21st century, we
ought to add data ICDs.

144
00:10:11,019 --> 00:10:11,949
Data ICDs, yeah.

145
00:10:11,949 --> 00:10:16,319
Well, now things have changed probably a lot
since I've been involved in it.

146
00:10:16,319 --> 00:10:17,949
I mean remember Mars Polar Lander.

147
00:10:17,949 --> 00:10:19,749
We had English units coming over.

148
00:10:19,749 --> 00:10:20,319
Yeah, that's right.

149
00:10:20,319 --> 00:10:23,100
And the people thought they were getting metric
units.

150
00:10:23,100 --> 00:10:23,980
So the data ICDs are also critical.

151
00:10:23,980 --> 00:10:29,100
You know, and a lot of things are very simple
and mundane.

152
00:10:29,100 --> 00:10:30,579
I could tell you a story with the crew.

153
00:10:30,579 --> 00:10:34,319
When we said you're going to have in-flight
maintenance with the crew.

154
00:10:34,319 --> 00:10:41,319
And we gave the crew to change out a component,
but instead of taking the screw out very easily,

155
00:10:42,779 --> 00:10:47,970
we had actually put Loctite in the screw so
we couldn't undo the screw.

156
00:10:47,970 --> 00:10:51,629
There are very simple interfaces that have
to be resolved.

157
00:10:51,629 --> 00:10:53,540
Some are mundane, some are very sophisticated.

158
00:10:53,540 --> 00:10:55,410
I don't know if that answers your question.

159
00:10:55,410 --> 00:10:57,540
The problem is there is no F=ma.

160
00:10:57,540 --> 00:11:02,689
There is no closed form solution that tells
you exactly how to do it.

161
00:11:02,689 --> 00:11:08,709
You just have to sit down and work it because
the design hasn't been complete but you have

162
00:11:08,709 --> 00:11:09,319
ideas.

163
00:11:09,319 --> 00:11:13,179
You float ideas, you negotiate, you arbitrate
and then you come up with a decision.

164
00:11:13,179 --> 00:11:17,550
The main thing is you've got to make a decision,
and somebody has to be there to make a decision.

165
00:11:17,550 --> 00:11:22,999
That was my job, to make a decision.

166
00:11:22,999 --> 00:11:29,079
That was what Joe Shea left me with.

167
00:11:29,079 --> 00:11:36,079
And people that know him will agree that he
was an outstanding engineer but also probably

168
00:11:37,600 --> 00:11:40,550
one of the best systems engineers.

169
00:11:40,550 --> 00:11:45,920
Larry, do you know what course Joe taught
when he was here?

170
00:11:45,920 --> 00:11:47,939
Yes, systems engineering.

171
00:11:47,939 --> 00:11:50,619
He taught this course?

172
00:11:50,619 --> 00:11:51,300
Yes.

173
00:11:51,300 --> 00:11:56,759
I thought he taught this course because I
know I came up and gave a lecture for him.

174
00:11:56,759 --> 00:11:59,259
Until he was taken ill.

175
00:11:59,259 --> 00:12:06,259
Let me talk about another person that was
very prominent in informing my career.

176
00:12:08,649 --> 00:12:10,429
His name was George Low.

177
00:12:10,429 --> 00:12:17,429
After Joe retired, George Low became the Apollo
Program Manager at the Johnson Space Center.

178
00:12:20,549 --> 00:12:23,949
And George was very much a disciplinarian.

179
00:12:23,949 --> 00:12:27,279
He always had time to see you.

180
00:12:27,279 --> 00:12:31,809
But there were several criteria in seeing
you.

181
00:12:31,809 --> 00:12:38,809
You had to tell him what your subject was,
how long he wanted and why you wanted to see

182
00:12:41,929 --> 00:12:42,509
him.

183
00:12:42,509 --> 00:12:46,600
So when you met with him, if you asked for
15 minutes, you had 15 minutes.

184
00:12:46,600 --> 00:12:50,509
And no matter how important the job was, if
you didn't finish was you had to say in 15

185
00:12:50,509 --> 00:12:52,559
minutes you were out the door.

186
00:12:52,559 --> 00:12:55,259
And you only did that once.

187
00:12:55,259 --> 00:12:59,110
But you didn't ask for too much time either
because you got penalized if you said you

188
00:12:59,110 --> 00:13:00,850
needed an hour and you only took 30 minutes.

189
00:13:00,850 --> 00:13:05,759
So, he was very much a disciplinarian.

190
00:13:05,759 --> 00:13:11,059
He actually, after he retired from NASA, after
he retired from the Apollo Program became

191
00:13:11,059 --> 00:13:17,589
Deputy Administrator at NASA and was very
instrumental in the Shuttle Program and then

192
00:13:17,589 --> 00:13:20,480
retired and became president of Rensselaer
Polytechnic Institute.

193
00:13:20,480 --> 00:13:22,670
So, George had a very illustrious career.

194
00:13:22,670 --> 00:13:25,649
He passed away several years ago.

195
00:13:25,649 --> 00:13:31,139
One incident might be of interest to you,
and I recall it very vividly.

196
00:13:31,139 --> 00:13:38,139
I don't know if everybody else does, but we
were getting ready to go to the Moon on Apollo

197
00:13:40,179 --> 00:13:40,429
11.

198
00:13:40,259 --> 00:13:40,610
This is on Apollo.

199
00:13:40,610 --> 00:13:44,980
And you monitor all the systems very carefully.

200
00:13:44,980 --> 00:13:51,589
And we were monitoring the inertial measurement
unit in the lunar module which is right here.

201
00:13:51,589 --> 00:13:57,079
And for some reason or other the drift rate
changed fairly drastically.

202
00:13:57,079 --> 00:14:00,040
Now, it was still in spec.

203
00:14:00,040 --> 00:14:04,929
The drift rate was still in spec but it changed.

204
00:14:04,929 --> 00:14:09,989
And to get to the lunar module and change
out an inertial measurement unit on the pad

205
00:14:09,989 --> 00:14:10,929
is pretty hard to do.

206
00:14:10,929 --> 00:14:13,600
First of all, the lunar module has very little
structure to it.

207
00:14:13,600 --> 00:14:15,509
It is made out of Reynolds Wrap really.

208
00:14:15,509 --> 00:14:18,470
I mean it doesn't have very much structure
to it.

209
00:14:18,470 --> 00:14:25,470
And so we reviewed it with all the people
involved, reviewed the data and we concluded

210
00:14:25,920 --> 00:14:29,179
very clearly that there was nothing wrong
with it.

211
00:14:29,179 --> 00:14:36,179
And so we went to George Low and we told him
a unified story, the MIT people, the MIT Instrumentation

212
00:14:38,559 --> 00:14:41,939
people, the Rockwell people, everybody, the
JSC people.

213
00:14:41,939 --> 00:14:48,359
And George Low listened very attentively and
then said can you explain why it changed?

214
00:14:48,359 --> 00:14:52,420
And we said no, we're going to change it out.

215
00:14:52,420 --> 00:14:58,189
That is the thing that always stuck in my
mind.

216
00:14:58,189 --> 00:15:03,859
If there is something that you don't understand
then you need to take some action to fix it.

217
00:15:03,859 --> 00:15:09,489
And to me that was a very important part of
my learning process.

218
00:15:09,489 --> 00:15:16,489
And so, those were the people you might say
that really had an effect on my systems engineering

219
00:15:20,299 --> 00:15:22,199
thought process.

220
00:15:22,199 --> 00:15:28,040
The other thing I want to talk about with
you today is you heard a lot of people talk

221
00:15:28,040 --> 00:15:32,889
that I would class as subsystem managers.

222
00:15:32,889 --> 00:15:35,329
You heard Tom Moser talk about structures.

223
00:15:35,329 --> 00:15:36,230
I don't know if I can recall them all.

224
00:15:36,230 --> 00:15:40,299
Al Louviere talk about the mechanical systems.

225
00:15:40,299 --> 00:15:47,299
You heard Bob Ried talk about aerothermodynamics,
Bass Redd about aerodynamics, Henry Pohl about

226
00:15:47,449 --> 00:15:53,249
reaction control systems, auxiliary power
units and hydraulics, Walt Guy about environmental

227
00:15:53,249 --> 00:15:54,269
control systems.

228
00:15:54,269 --> 00:15:55,619
You heard of these people.

229
00:15:55,619 --> 00:16:02,619
These people are what I call subsystem managers
that actually, in some regards, work for me

230
00:16:02,829 --> 00:16:05,519
and, in some regards, didn't work for me.

231
00:16:05,519 --> 00:16:08,439
And that is what you call a matrix management
system.

232
00:16:08,439 --> 00:16:10,739
Now, I'm going to talk about matrix management.

233
00:16:10,739 --> 00:16:16,629
Matrix management was actually a very popular
management system several years ago.

234
00:16:16,629 --> 00:16:22,220
I don't know if it is used today or not, but
matrix management is a type of management

235
00:16:22,220 --> 00:16:25,899
basically used by large organizations.

236
00:16:25,899 --> 00:16:31,989
Primarily what it is are large projects organized
with teams that work on a functional rather

237
00:16:31,989 --> 00:16:36,730
than a project basis.

238
00:16:36,730 --> 00:16:41,369
In other words, these people I just talked
about actually had two bosses.

239
00:16:41,369 --> 00:16:48,369
They had a boss in their engineering organization,
a very famous man named Max Faget.

240
00:16:49,470 --> 00:16:52,619
And then they had another boss, which was
so famous, me.

241
00:16:52,619 --> 00:16:58,660
And I was their boss that controlled the project's
cost schedule and performance.

242
00:16:58,660 --> 00:17:01,040
And they actually reported to me on that.

243
00:17:01,040 --> 00:17:03,230
But they actually had two bosses.

244
00:17:03,230 --> 00:17:06,089
And so, you do that for several reasons.

245
00:17:06,089 --> 00:17:08,799
You do that in order to conserve resources.

246
00:17:08,799 --> 00:17:12,690
And many large companies do that.

247
00:17:12,690 --> 00:17:19,050
The other way of doing it, actually, would
be to put all that engineering talent in the

248
00:17:19,050 --> 00:17:20,640
project office.

249
00:17:20,640 --> 00:17:22,270
And you could see the advantages and disadvantages.

250
00:17:22,270 --> 00:17:25,930
I will show you more explicitly the advantages
and disadvantages on that.

251
00:17:25,930 --> 00:17:29,940
The advantages are that you have a dedicated
organization.

252
00:17:29,940 --> 00:17:33,860
The disadvantages are that when the project
is over there is no place for these people

253
00:17:33,860 --> 00:17:34,920
to go.

254
00:17:34,920 --> 00:17:37,550
And I don't know what you use.

255
00:17:37,550 --> 00:17:40,730
You have a matrix at the Draper Labs today.

256
00:17:40,730 --> 00:17:42,410
I'm not that close to industry.

257
00:17:42,410 --> 00:17:43,550
Do most companies use matrix today?

258
00:17:43,550 --> 00:17:43,940
Yes, most aerospace companies.

259
00:17:43,940 --> 00:17:50,940
There are too many projects that change all
the time and somebody has to be responsible

260
00:17:54,740 --> 00:17:56,240
for the people.

261
00:17:56,240 --> 00:17:58,730
See, that's an interesting part.

262
00:17:58,730 --> 00:18:01,460
We, at the Johnson Space Center, didn't have
that many projects.

263
00:18:01,460 --> 00:18:05,890
We only had really maybe one or two projects,
but we still used matrix management because

264
00:18:05,890 --> 00:18:10,460
there is a check and balance on matrix management
which you don't get on centralized or project

265
00:18:10,460 --> 00:18:10,710
management.

266
00:18:10,710 --> 00:18:12,530
There is advantage to it.

267
00:18:12,530 --> 00:18:14,320
There are some disadvantages to it.

268
00:18:14,320 --> 00:18:18,900
His rotation lab was project-oriented at the
time when we did Apollo.

269
00:18:18,900 --> 00:18:23,770
You were project-oriented because you had
one project and you were just project-oriented

270
00:18:23,770 --> 00:18:24,890
all NASA project.

271
00:18:24,890 --> 00:18:27,140
So, there are advantages and disadvantages.

272
00:18:27,140 --> 00:18:34,140
I had some dealings with Ford Motor Company,
and I think Ford Motor Company actually uses

273
00:18:34,160 --> 00:18:35,120
matrix management.

274
00:18:35,120 --> 00:18:39,740
I think most large companies use matrix management
today.

275
00:18:39,740 --> 00:18:45,060
Anyway, that's really the advantages and disadvantages
of matrix management.

276
00:18:45,060 --> 00:18:50,530
And a lot of things I have already said.

277
00:18:50,530 --> 00:18:57,390
Under matrix management, all the people who
do one type of work are in a pool.

278
00:18:57,390 --> 00:19:03,280
For example, all the engineers in one engineering
department report to an engineering manager.

279
00:19:03,280 --> 00:19:04,310
That is what I said.

280
00:19:04,310 --> 00:19:05,890
They all report to an engineering manager.

281
00:19:05,890 --> 00:19:07,570
In this case, the name was Max Faget.

282
00:19:07,570 --> 00:19:12,230
And if you do history in the Space Program,
Max Faget becomes a very prominent name.

283
00:19:12,230 --> 00:19:19,230
He was really, you might say, the original
designer of the Mercury, Gemini and a lot

284
00:19:19,870 --> 00:19:22,690
to the Apollo spacecraft configuration.

285
00:19:22,690 --> 00:19:25,090
So, you will run into his name time and time
again.

286
00:19:25,090 --> 00:19:32,090
But, the interesting part about it is that
Max and I used to go at it tooth and nail.

287
00:19:32,390 --> 00:19:36,620
I mean his people would want to do something
like a Walt Guy.

288
00:19:36,620 --> 00:19:41,200
Walt at that time was a very stubborn person.

289
00:19:41,200 --> 00:19:44,340
And he would want to do something and I wouldn't
want to do it.

290
00:19:44,340 --> 00:19:46,560
Well, he would go to Max.

291
00:19:46,560 --> 00:19:51,120
And then Max and I and Walt would have a meeting
at the Summit with Chris Kraft.

292
00:19:51,120 --> 00:19:52,550
It was a common point.

293
00:19:52,550 --> 00:19:55,180
We all went to see Chris Kraft.

294
00:19:55,180 --> 00:19:59,780
Sometimes I won and sometimes I lost, but
I learned actually to love it.

295
00:19:59,780 --> 00:20:04,050
Even though it frustrated me, I learned to
love it because there was that check and balance

296
00:20:04,050 --> 00:20:08,610
on me that I could not do something that was
going to actually be wrong.

297
00:20:08,610 --> 00:20:11,350
Somebody else was going to catch me if I did
something wrong.

298
00:20:11,350 --> 00:20:17,160
Now, I honestly do not know today what NASA
uses, whether they use matrix management on

299
00:20:17,160 --> 00:20:21,350
the Shuttle program or on the Space Station
program or how they are going to do it.

300
00:20:21,350 --> 00:20:23,110
Do you have any idea?

301
00:20:23,110 --> 00:20:27,240
The Shuttle was certainly much more project-oriented.

302
00:20:27,240 --> 00:20:32,560
We don't have the same, I mean there is an
engineering division at JSC, but during the

303
00:20:32,560 --> 00:20:36,410
Shuttle that was -- It started out matrix
management.

304
00:20:36,410 --> 00:20:42,710
I think it really decreased, and there was
no more Max Faget running things in that same

305
00:20:42,710 --> 00:20:43,100
way.

306
00:20:43,100 --> 00:20:49,500
And so that, in my mind, at least in my way
of thinking, is probably one reason why problems

307
00:20:49,500 --> 00:20:54,200
occurs, because you don't have that independent
check and balance.

308
00:20:54,200 --> 00:21:00,810
There is a very good reason why matrix management,
I think, is important, even though you don't

309
00:21:00,810 --> 00:21:07,810
have a number of projects that would cause
you to do matrix management.

310
00:21:10,280 --> 00:21:13,490
Well, this says these same engineers may be
assigned to different projects and report

311
00:21:13,490 --> 00:21:16,110
to a project manager while working on that
project.

312
00:21:16,110 --> 00:21:20,510
Therefore, each engineer may have to work
under several managers to get his or her job

313
00:21:20,510 --> 00:21:20,760
done.

314
00:21:20,610 --> 00:21:21,680
And that is what I was saying.

315
00:21:21,680 --> 00:21:28,680
It just turns out that you do have that type
of problem in doing that.

316
00:21:31,660 --> 00:21:34,260
Proponents suggest there are two advantages
to matrix management.

317
00:21:34,260 --> 00:21:38,110
First, it allows team members to share information
more readily across task boundaries.

318
00:21:38,110 --> 00:21:43,550
Second, it allows for a specialization that
can increase the depth of knowledge.

319
00:21:43,550 --> 00:21:48,590
You can see how, if you have a matrix manager,
you can call on more resources to go back

320
00:21:48,590 --> 00:21:54,990
to your home organization facilities in terms
of testing.

321
00:21:54,990 --> 00:22:00,650
I have worked under, and when I was in industry,
total project organizations.

322
00:22:00,650 --> 00:22:04,010
And it has a lot of advantages and disadvantages
also.

323
00:22:04,010 --> 00:22:10,230
The disadvantage of matrix management is that
employees can be confused due to conflicting

324
00:22:10,230 --> 00:22:10,680
loyalties.

325
00:22:10,680 --> 00:22:15,550
I mean was Walt Guy loyal to Max Faget or
was he loyal to Aaron Cohen?

326
00:22:15,550 --> 00:22:22,550
Now, Max actually gave him in raises but
I controlled the dollars for the project.

327
00:22:30,930 --> 00:22:33,190
But, you're right, that is what counts.

328
00:22:33,190 --> 00:22:34,130
They call that pink ticket.

329
00:22:34,130 --> 00:22:37,870
He was actually pink ticketed to Max and actually
worked for me.

330
00:22:37,870 --> 00:22:44,870
And I had to use all my persuasion to get
him to do the type of job.

331
00:22:49,060 --> 00:22:55,550
Properly management cooperative environment,
however, can neutralize these disadvantages.

332
00:22:55,550 --> 00:22:59,360
In Apollo there was one program.

333
00:22:59,360 --> 00:23:00,100
Right.

334
00:23:00,100 --> 00:23:01,580
[AUDIENCE QUESTION]

335
00:23:01,580 --> 00:23:08,220
For example, we had Air Force, Navy and NASA.

336
00:23:08,220 --> 00:23:08,960
Right.

337
00:23:08,960 --> 00:23:15,040
And we were doing accuracy evaluation for
each one.

338
00:23:15,040 --> 00:23:20,250
And we were developing the whole software
from scratch for each one.

339
00:23:20,250 --> 00:23:24,870
It was different language and different computers
and you couldn't compare results.

340
00:23:24,870 --> 00:23:25,400
Right.

341
00:23:25,400 --> 00:23:31,530
And when you have one engineering organization
that does the same at least you can compare.

342
00:23:31,530 --> 00:23:33,330
You have the check and balance.

343
00:23:33,330 --> 00:23:33,750
Right.

344
00:23:33,750 --> 00:23:35,600
That's a very good point.

345
00:23:35,600 --> 00:23:36,000
Yes, Jerry.

346
00:23:36,000 --> 00:23:36,390
[AUDIENCE QUESTION]

347
00:23:36,390 --> 00:23:43,100
Whether you a project organization, you had
your thumb on the people who had to produce.

348
00:23:43,100 --> 00:23:50,040
In the matrix organization, off times the
engineering manager would shuffle his man

349
00:23:50,040 --> 00:23:53,260
right out from under you to another project.

350
00:23:53,260 --> 00:23:55,630
Oh, that's right.

351
00:23:55,630 --> 00:23:57,880
That's what I had to contend with.

352
00:23:57,880 --> 00:24:04,880
On the other hand, my strong personality and
my good will kept it from going that way.

353
00:24:05,010 --> 00:24:10,050
And there was a certain amount of esprit de
corps that they wanted to get the job done.

354
00:24:10,050 --> 00:24:14,010
In the end it is the people.

355
00:24:14,010 --> 00:24:15,400
It's the people.

356
00:24:15,400 --> 00:24:17,370
It doesn't matter what organization.

357
00:24:17,370 --> 00:24:17,980
That's right.

358
00:24:17,980 --> 00:24:22,770
If the people are wrong, no organization is
going to work.

359
00:24:22,770 --> 00:24:23,270
Well, that's absolutely right.

360
00:24:23,270 --> 00:24:23,730
You have to have the right people.

361
00:24:23,730 --> 00:24:27,620
Well, that's why I brought some of these people
to talk to these students because these people

362
00:24:27,620 --> 00:24:29,310
were the people I worked with.

363
00:24:29,310 --> 00:24:31,710
Even though we argued at times.

364
00:24:31,710 --> 00:24:38,500
There was nobody that argued more than Henry
Pohl and I did, but he is usually always right.

365
00:24:38,500 --> 00:24:44,660
But you can learn to live with it.

366
00:24:44,660 --> 00:24:48,650
Matrix management can put some difficult on
the project managers because they must work

367
00:24:48,650 --> 00:24:52,160
closely with other managers and workers in
order to complete the project.

368
00:24:52,160 --> 00:24:54,900
The functional mangers may have different
goals and objectives.

369
00:24:54,900 --> 00:25:00,640
I was saying this as I present these charts,
but I think you can visualize that.

370
00:25:00,640 --> 00:25:03,790
And don't hesitate to ask questions because
you've got a lot of people here in the room

371
00:25:03,790 --> 00:25:06,410
today that can help answer these questions.

372
00:25:06,410 --> 00:25:06,900
Yes.

373
00:25:06,900 --> 00:25:09,810
The point about checking and balance.

374
00:25:09,810 --> 00:25:11,750
You have several managers.

375
00:25:11,750 --> 00:25:12,230
Yes.

376
00:25:12,230 --> 00:25:19,230
From my understanding, you have maybe a functional
manager and then you have [OVERLAPPING VOICES].

377
00:25:19,510 --> 00:25:22,110
Well, let me tell you what I mean by check
and balance.

378
00:25:22,110 --> 00:25:25,330
Let me give you a specific example.

379
00:25:25,330 --> 00:25:30,760
Let's say Henry Pohl on the reaction control
system wanted to do more testing or wanted

380
00:25:30,760 --> 00:25:33,330
to do more stability testing.

381
00:25:33,330 --> 00:25:38,110
And I looked at my budget and I looked at
where we stood in the schedule, and I said

382
00:25:38,110 --> 00:25:41,390
we just cannot afford that, we're just not
going to do that.

383
00:25:41,390 --> 00:25:42,990
I made an arbitrary decision.

384
00:25:42,990 --> 00:25:49,990
Well, Henry would be very upset about that
and he would go to Max, his functional manager.

385
00:25:50,620 --> 00:25:54,510
And Max would say boy, you're right, Aaron
is crazy.

386
00:25:54,510 --> 00:26:01,230
He would then call a meeting with Chris Kraft
between Henry, Max and myself to decide what

387
00:26:01,230 --> 00:26:02,310
we were going to do.

388
00:26:02,310 --> 00:26:04,400
And then Chris was the final word, the man
you met.

389
00:26:04,400 --> 00:26:05,970
He's a pretty strong guy.

390
00:26:05,970 --> 00:26:11,610
And if he agreed with Max, if Max had a convincing
story, he would turn me around and then we

391
00:26:11,610 --> 00:26:12,480
would do the testing.

392
00:26:12,480 --> 00:26:14,470
And I would just have to find the money some
place.

393
00:26:14,470 --> 00:26:21,470
But if I could convince Chris that Max was
overdoing it then I would win.

394
00:26:22,530 --> 00:26:22,780
And normally I lost, but that is OK.

395
00:26:22,630 --> 00:26:29,120
Do the engineers in any of these organizations
out of matrix management feel that they have

396
00:26:29,120 --> 00:26:30,690
too many managers?

397
00:26:30,690 --> 00:26:33,820
Well, it is frustrating to them a little bit.

398
00:26:33,820 --> 00:26:34,780
Yes, they did.

399
00:26:34,780 --> 00:26:40,090
It's frustrating because they had to feel
that they were working for two different people.

400
00:26:40,090 --> 00:26:44,170
And it is a little bit confusing to them.

401
00:26:44,170 --> 00:26:45,910
I mean they had two loyalties.

402
00:26:45,910 --> 00:26:47,360
That's what I pointed out.

403
00:26:47,360 --> 00:26:49,310
They essentially had two different loyalties.

404
00:26:49,310 --> 00:26:55,650
One to their functional manager who paid their
salaries and the other to me who really had

405
00:26:55,650 --> 00:26:59,210
the money for the project and the schedule.

406
00:26:59,210 --> 00:27:02,070
So, yes, it is a very tough thing to do.

407
00:27:02,070 --> 00:27:07,530
But, as you could tell, by talking to these
people that experienced it both in Apollo

408
00:27:07,530 --> 00:27:12,610
and in the early days of Shuttle, they were
very satisfied with the results.

409
00:27:12,610 --> 00:27:19,610
That is why it is very frustrating to some
of us to see the way things are going because

410
00:27:22,250 --> 00:27:27,860
we carried over most of these people you talked
to or are here in the room today worked on

411
00:27:27,860 --> 00:27:32,590
Apollo and then went and worked on Shuttle.

412
00:27:32,590 --> 00:27:34,830
We had the same process.

413
00:27:34,830 --> 00:27:41,830
We had the same Joe Shea, George Low process
that we did.

414
00:27:53,580 --> 00:27:56,550
Yes.

415
00:27:56,550 --> 00:28:02,490
[AUDIENCE QUESTION]

416
00:28:02,490 --> 00:28:09,490
Well, that's absolutely true.

417
00:28:14,380 --> 00:28:19,940
Your decision-making process in matrix management
turns out to be a little bit more cumbersome

418
00:28:19,940 --> 00:28:23,320
than if you have direct control.

419
00:28:23,320 --> 00:28:28,800
In other words, if you have your thumb on
everybody you can get a decision pretty quickly.

420
00:28:28,800 --> 00:28:35,700
On the other hand, you can violate some good
checks and balances so you give up the process

421
00:28:35,700 --> 00:28:41,250
of a rapid decision-making process versus
having some good checks and balances.

422
00:28:41,250 --> 00:28:46,400
Does that answer your thoughts?

423
00:28:46,400 --> 00:28:46,650
OK.

424
00:28:46,620 --> 00:28:48,430
I am going to leave that subject, unless there
are any other questions on it.

425
00:28:48,430 --> 00:28:50,480
Can I just bring up something?

426
00:28:50,480 --> 00:28:50,830
Sure.

427
00:28:50,830 --> 00:28:54,250
And I welcome your opinion on this, Elliot.

428
00:28:54,250 --> 00:29:01,000
And that is there has been a big change in
NASA recently which is affecting management.

429
00:29:01,000 --> 00:29:07,340
And that is because NASA has gone over to
what they call full cost accounting which

430
00:29:07,340 --> 00:29:14,340
means that during all of this time the engineering
organization maintained their engineers.

431
00:29:14,670 --> 00:29:15,880
That's true.

432
00:29:15,880 --> 00:29:22,309
They paid the salary and then they would be
assigned to projects as required.

433
00:29:22,309 --> 00:29:29,309
The system, as it's now working, is it's supposed
to be much more like in industry where the

434
00:29:29,610 --> 00:29:33,870
time of every engineer has to be billed to
a specific project.

435
00:29:33,870 --> 00:29:40,770
And if you don't have a project to bill your
time to, ultimately you become redundant.

436
00:29:40,770 --> 00:29:47,640
And so, those of you who read Aviation Week
and Space News, I hope all of you do, are

437
00:29:47,640 --> 00:29:52,640
aware that NASA is doing a lot of downsizing.

438
00:29:52,640 --> 00:29:54,820
It puts a lot of constraint on management.

439
00:29:54,820 --> 00:30:00,770
Now, in industry, obviously people have to
be paid for.

440
00:30:00,770 --> 00:30:06,780
But are there always enough projects to bill
people to?

441
00:30:06,780 --> 00:30:13,780
Well, there is never a situation that you
have the exact number of people.

442
00:30:26,180 --> 00:30:33,180
You're either short of many people, which
is now the situtation at Draper, or you don't

443
00:30:56,270 --> 00:31:03,270
have enough money.

444
00:31:04,390 --> 00:31:09,040
What you do, in a place like Draper, we have
our own money.

445
00:31:09,040 --> 00:31:09,740
It's called internal research.

446
00:31:09,740 --> 00:31:10,980
And we use some money [NOISE OBSCURES]

447
00:31:10,980 --> 00:31:11,860
and we keep those people.

448
00:31:11,860 --> 00:31:14,510
But those have to be people the company believes
can be used in the future.

449
00:31:14,510 --> 00:31:16,270
There is no point in keeping people that nobody
wants.

450
00:31:16,270 --> 00:31:19,610
Actually, the matrix system, and I talked
about it with Jerry because we are doing the

451
00:31:19,610 --> 00:31:22,330
evaluation, that is why I have to leave, is
really sorting the people out.

452
00:31:22,330 --> 00:31:23,350
Because when it was a big project, it was
a big family.

453
00:31:23,350 --> 00:31:23,600
The guy owned them.

454
00:31:23,559 --> 00:31:23,809
He knows them.

455
00:31:23,710 --> 00:31:23,960
That's true.

456
00:31:23,809 --> 00:31:24,950
Now, you work at the function organization
and we send you to work, he wants the best

457
00:31:24,950 --> 00:31:25,300
people for his project.

458
00:31:25,300 --> 00:31:25,740
He will say well, I don't want this guy.

459
00:31:25,740 --> 00:31:27,470
Now, if it is a mediocre guy, we will say
look, you cannot have the best.

460
00:31:27,470 --> 00:31:28,940
But if it is somebody who is very weak we
will say for one time, OK, maybe he didn't

461
00:31:28,940 --> 00:31:30,550
get along with the program so put him in another
program.

462
00:31:30,550 --> 00:31:32,760
But if we get the feedback, it doesn't work
there, we know there is a problem.

463
00:31:32,760 --> 00:31:38,030
In that way we weeded out a lot of people
that, if you have the old system, probably

464
00:31:38,030 --> 00:31:41,040
will stay and make us less efficient.

465
00:31:41,040 --> 00:31:44,480
NASA will be more efficient now, I think.

466
00:31:44,480 --> 00:31:46,640
I think that is true.

467
00:31:46,640 --> 00:31:48,450
On the other hand, it causes a lot of pain.

468
00:31:48,450 --> 00:31:50,270
It is very difficult to let people go but
that is part of management.

469
00:31:50,270 --> 00:31:57,140
It's not just going to parties.

470
00:31:57,140 --> 00:31:59,429
[UNINTELLIGIBLE PHRASE]

471
00:31:59,429 --> 00:32:01,720
That's right.

472
00:32:01,720 --> 00:32:05,330
Yes, there is a question.

473
00:32:05,330 --> 00:32:11,160
I guess another angle on the "very tough to
let people go" I've noticed that it is extremely

474
00:32:11,160 --> 00:32:18,160
difficult from a bureaucratic standpoint to
let somebody go in government.

475
00:32:19,420 --> 00:32:26,420
And I'm just wondering with full cost accounting,
OK, you've decided it is time for this person

476
00:32:30,770 --> 00:32:33,000
to go, but there is just a lot of politics
and bureaucracy paperwork to go through to

477
00:32:33,000 --> 00:32:33,429
get somebody out of the organization.

478
00:32:33,429 --> 00:32:35,540
There are civil service laws which have to
be complied with.

479
00:32:35,540 --> 00:32:36,070
It is difficult.

480
00:32:36,070 --> 00:32:38,780
And, I have to say, I agree with Elliot.

481
00:32:38,780 --> 00:32:44,490
In the long-run I think NASA will become more
efficient.

482
00:32:44,490 --> 00:32:48,100
Well, to answer your question, there are several
ways to let people go.

483
00:32:48,100 --> 00:32:50,590
One is if they've really done something bad.

484
00:32:50,590 --> 00:32:54,070
You have a board and you go through it and
you release them.

485
00:32:54,070 --> 00:33:00,900
The other is if you NASA has a downsizing,
which is called a RIF, reduce in force.

486
00:33:00,900 --> 00:33:04,040
Then you can arbitrarily let people go.

487
00:33:04,040 --> 00:33:09,040
I didn't mean to interrupt you, Jeff, but
those are two ways.

488
00:33:09,040 --> 00:33:16,040
I will just add the third way, which NASA
is also doing, is you can offer what they

489
00:33:16,309 --> 00:33:16,920
call buyouts.

490
00:33:16,920 --> 00:33:23,920
They offer up to $25,000 if you will take
early retirement, so a lot of people do that.

491
00:33:29,000 --> 00:33:32,240
And companies, not government, have a much
easier time.

492
00:33:32,240 --> 00:33:37,270
I don't know how Draper is, but companies
have a much easier time letting people go

493
00:33:37,270 --> 00:33:39,620
than other places.

494
00:33:39,620 --> 00:33:42,080
But that was about all I was going to say
on management.

495
00:33:42,080 --> 00:33:48,820
Now I'm going to get into, you might say,
the interesting land of systems engineering

496
00:33:48,820 --> 00:33:51,100
and talk about systems engineering a little
bit.

497
00:33:51,100 --> 00:33:54,150
My charts are sort of yellow because they
go way back.

498
00:33:54,150 --> 00:33:59,120
It turns out systems engineering has been
studied a long time a lot.

499
00:33:59,120 --> 00:34:06,120
And there are many aspects of systems engineering
that some people in the room may or may not

500
00:34:06,179 --> 00:34:06,910
agree with.

501
00:34:06,910 --> 00:34:13,909
I've had time to reflect a little bit from
my days at NASA in both Apollo and Shuttle

502
00:34:14,469 --> 00:34:17,399
and in Space Station.

503
00:34:17,399 --> 00:34:24,399
And I was, as I say, Director of the Johnson
Space Center, I was Deputy Administrator,

504
00:34:25,089 --> 00:34:31,960
Acting Deputy Administration under Goldin
for a year, and then I taught at A&M.

505
00:34:31,960 --> 00:34:37,668
And all that time I have had time to reflect
and I have had time to do some, you might

506
00:34:37,668 --> 00:34:42,219
say, research of what I thought systems engineering
was and how you explain it.

507
00:34:42,219 --> 00:34:42,480
I want to try to do that today.

508
00:34:42,480 --> 00:34:47,389
People in the room may or may not agree with
it but it is my thought process.

509
00:34:47,389 --> 00:34:54,389
And it turns out that much of this information
was gathered from studies done by the military,

510
00:34:54,659 --> 00:34:56,099
mil specs on systems engineering.

511
00:34:56,099 --> 00:35:00,470
They have spent an awful lot of money studying
systems engineering.

512
00:35:00,470 --> 00:35:05,160
And many of these charts are plagiarized,
you might say, from those studies and they

513
00:35:05,160 --> 00:35:07,140
look sort of like they are yellow.

514
00:35:07,140 --> 00:35:08,019
And they are.

515
00:35:08,019 --> 00:35:13,589
But let's start off with systems engineering
heritage.

516
00:35:13,589 --> 00:35:18,630
Systems engineering, as we pointed out, is
not new.

517
00:35:18,630 --> 00:35:22,579
The pyramids, you know, that was a real systems
engineering problem.

518
00:35:22,579 --> 00:35:25,160
I don't know how they did it but it was a
real systems engineering problem.

519
00:35:25,160 --> 00:35:27,099
I don't know if even Jim Nevins could do that.

520
00:35:27,099 --> 00:35:29,930
That was a tough job.

521
00:35:29,930 --> 00:35:32,680
Broadcasting service and standards.

522
00:35:32,680 --> 00:35:36,499
Now, most of the time when you talk to people
about systems engineering, in fact, I've done

523
00:35:36,499 --> 00:35:41,769
a lot of research on that, they talk about
systems engineering as operations research.

524
00:35:41,769 --> 00:35:44,739
Well, I don't think that can be further from
the truth of systems engineering.

525
00:35:44,739 --> 00:35:50,160
I think it is a tool maybe used in systems
engineering, but it is not really systems

526
00:35:50,160 --> 00:35:50,630
engineering.

527
00:35:50,630 --> 00:35:54,140
The RAND Corporation developed the systems
analysis.

528
00:35:54,140 --> 00:35:55,609
The Bell Labs telephone systems.

529
00:35:55,609 --> 00:35:57,190
That was, of course, where Joe Shea came from.

530
00:35:57,190 --> 00:36:01,460
Joe Shea came from the Bell Labs at the time.

531
00:36:01,460 --> 00:36:03,799
NASA military systems, and these are mil standards.

532
00:36:03,799 --> 00:36:06,109
And the Army.

533
00:36:06,109 --> 00:36:11,059
And then systems engineering management guide
for Belvoir Defense Systems Management College.

534
00:36:11,059 --> 00:36:16,849
There were a lot of studies done for systems
engineering.

535
00:36:16,849 --> 00:36:22,220
And we, in Apollo, were trying to do systems
engineering in the 1960s for Apollo, is when

536
00:36:22,220 --> 00:36:23,999
we tried to start to do it.

537
00:36:23,999 --> 00:36:30,999
By the way, these charts are not on the Web
but I will give these to Jeff and they will

538
00:36:43,859 --> 00:36:44,849
be on the Web.

539
00:36:44,849 --> 00:36:51,849
Trans-highlighting the need for increased
systems engineering, there is a need to manage

540
00:36:52,039 --> 00:36:58,890
the total picture in the area of increasing
system size and cost, but even a small project

541
00:36:58,890 --> 00:37:01,579
has systems engineering.

542
00:37:01,579 --> 00:37:08,579
If you can think of how you design a bicycle,
how you design an automobile, I mean these

543
00:37:10,390 --> 00:37:12,369
are very complex problems.

544
00:37:12,369 --> 00:37:13,079
They are not simple.

545
00:37:13,079 --> 00:37:17,470
They are really systems engineering problems.

546
00:37:17,470 --> 00:37:24,470
But the increasing technological growth and
specialization, increasing systems complexity,

547
00:37:25,529 --> 00:37:29,910
you know, the PC is a fantastic systems engineering
problem.

548
00:37:29,910 --> 00:37:36,910
And I guess Bill Gates saw the need for a
true systems engineering job and made a lot

549
00:37:36,999 --> 00:37:42,420
of money on it in terms of what he did understanding
systems engineering.

550
00:37:42,420 --> 00:37:46,460
Increasing sensitivity to environmental factors
and increasing costs of life cycle.

551
00:37:46,460 --> 00:37:52,410
All those things make the trend in systems
engineering important.

552
00:37:52,410 --> 00:37:59,410
And, as I look at your reports, wherever you
got it, I think you have a pretty good knowledge

553
00:38:03,410 --> 00:38:06,680
of what systems engineering is.

554
00:38:06,680 --> 00:38:11,269
So, I am not sure this lecture is going to
help you that much because I think you do

555
00:38:11,269 --> 00:38:13,710
look at it from a systems engineering standpoint.

556
00:38:13,710 --> 00:38:16,289
By the way, my last topic is going to be cost
analysis.

557
00:38:16,289 --> 00:38:20,819
I'm sure you're waiting to hear that because
cost analysis is very hard for you to come

558
00:38:20,819 --> 00:38:21,329
to grips with.

559
00:38:21,329 --> 00:38:27,799
And I am going to talk a little bit about
how you would go about doing a cost analysis

560
00:38:27,799 --> 00:38:30,549
which may not dot all the Is and cross all
the Ts for you.

561
00:38:30,549 --> 00:38:32,009
But I am going to try to do that.

562
00:38:32,009 --> 00:38:36,769
We're going to take a break at about 11:00
and then I will have some time to go through

563
00:38:36,769 --> 00:38:37,480
cost analysis.

564
00:38:37,480 --> 00:38:39,140
Well, what is systems engineering?

565
00:38:39,140 --> 00:38:39,749
I don't know.

566
00:38:39,749 --> 00:38:45,150
You may not agree with this, but systems engineering
is the application of scientific and engineering

567
00:38:45,150 --> 00:38:52,150
to transfer an operational need into a description
of system performance parameters and of systems

568
00:38:53,489 --> 00:38:56,960
configuration through an iterative process.

569
00:38:56,960 --> 00:38:58,809
Now, let me give you a needs statement.

570
00:38:58,809 --> 00:39:05,259
When President Kennedy said we are going to
send men to the Moon and return them safety

571
00:39:05,259 --> 00:39:08,289
within a decade that is a perfect needs statement.

572
00:39:08,289 --> 00:39:15,039
It doesn't tell you how to do it, he didn't
put a cost constraint on it, he did put a

573
00:39:15,039 --> 00:39:19,829
time constraint on it, and he told you what
you had to do but didn't tell you anything

574
00:39:19,829 --> 00:39:21,539
about how to do it.

575
00:39:21,539 --> 00:39:24,460
That is a very good needs statement.

576
00:39:24,460 --> 00:39:31,460
I don't know if he did that for that reason,
but he gave the public and NASA a very good

577
00:39:31,589 --> 00:39:32,890
needs statement.

578
00:39:32,890 --> 00:39:38,619
Now, our job was to transform an operational
need into a description of system performance

579
00:39:38,619 --> 00:39:41,319
parameters and the system configuration through
an iterative process.

580
00:39:41,319 --> 00:39:44,460
And that is an exercise left to the student.

581
00:39:44,460 --> 00:39:48,009
That is not easy to do, but that is really
what you need to do.

582
00:39:48,009 --> 00:39:52,269
Now, to integrate technical parameters and
insure capability of all interfaces, that

583
00:39:52,269 --> 00:39:53,999
is what I talked about.

584
00:39:53,999 --> 00:39:56,069
You talked about physical, functional and
program interfaces.

585
00:39:56,069 --> 00:40:00,999
They call them different things but basically
interfaces become a very important part of

586
00:40:00,999 --> 00:40:03,160
the systems engineering process.

587
00:40:03,160 --> 00:40:10,160
And the other factors into the total engineering
are to meet cost, schedule and technical performance.

588
00:40:10,170 --> 00:40:11,309
What is that?

589
00:40:11,309 --> 00:40:13,759
I think Professor Hoffman talked about that.

590
00:40:13,759 --> 00:40:15,769
That is the three-legged stool.

591
00:40:15,769 --> 00:40:18,359
It is schedule, performance and cost.

592
00:40:18,359 --> 00:40:21,349
It is a continuous tradeoff because you are
going to have a criteria.

593
00:40:21,349 --> 00:40:27,769
Whereas, President Kennedy didn't give us
a cost, he did give us a schedule and, to

594
00:40:27,769 --> 00:40:28,910
a certain extent, performance.

595
00:40:28,910 --> 00:40:31,089
But he didn't give us a cost.

596
00:40:31,089 --> 00:40:35,109
Now, as it turns out, cost always becomes
a factor.

597
00:40:35,109 --> 00:40:36,720
It always becomes a factor.

598
00:40:36,720 --> 00:40:43,720
But those were the things that really comprised,
you might say, in simple-minded terms systems

599
00:40:44,950 --> 00:40:45,979
engineering.

600
00:40:45,979 --> 00:40:48,279
I cannot emphasize those enough.

601
00:40:48,279 --> 00:40:53,579
Now, I don't know if that is how you visualize
systems engineering or how everybody in the

602
00:40:53,579 --> 00:40:59,460
room visualizes systems engineering but that
and the design process is really what systems

603
00:40:59,460 --> 00:41:00,940
engineering and design is all about.

604
00:41:00,940 --> 00:41:07,660
Now, whether we utilize that always in design
of our projects I cannot say.

605
00:41:07,660 --> 00:41:13,119
I think we could have done a better upfront
job in systems engineering on the Shuttle.

606
00:41:13,119 --> 00:41:14,599
I really think we could have.

607
00:41:14,599 --> 00:41:19,410
I see things we could have done differently
on the Shuttle today if we really practiced

608
00:41:19,410 --> 00:41:20,869
a true systems engineering process.

609
00:41:20,869 --> 00:41:26,690
It is very difficult to do because you can
stay in this systems engineering loop a long

610
00:41:26,690 --> 00:41:29,269
time and not get anything done.

611
00:41:29,269 --> 00:41:36,269
There is an old adage, if you never get out
of this loop, you could do something, do something

612
00:41:36,390 --> 00:41:41,200
and do something and never get anything done,
so you've got to be very careful with that.

613
00:41:41,200 --> 00:41:44,509
What is the role of systems engineering?

614
00:41:44,509 --> 00:41:51,509
Technical customer interface requirements
definition.

615
00:41:51,549 --> 00:41:56,880
Requirements definition sound very, very simple,
but requirements definition is probably one

616
00:41:56,880 --> 00:42:00,589
of the hardest things you have to do, is to
find out what the true requirements are.

617
00:42:00,589 --> 00:42:05,759
And, unfortunately, requirements do change
as a function of time as you go through the

618
00:42:05,759 --> 00:42:07,150
phase of the program.

619
00:42:07,150 --> 00:42:13,940
But requirements definition is very, very
difficult to do and very important to come

620
00:42:13,940 --> 00:42:14,940
to grips with.

621
00:42:14,940 --> 00:42:21,720
As JR Thompson talked to you, I don't know
if you recall JR's review of the Shuttle main

622
00:42:21,720 --> 00:42:22,559
engine, SSME.

623
00:42:22,559 --> 00:42:29,559
He said had they widened the throat a little
bit they would have reduced the complexity

624
00:42:30,479 --> 00:42:35,299
of the design and development of the main
engine.

625
00:42:35,299 --> 00:42:38,670
Made it much more reliable but you would give
up some performance.

626
00:42:38,670 --> 00:42:42,339
Now, why didn't we do that?

627
00:42:42,339 --> 00:42:46,410
Because there was a requirement to advance
technology.

628
00:42:46,410 --> 00:42:50,519
And I think JR made a very, very important
point there that those are things you need

629
00:42:50,519 --> 00:42:52,069
to look at early in the program.

630
00:42:52,069 --> 00:42:55,979
So, requirements definition are very important,
extremely important.

631
00:42:55,979 --> 00:42:59,400
And it is very difficult to get to the bottom
of it.

632
00:42:59,400 --> 00:43:04,579
Requirements definition is a task in itself.

633
00:43:04,579 --> 00:43:09,089
And then, of course, there is a requirements
management analysis and flow down audit.

634
00:43:09,089 --> 00:43:11,470
And then, again, you get interface management.

635
00:43:11,470 --> 00:43:15,289
Risk management is becoming a big item today.

636
00:43:15,289 --> 00:43:19,369
You heard Bob Siemens talk, and I didn't mention
Bob Siemens.

637
00:43:19,369 --> 00:43:26,369
He is another very, very important man in
this whole era of Apollo and down to Shuttle.

638
00:43:27,369 --> 00:43:31,420
His role cannot be stated too highly.

639
00:43:31,420 --> 00:43:34,489
He did a fantastic job.

640
00:43:34,489 --> 00:43:37,940
His risk management was very interesting.

641
00:43:37,940 --> 00:43:40,210
I heard him talk not too long ago.

642
00:43:40,210 --> 00:43:45,869
And he said that the risk management in terms
of cost and in terms of human life for the

643
00:43:45,869 --> 00:43:52,589
Apollo program was accepted, not only by the
executive branch, the legislative branch but

644
00:43:52,589 --> 00:43:53,920
the public.

645
00:43:53,920 --> 00:43:55,400
So, risk management needs to be accepted.

646
00:43:55,400 --> 00:43:58,420
And that is one thing that is very, very important.

647
00:43:58,420 --> 00:44:05,420
Understand what the risk management is in
terms of life, in terms of cost and in terms

648
00:44:05,950 --> 00:44:07,069
of what it is going to do for you.

649
00:44:07,069 --> 00:44:08,859
So, risk management is very important.

650
00:44:08,859 --> 00:44:13,309
And it is the function of the systems engineer
to bring that to bear.

651
00:44:13,309 --> 00:44:17,539
Performance management, design process, so
forth and so on.

652
00:44:17,539 --> 00:44:18,989
Technology need identifier.

653
00:44:18,989 --> 00:44:25,989
I had the reputation of not wanting to advance
technology mainly because of budget constraints

654
00:44:28,539 --> 00:44:30,569
and I wanted to go with given technology.

655
00:44:30,569 --> 00:44:33,130
Is that right?

656
00:44:33,130 --> 00:44:38,410
I don't know if that is right or not, but
the point being is that technology can drive

657
00:44:38,410 --> 00:44:38,999
a program.

658
00:44:38,999 --> 00:44:44,519
The fact that you have to develop the technology
can drive the program.

659
00:44:44,519 --> 00:44:50,680
These are what the role of a systems engineering
needs to do to figure out how they are going

660
00:44:50,680 --> 00:44:57,680
to handle technology, if they are going to
advance technology or not.

661
00:44:59,849 --> 00:45:03,680
What is a system?

662
00:45:03,680 --> 00:45:10,680
A system is a complete solution to a defined
need in its full environment over the prescribed

663
00:45:11,390 --> 00:45:11,640
life.

664
00:45:11,519 --> 00:45:18,259
The system includes hardware, software, documentation,
human resources, nonhuman resources, esoteric

665
00:45:18,259 --> 00:45:18,720
factors.

666
00:45:18,720 --> 00:45:21,589
It includes almost everything you can think
about.

667
00:45:21,589 --> 00:45:25,259
Software turns out to be a big driver today.

668
00:45:25,259 --> 00:45:28,700
Software in the system turns out to be one
of the largest drivers.

669
00:45:28,700 --> 00:45:34,519
Some of you asked me, and I hope it helped,
what was the cost of the Shuttle Guidance

670
00:45:34,519 --> 00:45:36,039
and Navigation System.

671
00:45:36,039 --> 00:45:40,089
I don't know who asked me for that.

672
00:45:40,089 --> 00:45:41,609
Somebody asked me.

673
00:45:41,609 --> 00:45:46,140
Was that helpful to you, the number we gave
you?

674
00:45:46,140 --> 00:45:47,289
The Shuttle.

675
00:45:47,289 --> 00:45:50,210
They wanted to know the Shuttle.

676
00:45:50,210 --> 00:45:55,680
But the fact is it didn't separate the cost
of the software and the hardware.

677
00:45:55,680 --> 00:46:02,489
And the hardware came on pretty much on cost
but the software was a lot higher.

678
00:46:02,489 --> 00:46:06,079
It was exponentially growing.

679
00:46:06,079 --> 00:46:08,720
Software turns out to be, I don't know, Jerry,
do you want to say a word on software?

680
00:46:08,720 --> 00:46:15,720
Well, the problems of software were the issues
of validation, verifying tests and retests.

681
00:46:22,150 --> 00:46:29,150
And then every once in a while somebody would
come along and say well, but we really ought

682
00:46:34,390 --> 00:46:37,559
to add this.

683
00:46:37,559 --> 00:46:42,650
And adding on was a killer.

684
00:46:42,650 --> 00:46:49,650
Well, as Bill Tenel used to say, it is sort
of the garbage dump for things you couldn't

685
00:46:56,380 --> 00:46:58,359
do in the hardware.

686
00:46:58,359 --> 00:47:05,359
The number that I remember is at some point,
when you wanted to change one line of code,

687
00:47:05,849 --> 00:47:06,329
IBM wanted $1 million.

688
00:47:06,329 --> 00:47:08,349
Each time for one line of code because they
had to change the whole software, so that

689
00:47:08,349 --> 00:47:09,369
is why they have the software control mode
[NOISE OBSCURES].

690
00:47:09,369 --> 00:47:09,619
That's right.

691
00:47:09,579 --> 00:47:11,569
I remember when I retired, Dave Leestma, who
was head of the Astronaut Office at the time,

692
00:47:11,569 --> 00:47:13,069
said that he thought he could get a change
through.

693
00:47:13,069 --> 00:47:13,380
I forgot.

694
00:47:13,380 --> 00:47:13,880
It was something to do with return to launch
site.

695
00:47:13,880 --> 00:47:18,229
I don't remember what it was but it was some
kind of program he wanted to put in.

696
00:47:18,229 --> 00:47:21,450
He talked to everybody and they all approved
it, and it came to me and I turned it down.

697
00:47:21,450 --> 00:47:24,670
And he said at my retirement party, now Aaron
is leaving, maybe we can get it approved.

698
00:47:24,670 --> 00:47:25,739
But software changes do increase the cost
of the program.

699
00:47:25,739 --> 00:47:29,869
A simple system example.

700
00:47:29,869 --> 00:47:30,460
Space Shuttle.

701
00:47:30,460 --> 00:47:30,880
Your house.

702
00:47:30,880 --> 00:47:31,920
Electronic calculator.

703
00:47:31,920 --> 00:47:32,440
Freeway.

704
00:47:32,440 --> 00:47:34,529
The Golden Gate Bridge.

705
00:47:34,529 --> 00:47:35,569
Rapid transit.

706
00:47:35,569 --> 00:47:36,099
Computers.

707
00:47:36,099 --> 00:47:40,619
A system really involves everything.

708
00:47:40,619 --> 00:47:44,819
Almost everything you can think of is a system.

709
00:47:44,819 --> 00:47:49,150
The other thing that is interesting is this
chart right here.

710
00:47:49,150 --> 00:47:53,559
You may have seen this charge before, you've
seen it different ways, but the importance

711
00:47:53,559 --> 00:47:57,509
of systems engineering, the systems engineering
is doing the right thing right.

712
00:47:57,509 --> 00:48:03,479
As system increase in complexity and value,
early system decisions become increasingly

713
00:48:03,479 --> 00:48:03,979
important.

714
00:48:03,979 --> 00:48:10,979
It shows that actually in the impact of program
funding, when 10% of the program funding is

715
00:48:12,380 --> 00:48:17,410
accomplished you have actually committed 90%
of your dollars, systems engineering systems

716
00:48:17,410 --> 00:48:24,239
have actually committed 90% of your dollars,
so that means early decision-making on systems

717
00:48:24,239 --> 00:48:25,079
engineering.

718
00:48:25,079 --> 00:48:30,009
And whether you were going to make the engine
throat wide or not actually impacts the program

719
00:48:30,009 --> 00:48:30,849
very heavily earlier.

720
00:48:30,849 --> 00:48:36,200
That is why systems engineering is so important.

721
00:48:36,200 --> 00:48:43,200
Systems engineering is requirements in solution
management.

722
00:48:48,069 --> 00:48:54,069
Understanding analysis, allocations, so forth
and so on, which really affect performance,

723
00:48:54,069 --> 00:48:54,729
cost and schedule.

724
00:48:54,729 --> 00:48:58,489
And there is the three-legged stool again.

725
00:48:58,489 --> 00:49:03,690
This really says that if you remember certain
parameters, the continued tradeoff between

726
00:49:03,690 --> 00:49:10,559
cost, schedule and performance is really one
of the most important things you can understand

727
00:49:10,559 --> 00:49:17,559
that is really a systems engineering job.

728
00:49:26,710 --> 00:49:33,710
One thing I would like to talk about is the
attributes of a systems engineering.

729
00:49:34,289 --> 00:49:36,569
Systems engineers are very hard to come by.

730
00:49:36,569 --> 00:49:43,119
They really are very hard to come by, and
it is almost hard to tell you how to become

731
00:49:43,119 --> 00:49:43,369
one.

732
00:49:43,359 --> 00:49:49,190
I mean there are courses you can take, but
one thing I would suggest you do, if you really

733
00:49:49,190 --> 00:49:55,640
want to be a systems engineer is you specialize
in one technical area.

734
00:49:55,640 --> 00:50:02,640
Whether it be structures, whether it be thermodynamics,
heat transfer, guidance and control, control

735
00:50:02,930 --> 00:50:05,180
systems, specialize in one area.

736
00:50:05,180 --> 00:50:07,109
That is the first step.

737
00:50:07,109 --> 00:50:14,109
You need to be competent and show your competency
in one area, but then you need to broaden

738
00:50:14,910 --> 00:50:20,369
out a little bit and look at how people actually
operate.

739
00:50:20,369 --> 00:50:24,739
Good judgment of technical competence in others.

740
00:50:24,739 --> 00:50:28,029
Ability to build trust of team customer management.

741
00:50:28,029 --> 00:50:33,359
And this is what this course is about, to
work as a team and to have relationships with

742
00:50:33,359 --> 00:50:33,890
each other.

743
00:50:33,890 --> 00:50:40,890
I don't know if it was true at MIT, but many
years ago in universities it was very bad

744
00:50:41,420 --> 00:50:44,719
to work as a team because you tended to rely
on each other.

745
00:50:44,719 --> 00:50:50,499
And they wanted to have independent research,
independent development, so students were

746
00:50:50,499 --> 00:50:52,819
not really encouraged to work as teams.

747
00:50:52,819 --> 00:50:53,559
I don't know if that is true at MIT but it
was certainly true at some universities I

748
00:50:53,559 --> 00:50:53,809
knew.

749
00:50:53,650 --> 00:50:53,920
I don't know.

750
00:50:53,920 --> 00:50:54,200
Was it true?

751
00:50:54,200 --> 00:50:54,839
Well, we've gone through a major shift.

752
00:50:54,839 --> 00:50:56,239
And I think the rest of the country is as
well from individual to team effort.

753
00:50:56,239 --> 00:50:56,489
Yeah, team effort.

754
00:50:56,430 --> 00:50:56,680
Particularly in engineering.

755
00:50:56,630 --> 00:50:56,880
That's right.

756
00:50:56,759 --> 00:50:59,880
When I first went to A&M, they thought it
was terrible.

757
00:50:59,880 --> 00:51:01,039
Some of the professors thought it was terrible
working as teams.

758
00:51:01,039 --> 00:51:02,219
The difficult that all the top universities
have is you guys are all admitted on the basis

759
00:51:02,219 --> 00:51:03,039
of your high school or I guess in graduate
school your college grads, so those are individual

760
00:51:03,039 --> 00:51:03,380
efforts that got you here.

761
00:51:03,380 --> 00:51:05,140
Now we're saying put the emphasis on team
activity.

762
00:51:05,140 --> 00:51:07,289
The Japanese have traditionally done that
far better than we have.

763
00:51:07,289 --> 00:51:07,569
That's right.

764
00:51:07,569 --> 00:51:09,849
I just wanted to strengthen what you said
because a lot of people think that a systems

765
00:51:09,849 --> 00:51:11,690
engineer should be a "Jack of all trades"
and should know a little bit about everything.

766
00:51:11,690 --> 00:51:12,079
Absolutely not true.

767
00:51:12,079 --> 00:51:15,109
You have to be very good in one specific area,
and that is your baseline.

768
00:51:15,109 --> 00:51:15,359
That's right.

769
00:51:15,319 --> 00:51:22,319
The most difficult thing in engineering as
a manager is to make a decision when the guy

770
00:51:24,809 --> 00:51:31,809
who comes to you knows about the subject more
than you because he is the expert.

771
00:51:51,749 --> 00:51:58,749
And the only way you can do it is by knowing
how an expert behaves.

772
00:52:01,589 --> 00:52:08,589
And, in your area of expertise, you can make
that judgment.

773
00:52:08,910 --> 00:52:15,910
Some people are confused on this, but I think
that I 100% agree with you.

774
00:52:17,489 --> 00:52:24,489
You have to be an expert in one field, otherwise
it won't work.

775
00:52:25,190 --> 00:52:32,190
I should say there have been a lot of discussions
among the faculty in the Aero-Astro Department

776
00:52:35,489 --> 00:52:42,489
and perhaps in other departments as well as
to when systems engineering should be taught.

777
00:52:42,619 --> 00:52:46,279
Should it be taught at the undergraduate level?

778
00:52:46,279 --> 00:52:53,279
And right now the guiding philosophy in the
Aero-Astro Department is to devote undergraduate

779
00:52:55,739 --> 00:53:02,739
time really to building up expertise in the
individual subject matter, as Elliot said.

780
00:53:03,509 --> 00:53:10,509
The idea being that wait until you really
have a good grounding as an engineer in one

781
00:53:10,979 --> 00:53:17,979
or more specific subjects, and then, at the
graduate level, that is time enough to actually

782
00:53:18,710 --> 00:53:23,690
look at systems engineering as a discipline
so that you can draw on the expertise that

783
00:53:23,690 --> 00:53:24,680
you have built up.

784
00:53:24,680 --> 00:53:29,219
That is not a universally held decision.

785
00:53:29,219 --> 00:53:36,219
And there are people who make a good pedagogical
case that you can actually start teaching

786
00:53:36,380 --> 00:53:41,079
principles of systems engineering right from
the freshman level.

787
00:53:41,079 --> 00:53:48,079
And, in the whole CDIO philosophy that our
department tries to follow, we try to develop

788
00:53:50,269 --> 00:53:55,529
these principles which we think will then
be useful as you get into more technical aspects

789
00:53:55,529 --> 00:53:57,079
of systems engineering levels.

790
00:53:57,079 --> 00:54:04,079
We always try to balance the technical expertise
with the systems level thinking.

791
00:54:04,900 --> 00:54:06,259
Yes, Jerry.

792
00:54:06,259 --> 00:54:13,259
I would like to point out that in the organizing
teams, a key important feature of that is

793
00:54:20,130 --> 00:54:22,269
to have that systems engineer who is organizing
the team and setting up the meetings to know

794
00:54:22,269 --> 00:54:23,019
why and who should be attending.

795
00:54:23,019 --> 00:54:27,969
I attended a number of Space Station meetings
when I had these product teams where every

796
00:54:27,969 --> 00:54:30,140
company in the world was there.

797
00:54:30,140 --> 00:54:31,950
And you had 50 people sitting around and there
were only four who were real contributors,

798
00:54:31,950 --> 00:54:32,900
and the others I could swear were falling
asleep.

799
00:54:32,900 --> 00:54:39,900
You fundamentally have to have good management
in team setups.

800
00:54:44,430 --> 00:54:46,609
Well, you're absolutely right.

801
00:54:46,609 --> 00:54:51,829
Of course, one of the problems with Space
Station, in all honesty, if you had tried

802
00:54:51,829 --> 00:54:56,690
to devise a management system that was complex
the Space Station system would have won because

803
00:54:56,690 --> 00:54:57,950
it is probably the most complex management
system.

804
00:54:57,950 --> 00:54:59,269
In fact, I tried to explain it to the Bob
Gilruth one time, who is another man I should

805
00:54:59,269 --> 00:54:59,519
have mentioned.

806
00:54:59,329 --> 00:55:03,029
He was the Director of the Johnson Space Center.

807
00:55:03,029 --> 00:55:07,910
And he came to see me and his eyes just rolled
back in his head.

808
00:55:07,910 --> 00:55:12,089
He couldn't figure out what I was talking
about.

809
00:55:12,089 --> 00:55:13,089
So, you're right.

810
00:55:13,089 --> 00:55:16,779
But, you know, the other person that agrees
with you, Elliot, is Chris Kraft.

811
00:55:16,779 --> 00:55:23,779
Chris Kraft feels very strongly about becoming
an expert in one field before you tackle something.

812
00:55:24,549 --> 00:55:25,359
Self-motivated.

813
00:55:25,359 --> 00:55:26,599
Able to motivate others.

814
00:55:26,599 --> 00:55:33,599
In matrix management, that becomes a good
systems engineer, if you're able to motivate

815
00:55:34,239 --> 00:55:34,690
others.

816
00:55:34,690 --> 00:55:35,130
Methodical.

817
00:55:35,130 --> 00:55:35,579
Analytical.

818
00:55:35,579 --> 00:55:36,029
Intuitive.

819
00:55:36,029 --> 00:55:36,279
Questioning.

820
00:55:36,180 --> 00:55:36,829
Open-minded.

821
00:55:36,829 --> 00:55:41,569
I think that is one of the key points, being
open-minded.

822
00:55:41,569 --> 00:55:48,569
If you make a mistake be ready to accept the
mistake and correct it and make a change to

823
00:55:51,479 --> 00:55:52,209
it.

824
00:55:52,209 --> 00:55:53,950
That is very important in systems engineering,
especially if you have to make some important

825
00:55:53,950 --> 00:55:54,200
decisions.

826
00:55:54,059 --> 00:55:54,309
Confident.

827
00:55:54,180 --> 00:55:55,450
A good communicator.

828
00:55:55,450 --> 00:55:58,069
High level of integrity.

829
00:55:58,069 --> 00:56:04,079
So, that is really some of my background on
systems engineering.

830
00:56:04,079 --> 00:56:10,499
Now, let's talk a little bit about more how
you go about doing it.

831
00:56:10,499 --> 00:56:12,180
Thank you, Elliot.

832
00:56:12,180 --> 00:56:19,180
A system is generally considered a conglomeration
of objects that perform a specific function.

833
00:56:35,319 --> 00:56:36,339
And you can think of those.

834
00:56:36,339 --> 00:56:40,979
I mean you can think of all those in the laptop
computers you have, many things like that

835
00:56:40,979 --> 00:56:43,109
which we talked about.

836
00:56:43,109 --> 00:56:45,400
And I mentioned this before.

837
00:56:45,400 --> 00:56:47,809
This is what I say.

838
00:56:47,809 --> 00:56:51,599
It is a postulate or whatever you want to
call it.

839
00:56:51,599 --> 00:56:57,219
What is important for the whole system is
nearly identical to what would be the best

840
00:56:57,219 --> 00:57:00,609
in the long-run for each of its components.

841
00:57:00,609 --> 00:57:04,190
However, what is best for each individual
constituent may not be the best for the whole

842
00:57:04,190 --> 00:57:04,440
system.

843
00:57:04,269 --> 00:57:08,479
And so that really is something you need to
keep in mind.

844
00:57:08,479 --> 00:57:12,019
That is really the postulate of systems engineering.

845
00:57:12,019 --> 00:57:19,019
It may be the right thing for the system but
it may not be the right thing for the total

846
00:57:22,660 --> 00:57:23,789
configuration.

847
00:57:23,789 --> 00:57:30,789
And the systems engineering process is the
customer/user need analysis, mission requirements,

848
00:57:32,190 --> 00:57:36,180
functional analysis, system concept, development,
tradeoffs, design.

849
00:57:36,180 --> 00:57:42,259
Now, this is systems engineering as it pertains
to design.

850
00:57:42,259 --> 00:57:46,849
Now, there is systems engineering as it pertains
to operations, but this is systems engineering

851
00:57:46,849 --> 00:57:48,479
as it pertains to design.

852
00:57:48,479 --> 00:57:51,989
Design optimization, requirements flow down,
telecommunication, design insurance, verification

853
00:57:51,989 --> 00:57:55,400
audit and then you go through this iteration
process.

854
00:57:55,400 --> 00:57:58,890
In fact, this is what you do in your mind.

855
00:57:58,890 --> 00:57:59,979
I am sure you do that.

856
00:57:59,979 --> 00:58:03,579
When I taught this, when I spoke about this
to several professors, my friends at Texas

857
00:58:03,579 --> 00:58:06,719
A&M, they say well, there is nothing in this.

858
00:58:06,719 --> 00:58:10,509
They said it is all commonsense.

859
00:58:10,509 --> 00:58:13,859
And, to a greater extent, it is commonsense.

860
00:58:13,859 --> 00:58:17,029
But the problem is do you really practice
it?

861
00:58:17,029 --> 00:58:21,150
Do you really have a process in place and
do you really practice it?

862
00:58:21,150 --> 00:58:25,979
And we find that we don't really practice
it too well.

863
00:58:25,979 --> 00:58:28,239
And we say this over and over again.

864
00:58:28,239 --> 00:58:35,239
But the three factors of systems engineering
will be the cost, schedule and performance.

865
00:58:35,849 --> 00:58:39,499
And that is the famous three-legged stool
that we talked about many times.

866
00:58:39,499 --> 00:58:41,969
But that really is the essence.

867
00:58:41,969 --> 00:58:47,079
And, you as an engineer going to work and
getting paid, that is what you're going to

868
00:58:47,079 --> 00:58:50,739
be paid for, getting a product out the door
that essentially meets the cost schedule and

869
00:58:50,739 --> 00:58:50,989
performance.

870
00:58:50,969 --> 00:58:51,219
You look quizzical.

871
00:58:51,019 --> 00:58:51,269
Excuse me.

872
00:58:51,069 --> 00:58:51,319
No, go ahead.

873
00:58:51,160 --> 00:58:51,920
The statistic that you gave.

874
00:58:51,920 --> 00:58:53,599
After 10% of the project funding then you
have 90% of your requirements defined, do

875
00:58:53,599 --> 00:59:00,599
you think that function there might change
today now that we have faster prototyping

876
00:59:18,170 --> 00:59:20,049
tools?

877
00:59:20,049 --> 00:59:21,339
It could change, yes.

878
00:59:21,339 --> 00:59:23,940
This was done some time ago.

879
00:59:23,940 --> 00:59:24,599
Technology is changing.

880
00:59:24,599 --> 00:59:25,880
Rapid prototype is changing.

881
00:59:25,880 --> 00:59:28,640
CAD design.

882
00:59:28,640 --> 00:59:31,219
A lot of things are changing.

883
00:59:31,219 --> 00:59:38,219
It might be not quite as dramatic, you might
say, but it is still going to be there.

884
00:59:42,130 --> 00:59:46,789
A lot of your system engineering decisions
are going to affect your end product very

885
00:59:46,789 --> 00:59:48,299
early in the game.

886
00:59:48,299 --> 00:59:55,299
And one reason why that 90% gets locked in
is because of the difficulty of making changes

887
00:59:58,089 --> 00:59:59,950
once you've got all the plans drawn.

888
00:59:59,950 --> 01:00:00,200
That's right.

889
01:00:00,160 --> 01:00:07,160
And so, to the extent that CAD systems make
it easier to change drawings and propagate

890
01:00:07,900 --> 01:00:14,579
changes throughout the system, it may be that
making certain changes are less expensive

891
01:00:14,579 --> 01:00:15,519
than they might have been.

892
01:00:15,519 --> 01:00:22,519
As Jerry said about software, you start changing
your software around, you've got to do re-verification

893
01:00:27,029 --> 01:00:27,599
and so on.

894
01:00:27,599 --> 01:00:34,599
Certainly, to the degree that modern techniques
might make things more flexible, that number

895
01:00:38,999 --> 01:00:40,869
might get pushed down a little bit.

896
01:00:40,869 --> 01:00:47,869
But generally, once you've gone through preliminary
design review, you have basically set the

897
01:00:49,670 --> 01:00:56,069
parameters of your project and things have
a momentum and it really does get harder to

898
01:00:56,069 --> 01:00:56,319
change.

899
01:00:56,109 --> 01:01:01,369
And, of course, once you've gotten to the
critical design review -- That's right.

900
01:01:01,369 --> 01:01:05,680
At that point you're starting to cut metal.

901
01:01:05,680 --> 01:01:12,680
And then, once you've actually started to
build something, the price of changes goes

902
01:01:12,799 --> 01:01:14,349
up astronomically.

903
01:01:14,349 --> 01:01:21,349
When we did the Apollo Program we started
in '61.

904
01:01:22,130 --> 01:01:29,130
It is my guess that by '64, and not too far
into '64, I remember June of '64 we had the

905
01:01:32,059 --> 01:01:33,319
implementation meetings with Apollo.

906
01:01:33,319 --> 01:01:35,920
By that time all the hardware definition was
in place.

907
01:01:35,920 --> 01:01:36,440
That's right.

908
01:01:36,440 --> 01:01:37,999
The program went on after that.

909
01:01:37,999 --> 01:01:44,999
And the only thing that kept growing or changing
was software, but the basic hardware picture

910
01:01:45,619 --> 01:01:49,749
was pretty much the nail was in, in 1964,
three years into the program.

911
01:01:49,749 --> 01:01:53,680
Well, as Jeff said, once you start cutting
chips and start cutting hardware it is pretty

912
01:01:53,680 --> 01:01:55,660
hard to make changes.

913
01:01:55,660 --> 01:02:00,099
If I could clarify.

914
01:02:00,099 --> 01:02:05,380
What exactly is it that is being locked in?

915
01:02:05,380 --> 01:02:06,529
Well, usually your requirements.

916
01:02:06,529 --> 01:02:11,449
Primarily what your requirements are is the
big driver.

917
01:02:11,449 --> 01:02:18,180
And then, once your requirements are defined,
you can actually start your iterative process

918
01:02:18,180 --> 01:02:20,109
of design.

919
01:02:20,109 --> 01:02:26,329
Once you get your functional performance requirements
defined, you then start your design iteration

920
01:02:26,329 --> 01:02:31,489
and you start locking into things and, pretty
quick, you start cutting hardware and software.

921
01:02:31,489 --> 01:02:36,209
Hopefully you have gotten a sense, from all
of the lectures on the subsystems of the Shuttle,

922
01:02:36,209 --> 01:02:40,719
the degree to which they interact with one
another.

923
01:02:40,719 --> 01:02:42,969
That's a good point.

924
01:02:42,969 --> 01:02:49,969
I mean imagine if halfway down the road there
had been a major change to the thermal protection

925
01:02:52,690 --> 01:02:56,599
system which had bumped the weight up by,
say, 20%.

926
01:02:56,599 --> 01:03:02,420
Imagine the ripples that that would have affected.

927
01:03:02,420 --> 01:03:09,420
Or, if somebody wanted to change the requirements
for cross-range, after you had done the basic

928
01:03:10,140 --> 01:03:10,829
design of the system.

929
01:03:10,829 --> 01:03:14,829
Let me give you a very real example in that.

930
01:03:14,829 --> 01:03:15,979
We had the wing basically designed.

931
01:03:15,979 --> 01:03:21,670
And you know that the landing gear comes up
in the wing box.

932
01:03:21,670 --> 01:03:28,670
Well, we came to the conclusion that if you
blew one tire on touchdown you could blow

933
01:03:28,920 --> 01:03:30,499
all the tires.

934
01:03:30,499 --> 01:03:35,229
And so, what they wanted to do is increase
the number of tires that went into the wheel

935
01:03:35,229 --> 01:03:35,670
well.

936
01:03:35,670 --> 01:03:37,779
And that was pretty late in the program.

937
01:03:37,779 --> 01:03:43,549
I mean we had already decided, from the systems
engineering point of view, we were going to

938
01:03:43,549 --> 01:03:47,499
have the number of wheels we had and go up
into the wheel well.

939
01:03:47,499 --> 01:03:53,299
But then came along a study that showed well,
if you blew one tire you're liable to lose

940
01:03:53,299 --> 01:03:54,509
the vehicle on touchdown.

941
01:03:54,509 --> 01:03:59,999
They wanted to go put more tires into that
wheel well.

942
01:03:59,999 --> 01:04:01,249
Well, can you imagine what that would do?

943
01:04:01,249 --> 01:04:06,630
That would be redesigning the whole wing when
the wing was built, so we lived with that.

944
01:04:06,630 --> 01:04:12,219
We lived with that with some risk, but you
didn't make the change to it.

945
01:04:12,219 --> 01:04:16,329
That's the type of thing that gets into it.

946
01:04:16,329 --> 01:04:20,839
And I'm sure there are a number of items I
could talk about that would cause that problem

947
01:04:20,839 --> 01:04:22,479
to occur.

948
01:04:22,479 --> 01:04:29,479
Sometimes if you come up with a problem that
really would be fatal, you've got to change

949
01:04:34,640 --> 01:04:34,890
it.
You've got to stop, yeah.

950
01:04:36,619 --> 01:04:40,749
And one of the examples being the O ring seal
on the solid booster.

951
01:04:40,749 --> 01:04:45,869
In retrospect, people have said we should
have changed that at the beginning.

952
01:04:45,869 --> 01:04:49,089
Now, who knows?

953
01:04:49,089 --> 01:04:53,569
If on one of the Shuttle flights a tire had
blown and we had lost the entire vehicle,

954
01:04:53,569 --> 01:04:56,160
there would have been an accident investigation
board.

955
01:04:56,160 --> 01:05:01,199
And they would have gone back and said we
were flying with an untenable system.

956
01:05:01,199 --> 01:05:02,690
And we should have made that change.

957
01:05:02,690 --> 01:05:09,690
And that is where the engineer who has become
a manager is really faced with a tough potentially

958
01:05:11,739 --> 01:05:14,009
life or death decision.

959
01:05:14,009 --> 01:05:21,009
And there is always an element of risk, but
if you try to take out all the elements of

960
01:05:22,329 --> 01:05:24,440
risk you will never fly.

961
01:05:24,440 --> 01:05:25,380
Well, that's absolutely right.

962
01:05:25,380 --> 01:05:28,920
And so how do you make that judgment?

963
01:05:28,920 --> 01:05:33,949
As I said, that is why they pay you such a
big salary.

964
01:05:33,949 --> 01:05:37,749
Well, the interesting part about that is after
the Challenger accident my deputy, who became

965
01:05:37,749 --> 01:05:41,019
a very good friend of yours, Paul Whites,
who was an astronaut, a Navy fighter pilot.

966
01:05:41,019 --> 01:05:48,019
And our job was to get the Shuttle flying
again after the Challenger accident.

967
01:05:49,069 --> 01:05:53,759
I thought I brought it with me.

968
01:05:53,759 --> 01:06:00,759
We found this picture of a ship on a very
ominous sea.

969
01:06:01,459 --> 01:06:06,019
It said a ship in the harbor is safe, but
that is not what ships are built for.

970
01:06:06,019 --> 01:06:09,880
You could make it so safe that you could never
fly or you could never fly cross-country or

971
01:06:09,880 --> 01:06:11,229
you could never take a train.

972
01:06:11,229 --> 01:06:12,729
You've got to use some judgment.

973
01:06:12,729 --> 01:06:19,729
And, of course, that risk today, that risk
level today is not very well accepted.

974
01:06:22,269 --> 01:06:29,269
A point I was trying to make is that during
the Apollo Program it was accepted.

975
01:06:35,229 --> 01:06:40,380
We knew we had to take risk.

976
01:06:40,380 --> 01:06:42,119
Tell me when you want to break, Jeff.

977
01:06:42,119 --> 01:06:44,799
Do you want to break now?

978
01:06:44,799 --> 01:06:45,619
This is probably a good time.

979
01:06:45,619 --> 01:06:45,869
OK.

980
01:06:45,749 --> 01:06:46,519
We will take about a five minute break.

981
01:06:46,519 --> 01:06:47,880
I want to get to cost, but I have got a few
more things to say.

982
01:06:47,880 --> 01:06:48,309
Let me talk about process.

983
01:06:48,309 --> 01:06:53,369
We had a contract with Ford Motor Company,
and I showed this to Ford Motor Company of

984
01:06:53,369 --> 01:06:53,619
a design process I would go through.

985
01:06:53,549 --> 01:06:57,099
And they said that is very interesting, we
studied for a long time and this is exactly

986
01:06:57,099 --> 01:07:04,099
what we use to understand the need.

987
01:07:04,130 --> 01:07:04,549
And the need was not the Ford Edsel.

988
01:07:04,549 --> 01:07:07,680
That was not a good needs statement.

989
01:07:07,680 --> 01:07:14,680
But you need a need, you need a need analysis
[OVERLAPPING VOICES].

990
01:07:16,920 --> 01:07:19,430
Well, I tell you.

991
01:07:19,430 --> 01:07:20,869
You need to talk to Henry Pohl.

992
01:07:20,869 --> 01:07:23,449
Henry Pohl, who heard lecture, still has an
Edsel.

993
01:07:23,449 --> 01:07:28,288
Henry is probably one of the few people that
still has an Edsel in his garage.

994
01:07:28,288 --> 01:07:34,449
Henry has a lot of things like that, but Henry
still has an Edsel.

995
01:07:34,449 --> 01:07:38,130
But you need to do a need analysis, understand
what the need means.

996
01:07:38,130 --> 01:07:39,160
You need a function structure.

997
01:07:39,160 --> 01:07:45,890
And, actually, in some of your reports I saw
very good examples of all these.

998
01:07:45,890 --> 01:07:50,940
Now, maybe not in all the reports, but I saw
somebody talking about the needs, somebody

999
01:07:50,940 --> 01:07:55,999
talking about needs analysis, somebody talking
about function structure requirements.

1000
01:07:55,999 --> 01:08:01,489
The hardest thing for engineers to do, especially
young engineers to do, is to make some assumptions

1001
01:08:01,489 --> 01:08:04,619
and constraints and do some preliminary calculations.

1002
01:08:04,619 --> 01:08:07,299
It is very difficult for them to do that.

1003
01:08:07,299 --> 01:08:09,130
They like to see F=ma.

1004
01:08:09,130 --> 01:08:14,799
And I once mentioned that to Joe Shea, and
he said I am glad you were still on your formula

1005
01:08:14,799 --> 01:08:17,469
when I talked about F=ma.

1006
01:08:17,469 --> 01:08:22,380
You need some calculations and then you need
to do some conceptual designs.

1007
01:08:22,380 --> 01:08:25,490
And that really is, you might say, the design
process.

1008
01:08:25,490 --> 01:08:32,490
And you can use various forms of it, but that
is probably the best form of it you can think

1009
01:08:35,460 --> 01:08:39,399
about.

1010
01:08:39,399 --> 01:08:46,399
Systems engineering.

1011
01:08:47,300 --> 01:08:54,300
Interdisciplinary activities.

1012
01:08:55,200 --> 01:08:55,710
Customer need.

1013
01:08:55,710 --> 01:08:57,200
Functional identification requirements.

1014
01:08:57,200 --> 01:09:02,200
Iterative design process with reviews at various
stages.

1015
01:09:02,200 --> 01:09:03,180
Conceptual, preliminary and final designs.

1016
01:09:03,180 --> 01:09:05,229
And that is what Professor Hoffman was talking
about.

1017
01:09:05,229 --> 01:09:12,229
As you go downstream very quickly on that
it becomes harder and harder and harder to

1018
01:09:12,839 --> 01:09:14,068
make changes.

1019
01:09:14,068 --> 01:09:18,920
Let me now stop with systems engineering,
not that I couldn't go longer.

1020
01:09:18,920 --> 01:09:25,720
But let me stop with systems engineering and
talk to you about a subject you're probably

1021
01:09:25,720 --> 01:09:31,130
having more problems with than anything, and
that is how you do a cost analysis.

1022
01:09:31,130 --> 01:09:38,130
And, unfortunately, I am not going to give
you a closed form solution for it.

1023
01:09:38,390 --> 01:09:39,309
There is none.

1024
01:09:39,309 --> 01:09:45,410
Not only that, I did the 90-day study when
President Bush was president.

1025
01:09:45,410 --> 01:09:50,050
I stood on the steps of the Smithsonian.

1026
01:09:50,050 --> 01:09:55,230
Dick Truly appointed me to do it.

1027
01:09:55,230 --> 01:10:01,170
And that study was really paned by everything.

1028
01:10:01,170 --> 01:10:02,940
Because they said give us a cost analysis.

1029
01:10:02,940 --> 01:10:07,309
And I really thought they wanted to know what
it was going to cost so I told them.

1030
01:10:07,309 --> 01:10:10,420
Well, they didn't like it.

1031
01:10:10,420 --> 01:10:16,059
And, in all honesty, I really wasn't that
high because it was a 30-year project.

1032
01:10:16,059 --> 01:10:23,059
I gave them the cost for a 30-year project
actually building launch vehicles, technology.

1033
01:10:23,890 --> 01:10:28,600
But, nevertheless, I had my day in the barrel
on cost analysis.

1034
01:10:28,600 --> 01:10:31,750
Let me talk a little bit about cost estimation
techniques.

1035
01:10:31,750 --> 01:10:33,559
It turns out some of you were trying to use
it.

1036
01:10:33,559 --> 01:10:38,520
Here are the cost estimating techniques.

1037
01:10:38,520 --> 01:10:39,880
Expert interview.

1038
01:10:39,880 --> 01:10:43,410
You talk to an expert in the field, if you
can find one.

1039
01:10:43,410 --> 01:10:44,170
Parametric analysis.

1040
01:10:44,170 --> 01:10:48,210
I am going to spend most of my time talking
about parametric cost analysis because I think

1041
01:10:48,210 --> 01:10:52,010
that is the one that pertains to an engineering
design course.

1042
01:10:52,010 --> 01:10:52,490
Parametrics.

1043
01:10:52,490 --> 01:10:57,920
Because it is in terms that you understand,
weight, performance, that type of thing.

1044
01:10:57,920 --> 01:11:02,340
I am going to talk about that.

1045
01:11:02,340 --> 01:11:03,210
Analogies.

1046
01:11:03,210 --> 01:11:08,870
And then the engineering which is really,
you might say, the grassroots or the buildup

1047
01:11:08,870 --> 01:11:13,670
of what it is going to cost for engineering
hours, manufacturing hours and that type of

1048
01:11:13,670 --> 01:11:14,930
thing, which is pretty hard to do.

1049
01:11:14,930 --> 01:11:16,190
I am going to go through those.

1050
01:11:16,190 --> 01:11:18,480
Now, some of you have tried to do some of
these.

1051
01:11:18,480 --> 01:11:23,340
Some of you have tried to call contractors
to find out what a particular thing was going

1052
01:11:23,340 --> 01:11:23,830
to cost.

1053
01:11:23,830 --> 01:11:28,770
Some of you have tried to find out what the
Shuttle Guidance and Navigation System cost.

1054
01:11:28,770 --> 01:11:32,059
You were trying to do it, but it's not easy
to come by.

1055
01:11:32,059 --> 01:11:35,610
You call a vendor and he says well, he will
quote you, but he wants to know what you're

1056
01:11:35,610 --> 01:11:36,050
going to use it for.

1057
01:11:36,050 --> 01:11:36,690
It is hard to do.

1058
01:11:36,690 --> 01:11:38,750
I know it is very, very hard to do.

1059
01:11:38,750 --> 01:11:44,730
And they want to know how many hundred you
are going to buy.

1060
01:11:44,730 --> 01:11:50,170
I don't minimize the job you have in trying
to do a cost estimate, but you've got to recognize

1061
01:11:50,170 --> 01:11:54,760
that, as an engineer and you go out and start
to work, they are going to want to know what

1062
01:11:54,760 --> 01:12:01,760
this thing is going to cost you and what allies
you have to make.

1063
01:12:01,980 --> 01:12:08,750
Now, these are the cost estimating techniques
over the project cycles.

1064
01:12:08,750 --> 01:12:15,690
You can see, as the project goes along, you
have this so-called parametric method, analogies,

1065
01:12:15,690 --> 01:12:20,280
judgments, system level CERs, cost estimate
relationships.

1066
01:12:20,280 --> 01:12:22,780
And that is a very key thing, cost estimate
relationships.

1067
01:12:22,780 --> 01:12:26,600
I am going to talk more about that, but let
me tell you what a cost estimate relationship

1068
01:12:26,600 --> 01:12:28,640
in a very simple-minded approach is.

1069
01:12:28,640 --> 01:12:31,980
Let's say you're going to go buy a house.

1070
01:12:31,980 --> 01:12:34,610
What is the first thing you want to know about
buying a house?

1071
01:12:34,610 --> 01:12:37,280
How many square feet do you have, right?

1072
01:12:37,280 --> 01:12:41,960
You take how many square feet you're going
to have and you usually know about what it

1073
01:12:41,960 --> 01:12:46,309
cost per square foot in the area you want
to buy a house in.

1074
01:12:46,309 --> 01:12:48,520
You multiply that and you get a rough estimate
of the house.

1075
01:12:48,520 --> 01:12:51,180
Now, there is another factor that goes into
that.

1076
01:12:51,180 --> 01:12:52,580
There is the culture.

1077
01:12:52,580 --> 01:12:57,040
Where you are going to build this house is
going to affect the cost per square foot.

1078
01:12:57,040 --> 01:13:01,070
If you build it in a very expensive neighborhood
-- I always give the relationship in Houston

1079
01:13:01,070 --> 01:13:02,000
because I know Houston.

1080
01:13:02,000 --> 01:13:07,350
If you're going to build it in Clear Lake
where NASA is, it is one culture.

1081
01:13:07,350 --> 01:13:12,640
If you build it in River Oaks where the very
wealth oil people, it is another culture.

1082
01:13:12,640 --> 01:13:17,600
The cost per square foot in Clear Lake is
much lower than the cost per square foot in

1083
01:13:17,600 --> 01:13:18,670
Clear Lake.

1084
01:13:18,670 --> 01:13:21,630
You can get a rough estimate, but that is
a very simple-minded CER.

1085
01:13:21,630 --> 01:13:23,390
It is a cost estimate relationship.

1086
01:13:23,390 --> 01:13:27,790
They become very, very complicated when you
talk about past history, when you talk about

1087
01:13:27,790 --> 01:13:29,330
weight of spacecraft and so forth and so on.

1088
01:13:29,330 --> 01:13:31,430
I will develop a little bit of that.

1089
01:13:31,430 --> 01:13:34,600
General subsystem CERs and calibrated system
CERs.

1090
01:13:34,600 --> 01:13:36,400
But these are the parametric methods.

1091
01:13:36,400 --> 01:13:41,309
And you can see where they are used, analogies
and judgment.

1092
01:13:41,309 --> 01:13:43,400
And they are used for a certain period of
time.

1093
01:13:43,400 --> 01:13:50,400
Then, when you get to the phase B or CD, you
talk about component buildup estimates, detailed

1094
01:13:54,059 --> 01:13:56,880
estimates and vendor quotes.

1095
01:13:56,880 --> 01:13:58,920
That is sort of the schedule.

1096
01:13:58,920 --> 01:14:04,260
And this was provided by the cost technology
over the project cycles by the Lunar and Mars

1097
01:14:04,260 --> 01:14:04,980
Exploration Office.

1098
01:14:04,980 --> 01:14:07,520
That goes way back.

1099
01:14:07,520 --> 01:14:09,540
It has abolished and now restored again.

1100
01:14:09,540 --> 01:14:13,610
So, that is the cycle you go through.

1101
01:14:13,610 --> 01:14:19,750
And, of course, you're in really the phase
A so it is going to be pretty hard for you

1102
01:14:19,750 --> 01:14:21,830
to do the direct methods.

1103
01:14:21,830 --> 01:14:25,190
Even though you might try, it is going to
be pretty hard for you to do that.

1104
01:14:25,190 --> 01:14:26,670
You almost have to use some type of a parametrics.

1105
01:14:26,670 --> 01:14:30,059
And I am going to talk about how you do a
parametric.

1106
01:14:30,059 --> 01:14:37,059
I am going to talk to you about how you do
both of them.

1107
01:14:39,020 --> 01:14:41,750
And this can get you into more trouble.

1108
01:14:41,750 --> 01:14:46,200
Cost estimates get you in a lot of trouble.

1109
01:14:46,200 --> 01:14:52,190
How many in here are familiar with what we
call a work breakdown structure?

1110
01:14:52,190 --> 01:14:53,550
OK, good.

1111
01:14:53,550 --> 01:14:59,690
Work breakdown structure is really the management
tool that is needed in doing a project.

1112
01:14:59,690 --> 01:15:05,940
Whenever you get a project you form a work
breakdown structure which is a hierarchical

1113
01:15:05,940 --> 01:15:08,920
breakdown, the work necessary to complete
a project.

1114
01:15:08,920 --> 01:15:12,960
Work breakdown structure elements should be
identified by title, by numbering system that

1115
01:15:12,960 --> 01:15:14,290
performs the following functions.

1116
01:15:14,290 --> 01:15:19,240
It identifies the level of the breakdown structure
element, it identifies the higher level to

1117
01:15:19,240 --> 01:15:22,470
which the breakdown structure would be integrated
and shows the cost.

1118
01:15:22,470 --> 01:15:23,300
That is the key thing.

1119
01:15:23,300 --> 01:15:28,120
It shows the cost account number of the element
and how much it costs, how much labor hours

1120
01:15:28,120 --> 01:15:28,740
is going to take.

1121
01:15:28,740 --> 01:15:32,430
That is a very, very important tool.

1122
01:15:32,430 --> 01:15:37,350
In fact, when you go out and work in industry,
actually, if you become a member of a work

1123
01:15:37,350 --> 01:15:41,240
breakdown structure, you're graded on how
well you perform under this work breakdown

1124
01:15:41,240 --> 01:15:41,490
structure.

1125
01:15:41,440 --> 01:15:43,170
That is the tool they grade you on.

1126
01:15:43,170 --> 01:15:45,040
We will put these charts on the Internet.

1127
01:15:45,040 --> 01:15:48,890
Roll of the work breakdown structure.

1128
01:15:48,890 --> 01:15:53,450
Project and technical planning and scheduling.

1129
01:15:53,450 --> 01:15:55,140
Cost estimating and budget formulation.

1130
01:15:55,140 --> 01:15:59,730
Project status reporting and plans such as
specifications and drawings.

1131
01:15:59,730 --> 01:16:04,010
Now, those that raised their hands, where
did you learn to do work breakdown structure?

1132
01:16:04,010 --> 01:16:05,570
Can anybody answer me?

1133
01:16:05,570 --> 01:16:09,559
Where did you learn to do that, those that
said you were familiar with it?

1134
01:16:09,559 --> 01:16:11,230
Yes, sir.

1135
01:16:11,230 --> 01:16:13,270
I learned it here at MIT.

1136
01:16:13,270 --> 01:16:13,520
Did you?

1137
01:16:13,520 --> 01:16:13,770
Very good.

1138
01:16:13,770 --> 01:16:14,140
They taught you.

1139
01:16:14,140 --> 01:16:14,510
That's very good.

1140
01:16:14,510 --> 01:16:15,320
What course was it?

1141
01:16:15,320 --> 01:16:15,670
It is not a course.

1142
01:16:15,670 --> 01:16:18,690
It is my research.

1143
01:16:18,690 --> 01:16:20,090
Oh, your research.

1144
01:16:20,090 --> 01:16:24,690
Did you find it hard to do?

1145
01:16:24,690 --> 01:16:31,690
I found it initially difficult, but once I
realized how to work it, I definitely saw

1146
01:16:34,340 --> 01:16:36,150
the benefits of that approach.

1147
01:16:36,150 --> 01:16:36,400
Very good.
Well, it is an important tool that you need
to do.

1148
01:16:36,690 --> 01:16:42,080
And that is why it is probably a little hard
for you to do cost estimation because you

1149
01:16:42,080 --> 01:16:47,740
are not used to some of these things you need
to do to get the cost done.

1150
01:16:47,740 --> 01:16:54,740
Well, some of the basic elements of cost estimation
is understand the product, develop a detailed

1151
01:16:56,190 --> 01:16:57,450
work breakdown structure.

1152
01:16:57,450 --> 01:16:58,380
Did you do that for this project?

1153
01:16:58,380 --> 01:16:58,630
No.

1154
01:16:58,400 --> 01:16:58,970
That is OK.

1155
01:16:58,970 --> 01:17:05,650
I didn't mean to put you under the gun.

1156
01:17:05,650 --> 01:17:07,300
Understand the development of the culture.

1157
01:17:07,300 --> 01:17:08,720
Understand the culture.

1158
01:17:08,720 --> 01:17:15,720
It turns out the NASA culture is pretty high
level because it does involve risk.

1159
01:17:16,360 --> 01:17:21,710
And I will give you a little story about.

1160
01:17:21,710 --> 01:17:28,710
Somebody asked von Braun one time why NASA
had to gold plate everything.

1161
01:17:28,710 --> 01:17:33,700
He said because you made it out of pure gold
it would cost too much.

1162
01:17:33,700 --> 01:17:40,440
It is not that they gold plate everything,
but it is that you really have a life concern

1163
01:17:40,440 --> 01:17:43,010
in a very hostile environment.

1164
01:17:43,010 --> 01:17:46,070
You have to understand the development culture.

1165
01:17:46,070 --> 01:17:50,580
Gather data from close analogs, develop the
cost estimate relationships.

1166
01:17:50,580 --> 01:17:54,640
There are textbooks that show you how to develop
CERs.

1167
01:17:54,640 --> 01:17:59,530
There is a large society called parametric
cost analysis.

1168
01:17:59,530 --> 01:18:00,920
And in that you can get how to develop CERs.

1169
01:18:00,920 --> 01:18:06,790
I am not suggesting you necessarily do that
because it is very, very time consuming, but

1170
01:18:06,790 --> 01:18:09,010
just know that they do exist.

1171
01:18:09,010 --> 01:18:14,480
There are, by the way, very sophisticated
computer programs, which I want to talk about,

1172
01:18:14,480 --> 01:18:18,390
that develop the CERs for you based on inputs
you give them.

1173
01:18:18,390 --> 01:18:20,190
These programs are very expensive.

1174
01:18:20,190 --> 01:18:21,630
One is called PRICE.

1175
01:18:21,630 --> 01:18:24,980
I will show you what the other one is.

1176
01:18:24,980 --> 01:18:28,040
And it was actually developed by RCA.

1177
01:18:28,040 --> 01:18:31,250
It now is owned by the Martin Company.

1178
01:18:31,250 --> 01:18:35,230
Or was owned by the Martin Company.

1179
01:18:35,230 --> 01:18:38,320
Group CERs according to the work breakdown
structure.

1180
01:18:38,320 --> 01:18:41,270
And you need to quantify the risk and the
method.

1181
01:18:41,270 --> 01:18:43,820
Estimate the cost, spread the cost and do
a reality check.

1182
01:18:43,820 --> 01:18:45,600
Those are some of the things you need to do.

1183
01:18:45,600 --> 01:18:52,600
Now, understanding the costing and the analysis
process, as I said, you need to understand

1184
01:19:00,750 --> 01:19:04,400
the project, participate in the design team
activities.

1185
01:19:04,400 --> 01:19:07,750
Scoping, inheritance and complexity.

1186
01:19:07,750 --> 01:19:12,330
Identification of content, schedule and goals.

1187
01:19:12,330 --> 01:19:18,750
Collecting cost, estimating inputs by weight
statements, technical characteristics, preparing

1188
01:19:18,750 --> 01:19:23,250
the estimates, CERs, translating labor hours
and dollar hours.

1189
01:19:23,250 --> 01:19:29,290
And this is a lot of details that you are
not going to have time to do, but you do need

1190
01:19:29,290 --> 01:19:33,790
to understand the complexity that doing cost
analysis entails.

1191
01:19:33,790 --> 01:19:36,600
It is very, very difficult and very hard to
do.

1192
01:19:36,600 --> 01:19:39,100
And you have groups set up to do that.

1193
01:19:39,100 --> 01:19:44,390
But, on the other hand, engineers working
on a project are going to be called upon to

1194
01:19:44,390 --> 01:19:48,640
do it because you are the best source of information
that they have to do it.

1195
01:19:48,640 --> 01:19:55,640
Here is the basic building block of the parametric
cost estimating, is the relationship of the

1196
01:20:04,270 --> 01:20:05,559
cost estimating relationship.

1197
01:20:05,559 --> 01:20:10,000
And the dependent value variable is cost.

1198
01:20:10,000 --> 01:20:13,870
The independent variable is something that
you can understand.

1199
01:20:13,870 --> 01:20:16,420
It is weight, power, ISP and so forth.

1200
01:20:16,420 --> 01:20:17,360
They are specific engineering terms.

1201
01:20:17,360 --> 01:20:24,360
And there is some relationship between weight,
certainly for structures, there is certainly

1202
01:20:26,559 --> 01:20:33,559
a value based on historical data points of
weight of the structure, the type of materials

1203
01:20:35,270 --> 01:20:38,770
and so forth on the CER, the cost relationship.

1204
01:20:38,770 --> 01:20:41,420
So, that is how it is developed.

1205
01:20:41,420 --> 01:20:46,290
A lot of students had a very difficult time
using this program because they didn't know

1206
01:20:46,290 --> 01:20:48,070
how the CERs were developed.

1207
01:20:48,070 --> 01:20:52,230
It frustrated them because here was this magic
program that calculated the CERs for them.

1208
01:20:52,230 --> 01:20:52,800
Right, Josh?

1209
01:20:52,800 --> 01:20:55,270
Josh is one of my students.

1210
01:20:55,270 --> 01:20:58,880
And they had a very difficult time trying
to understand how to use the program because

1211
01:20:58,880 --> 01:21:02,670
they couldn't understand how to do the CERs.

1212
01:21:02,670 --> 01:21:04,820
But there is a methodology in it.

1213
01:21:04,820 --> 01:21:06,440
And there are many books that explain it.

1214
01:21:06,440 --> 01:21:11,340
The other thing that is important, as I pointed
out, is the cultural variable.

1215
01:21:11,340 --> 01:21:16,330
The cultural variable is this.

1216
01:21:16,330 --> 01:21:20,059
See, this is one, unit dry weight.

1217
01:21:20,059 --> 01:21:24,820
And it can show us the total cost, but it
can show that for aircraft you're on one culture

1218
01:21:24,820 --> 01:21:29,920
plane, on an unmanned spacecraft you're on
another cultural plane, on manned spacecraft

1219
01:21:29,920 --> 01:21:36,309
you're on another cultural plane and on a
planetary spacecraft you're on another cultural

1220
01:21:36,309 --> 01:21:36,800
plane.

1221
01:21:36,800 --> 01:21:38,690
That is a very big issue.

1222
01:21:38,690 --> 01:21:43,170
One of the issues with NASA they say, to get
the cost down for NASA, is you need to change

1223
01:21:43,170 --> 01:21:43,980
the cultural plane.

1224
01:21:43,980 --> 01:21:48,200
Of course, if you change the cultural plane
you probably have to give up something.

1225
01:21:48,200 --> 01:21:53,370
You probably would give up some risk in terms
of safety or margin.

1226
01:21:53,370 --> 01:21:58,800
So, you have got to recognize that the cultural
variable is a very strong indicator.

1227
01:21:58,800 --> 01:22:02,110
Are you happy to note that the aircraft is
so low and everybody flies on an airplane,

1228
01:22:02,110 --> 01:22:06,020
but that is the cultural variable that it
shows.

1229
01:22:06,020 --> 01:22:13,020
For a structure, you can see how the structure
varies based on the cultural plane.

1230
01:22:15,920 --> 01:22:22,590
When you were talking about like using square
footage as a rough estimate for a house, and

1231
01:22:22,590 --> 01:22:29,590
in the space business we do use weight as
kind of a substitute.

1232
01:22:31,010 --> 01:22:33,400
You can translate from weight into cost.

1233
01:22:33,400 --> 01:22:39,790
And yet people always point out, well, yeah,
but a kilogram of aluminum structure costs

1234
01:22:39,790 --> 01:22:44,260
a lot less than a kilogram of computer chips.

1235
01:22:44,260 --> 01:22:48,790
But, nevertheless, when you put everything
down when you're designing a planetary spacecraft,

1236
01:22:48,790 --> 01:22:51,190
they do have these estimated relationships.

1237
01:22:51,190 --> 01:22:58,190
If you're going to have a project which lands
150 kilograms on Mars compared to a ton on

1238
01:23:00,990 --> 01:23:03,960
Mars, we do have these relationships.

1239
01:23:03,960 --> 01:23:10,260
Because, in general, we figure that the ratio
of hardware and software and computer chips

1240
01:23:10,260 --> 01:23:14,230
to aluminum and all that is roughly similar.

1241
01:23:14,230 --> 01:23:19,510
But if you are going to try to build something
that is fundamentally new, which involves

1242
01:23:19,510 --> 01:23:26,170
a lot of new technology, then of course you
can go way off of these rough estimates.

1243
01:23:26,170 --> 01:23:31,309
To get a good understanding, you might say
a very quick understanding, you might go to

1244
01:23:31,309 --> 01:23:36,770
the NASA website and get parametric cost analysis.

1245
01:23:36,770 --> 01:23:40,420
It will give you a very good understanding
of what you can get.

1246
01:23:40,420 --> 01:23:44,670
And you can get some of these programs off
the Internet today.

1247
01:23:44,670 --> 01:23:51,200
They are not very sophisticated but they do
have some CER relationships, as Jeff was talking

1248
01:23:51,200 --> 01:23:51,620
about.

1249
01:23:51,620 --> 01:23:58,320
It might be a little bit more difficult for
some of the items.

1250
01:23:58,320 --> 01:23:59,850
Structures would be an easy item to do but,
unfortunately, I don't think anybody is doing

1251
01:23:59,850 --> 01:24:00,100
that.
But it would even be good for the avionics
system, I think, in life support.

1252
01:24:01,870 --> 01:24:02,120
Yes.

1253
01:24:01,950 --> 01:24:05,220
Are these software packages basically just
a large database?

1254
01:24:05,220 --> 01:24:12,220
They are basically a large database that you
put in the data.

1255
01:24:12,610 --> 01:24:19,610
In fact, they are so sophisticated now, you
can put in the software, the type of software

1256
01:24:20,230 --> 01:24:24,309
you are using, the number of lines of code
and, based on the CER for that particular

1257
01:24:24,309 --> 01:24:28,760
subject, it can tell you how much it is going
to cost per line of code.

1258
01:24:28,760 --> 01:24:31,210
It is very, very sophisticated.

1259
01:24:31,210 --> 01:24:31,930
And NASA uses it quite a bit.

1260
01:24:31,930 --> 01:24:32,200
Yes?

1261
01:24:32,200 --> 01:24:39,200
With this software, you must have to update
it very, very frequently because of technology

1262
01:24:41,309 --> 01:24:42,480
changes.

1263
01:24:42,480 --> 01:24:44,980
They update it very frequently.

1264
01:24:44,980 --> 01:24:45,230
That's right.

1265
01:24:45,080 --> 01:24:46,540
And that is why they are so expensive.

1266
01:24:46,540 --> 01:24:48,710
They are very expensive, but they update them.

1267
01:24:48,710 --> 01:24:52,950
For example, I started using it at A&M, and
they didn't have even a software package in

1268
01:24:52,950 --> 01:24:53,340
there for composites.

1269
01:24:53,340 --> 01:24:55,100
Now they have one for composites.

1270
01:24:55,100 --> 01:25:02,100
And there are a lot of these which are proprietary
and you cannot even get access to them.

1271
01:25:03,710 --> 01:25:10,710
I mean the Aerospace Corporation has compiled
a huge database of costs of a lot of components

1272
01:25:12,330 --> 01:25:16,950
historical to how much it was estimated, how
much it actually cost.

1273
01:25:16,950 --> 01:25:23,580
And so NASA and the DOD often go to Aerospace
to do cost analysis for them.

1274
01:25:23,580 --> 01:25:25,100
And, of course, Aerospace charges.

1275
01:25:25,100 --> 01:25:30,390
And they don't release their database because
it is proprietary.

1276
01:25:30,390 --> 01:25:35,090
Knowledge is power and money in this situation.

1277
01:25:35,090 --> 01:25:42,090
They put a lot of work into putting the database
together.

1278
01:25:47,440 --> 01:25:47,690
Yes.

1279
01:25:47,600 --> 01:25:47,920
[AUDIENCE QUESTION]

1280
01:25:47,920 --> 01:25:48,730
Were you able to try to use it?

1281
01:25:48,730 --> 01:25:51,470
Did you try to check it out?

1282
01:25:51,470 --> 01:25:54,850
[AUDIENCE QUESTION]

1283
01:25:54,850 --> 01:26:00,110
And, of course, it may not have the update
as this gentleman was talking about, but it

1284
01:26:00,110 --> 01:26:01,460
has a certain amount.

1285
01:26:01,460 --> 01:26:03,400
You could do a pretty quick analysis.

1286
01:26:03,400 --> 01:26:10,400
I would suggest you try to do that because
I know that I had looked at it, and there

1287
01:26:11,460 --> 01:26:12,160
are ways to do it.

1288
01:26:12,160 --> 01:26:15,150
Now, you've got to be careful of the results
because sometimes the results become pretty

1289
01:26:15,150 --> 01:26:15,830
high.

1290
01:26:15,830 --> 01:26:22,780
But you've got to be careful of the results
and look at it pretty carefully.

1291
01:26:22,780 --> 01:26:25,830
This is another development of cost scaling
law.

1292
01:26:25,830 --> 01:26:32,830
This happens to be weight versus -- You can
see the two different cultural planes, known

1293
01:26:33,380 --> 01:26:36,680
cost and weight, unknown cost from a given
weight.

1294
01:26:36,680 --> 01:26:39,320
This is basically how they generate the CERs.

1295
01:26:39,320 --> 01:26:40,500
It is a process.

1296
01:26:40,500 --> 01:26:43,860
It isn't an analytical process, but there
also is a lot of empirical data that goes

1297
01:26:43,860 --> 01:26:45,480
with it.

1298
01:26:45,480 --> 01:26:52,320
And, as Jeff said, it is based on a large
database.

1299
01:26:52,320 --> 01:26:59,320
Now, cost estimation, I am telling you hard
it is to do and you probably already know

1300
01:26:59,600 --> 01:27:00,110
it.

1301
01:27:00,110 --> 01:27:05,050
But, for a cost estimation, now, this is after
you get over the errors of parametric cost

1302
01:27:05,050 --> 01:27:05,300
analysis.

1303
01:27:05,290 --> 01:27:10,880
Now we're talking about how you do a cost
analysis when you're really trying to figure

1304
01:27:10,880 --> 01:27:15,760
how you go forward to ask for a budget.

1305
01:27:15,760 --> 01:27:18,280
You need fairly detailed information.

1306
01:27:18,280 --> 01:27:23,010
It is unrealistic for designers to give cost
information for conceptualist designs, but

1307
01:27:23,010 --> 01:27:27,230
designers must be able to make rough cost
estimates.

1308
01:27:27,230 --> 01:27:33,160
That is what I am saying, that you do need
to be able to make rough cost estimates.

1309
01:27:33,160 --> 01:27:39,580
Now, if you are trying to make a detailed
cost estimate, and that goes back to the work

1310
01:27:39,580 --> 01:27:45,570
breakdown structure, you need the number of
labor hours and number of people on the project.

1311
01:27:45,570 --> 01:27:49,590
I mean people is probably going to be the
most expensive.

1312
01:27:49,590 --> 01:27:55,350
You need the number of labor hours, the number
of people on the project.

1313
01:27:55,350 --> 01:27:59,630
During the Shuttle Program, Orbiter Project
Manager, I used to call my counterpart at

1314
01:27:59,630 --> 01:28:01,390
Rockwell every Saturday morning.

1315
01:28:01,390 --> 01:28:07,240
And we would go through how many people we
had on each work breakdown structure for each

1316
01:28:07,240 --> 01:28:07,490
contractor.

1317
01:28:07,380 --> 01:28:12,570
We would talk about it to be sure we had the
budget to cover it.

1318
01:28:12,570 --> 01:28:15,430
And then, of course, we had to be sure we
got the product out the door.

1319
01:28:15,430 --> 01:28:19,580
But the number of labor hours, the number
of people on the project, the length of time

1320
01:28:19,580 --> 01:28:24,559
these people are going to be on it, the overhead
rates, office space, computer, benefits, cost

1321
01:28:24,559 --> 01:28:28,260
per hour of a person basically is what you
are looking for, that is how you really do

1322
01:28:28,260 --> 01:28:30,130
a cost estimate.

1323
01:28:30,130 --> 01:28:35,670
And, of course, it is going to vary, depending
on the locale of the contractor.

1324
01:28:35,670 --> 01:28:36,590
What are the cost drivers?

1325
01:28:36,590 --> 01:28:43,590
Now, here is where the cost comes out, some
of the things that Dr.

1326
01:28:44,290 --> 01:28:45,030
Hoffman was talking about.

1327
01:28:45,030 --> 01:28:47,340
The errors in the design.

1328
01:28:47,340 --> 01:28:49,520
Change in requirements.

1329
01:28:49,520 --> 01:28:52,630
Core interface definition.

1330
01:28:52,630 --> 01:28:53,170
Technology development.

1331
01:28:53,170 --> 01:28:54,100
Poor communication.

1332
01:28:54,100 --> 01:28:55,500
Poor teamwork.

1333
01:28:55,500 --> 01:29:02,190
These are all elements of systems engineering
that have gone awry, that have not gone in

1334
01:29:02,190 --> 01:29:03,550
the right direction.

1335
01:29:03,550 --> 01:29:06,140
But this is what caused the cost to go up.

1336
01:29:06,140 --> 01:29:12,090
In my experience, the biggest problem in cost
estimates have been the changes in requirements.

1337
01:29:12,090 --> 01:29:16,710
That is really the thing that has caused most
of the problems.

1338
01:29:16,710 --> 01:29:21,160
That is why you need really good definition
requirements and not change it.

1339
01:29:21,160 --> 01:29:27,020
Technology development, that can cost you
a lot of money and sometimes you have to go

1340
01:29:27,020 --> 01:29:31,590
forward with it.

1341
01:29:31,590 --> 01:29:38,590
Now, this is old data.

1342
01:29:39,280 --> 01:29:43,150
This is 1993 data, but I got this from a contractor
a long time ago.

1343
01:29:43,150 --> 01:29:47,440
First of all what you need to do, and of course
it is a lot different, I don't know if it

1344
01:29:47,440 --> 01:29:47,690
is higher or lower today with CAD systems,
this was before they really used CAD systems,

1345
01:29:47,630 --> 01:29:48,320
but this the average cost of an engineering
drawing.

1346
01:29:48,320 --> 01:29:52,200
And it goes through how you do it.

1347
01:29:52,200 --> 01:29:57,110
It is probably the labor rates are higher
today.

1348
01:29:57,110 --> 01:29:59,210
The average hours per drawing are probably
lower.

1349
01:29:59,210 --> 01:30:02,520
But this says in 1993, the rate is an estimate
for 1993 government year of about $3,000 in

1350
01:30:02,520 --> 01:30:05,710
average drawing, so you have to figure out
how many drawings you need.

1351
01:30:05,710 --> 01:30:12,210
I mean that is a good way to understand the
cost estimate of what it is going to be.

1352
01:30:12,210 --> 01:30:18,210
You need to know how many drawings you need
and you need an understanding of the cost

1353
01:30:18,210 --> 01:30:18,460
settlement.
The elements of cost during manufacturing
then are the number and complexity of the

1354
01:30:25,030 --> 01:30:30,630
drawings, cost of a drawing, number and complexity
of machine and fabricated parts and what the

1355
01:30:30,630 --> 01:30:32,570
parts are made out of.

1356
01:30:32,570 --> 01:30:39,240
Aluminum, steel, titanium, nickel, inconel
and so forth, because some are more complicated

1357
01:30:39,240 --> 01:30:40,760
to machine than others.

1358
01:30:40,760 --> 01:30:46,280
But some are used in the Space Program, some
are not.

1359
01:30:46,280 --> 01:30:50,130
Titanium was becoming more and more important.

1360
01:30:50,130 --> 01:30:57,130
Other things of manufacturing are tolerance
of parts.

1361
01:31:00,010 --> 01:31:04,880
It turns out engineers tend to, you might
say, put extra tolerances on parts that they

1362
01:31:04,880 --> 01:31:05,700
don't need.

1363
01:31:05,700 --> 01:31:08,940
And that can cause parts to be very, very
expensive.

1364
01:31:08,940 --> 01:31:12,880
The finish of parts, the quality of parts,
assembly, complexity and time qualification

1365
01:31:12,880 --> 01:31:13,630
of checking a test.

1366
01:31:13,630 --> 01:31:17,020
I remember my first day at work, when I worked
for RCA, I was designing a microwave tube,

1367
01:31:17,020 --> 01:31:17,270
magnetron.

1368
01:31:17,260 --> 01:31:20,570
I was a terrible draftsman, as a matter of
fact.

1369
01:31:20,570 --> 01:31:20,820
That was a long time ago.

1370
01:31:20,650 --> 01:31:22,550
I would probably be better with CAD systems.

1371
01:31:22,550 --> 01:31:28,760
But I sent out this part to the shop.

1372
01:31:28,760 --> 01:31:35,760
And this guy coming in showing this stogy
out in the machine shop.

1373
01:31:39,850 --> 01:31:45,920
At that time we sat in these big bullpens,
each person at a desk, and I had my name on

1374
01:31:45,920 --> 01:31:46,590
my desk.

1375
01:31:46,590 --> 01:31:53,260
He comes in this big room and says where the
hell is Cohen?

1376
01:31:53,260 --> 01:31:53,530
That is me.

1377
01:31:53,530 --> 01:31:56,930
He said this is the dumbest drawing I have
ever seen in my life.

1378
01:31:56,930 --> 01:31:59,130
How do you expect anybody to make this part?

1379
01:31:59,130 --> 01:32:02,190
So, you have to be sure you understand what
you are doing.

1380
01:32:02,190 --> 01:32:07,920
It turns out I learned to work with that guy
and became good friends.

1381
01:32:07,920 --> 01:32:12,280
That never happened again, but he wanted to
know where the hell was this Cohen who put

1382
01:32:12,280 --> 01:32:12,830
out this dumb drawing?

1383
01:32:12,830 --> 01:32:14,330
He said there is no way I can make this part.

1384
01:32:14,330 --> 01:32:18,580
And that was a small part, but you need to
understand the complexity of the part and

1385
01:32:18,580 --> 01:32:21,670
what you're putting out to the shop.

1386
01:32:21,670 --> 01:32:24,240
But today they have checks and balances.

1387
01:32:24,240 --> 01:32:28,730
And, with the CAD systems, I am not even sure
this is accurate anymore.

1388
01:32:28,730 --> 01:32:32,350
This about brings me to the close of what
I wanted to talk to you about, but let me

1389
01:32:32,350 --> 01:32:34,320
just give you one final chart.

1390
01:32:34,320 --> 01:32:39,090
And then I would like you to ask some questions
about anything I talked about or anything

1391
01:32:39,090 --> 01:32:43,860
anybody else covered that you heard about.

1392
01:32:43,860 --> 01:32:48,220
I would stress that for you today in your
course is to try to use parametric cost analysis.

1393
01:32:48,220 --> 01:32:49,590
That is what I would think you should do.

1394
01:32:49,590 --> 01:32:52,309
At least that is what I think you ought to
do.

1395
01:32:52,309 --> 01:32:54,880
And I think you can get information off the
Internet that will allow you to do something.

1396
01:32:54,880 --> 01:32:56,980
Now, it may not pertain to all projects.

1397
01:32:56,980 --> 01:33:01,990
I am not saying you have to do it, but it
might be the thing to do.

1398
01:33:01,990 --> 01:33:02,510
It is going to be very difficult.

1399
01:33:02,510 --> 01:33:04,420
Doing the right thing by creating a work breakdown
structure in trying to do that, it is going

1400
01:33:04,420 --> 01:33:10,380
to be very difficult for you to do it in this
course.

1401
01:33:10,380 --> 01:33:15,530
But the parametric approach to cost, hardware
costs are not produced from parts less than

1402
01:33:15,530 --> 01:33:17,400
labor tables, so you don't need that.

1403
01:33:17,400 --> 01:33:21,120
They are produced from general measures of
the impact of such items costs and in preparing

1404
01:33:21,120 --> 01:33:26,570
inexpensive and more realistic and manageable
costs based on cost estimating relationships.

1405
01:33:26,570 --> 01:33:31,250
You might want to do some reading on cost
estimating relationships, and there are a

1406
01:33:31,250 --> 01:33:35,180
lot of references to that, just to be sure
you understand what I am talking about and

1407
01:33:35,180 --> 01:33:39,690
what they are so it is not a complete mystery
to you so you get a better feel for what it

1408
01:33:39,690 --> 01:33:40,160
is.

1409
01:33:40,160 --> 01:33:43,990
A lot of textbooks have cost estimating relationships.

1410
01:33:43,990 --> 01:33:50,990
CERs make use of characteristics that can
be readily quantifiable such as weight, size

1411
01:33:51,340 --> 01:33:57,010
and to estimate variables that are difficult
to quantify such as cost and production schedules.

1412
01:33:57,010 --> 01:34:02,400
As I said, parametric models can range from
a simple arithmetical relationship to a sophisticated

1413
01:34:02,400 --> 01:34:08,130
computerized model so they can be very, very
complicated or very simple.

1414
01:34:08,130 --> 01:34:10,700
And I suggest you try to do that.

1415
01:34:10,700 --> 01:34:17,700
Let me just show you what the programs are,
not that you will be able to get them, but

1416
01:34:19,070 --> 01:34:25,410
when you get out of industry, if I can find
them here.

1417
01:34:25,410 --> 01:34:32,410
Well, it may not happen.

1418
01:34:36,930 --> 01:34:39,880
There is certainly a lot of information.

1419
01:34:39,880 --> 01:34:42,059
Here is the price model.

1420
01:34:42,059 --> 01:34:46,270
That is price H.

1421
01:34:46,270 --> 01:34:51,639
And you can actually do structural weight
material types, tolerances.

1422
01:34:51,639 --> 01:34:56,600
You can actually do electronics, digital,
analog technology.

1423
01:34:56,600 --> 01:35:00,780
So, you do have quite a bit of latitude of
what you do.

1424
01:35:00,780 --> 01:35:03,250
Now, you're probably not going to get that
program here because it is pretty expensive

1425
01:35:03,250 --> 01:35:03,830
and pretty hard to do.

1426
01:35:03,830 --> 01:35:07,990
But when you go out in industry or government
there is a very high probability you will

1427
01:35:07,990 --> 01:35:09,930
be using some of these programs.

1428
01:35:09,930 --> 01:35:15,670
And Johnson Space Center uses this one and
the Marshall Space Flight Center uses this

1429
01:35:15,670 --> 01:35:16,130
one, CRH.

1430
01:35:16,130 --> 01:35:23,130
There are two programs that are used by NASA,
I know, and they are becoming more and more

1431
01:35:26,880 --> 01:35:28,400
important as we go ahead.

1432
01:35:28,400 --> 01:35:35,400
Aaron, one thing you haven't discussed in
all this discussion of cost is margin.

1433
01:35:36,770 --> 01:35:43,770
And, as a manager, you have to deal with that,
so maybe you can make a few comments on that.

1434
01:35:43,770 --> 01:35:44,170
Sure.

1435
01:35:44,170 --> 01:35:45,719
Well, let me start off this way.

1436
01:35:45,719 --> 01:35:52,719
What Professor Hoffman is alluding to is that
when you create a budget you want to have

1437
01:35:53,830 --> 01:35:59,500
some type of reserve or margin over and above
your cost estimate.

1438
01:35:59,500 --> 01:36:01,650
And that takes care of several things.

1439
01:36:01,650 --> 01:36:08,650
It takes care of changes, it takes care of
I forgots and it takes care of inefficiencies.

1440
01:36:14,290 --> 01:36:21,290
Studies have found that in a high technology
program for the first time you are about 30%

1441
01:36:24,719 --> 01:36:27,410
inefficient, so you are only about 70% efficient.

1442
01:36:27,410 --> 01:36:30,800
Now, this was data that was taken some time
ago.

1443
01:36:30,800 --> 01:36:37,800
You are 70% efficient, so you need to allow
some type of margin in that for your inefficiencies.

1444
01:36:38,660 --> 01:36:45,550
If you do that and you talk about changes
and you talk about I forgots, your reserve

1445
01:36:45,550 --> 01:36:46,130
becomes very high.

1446
01:36:46,130 --> 01:36:52,120
And normally your boss, the guy that is over
you is not going to let you have that much

1447
01:36:52,120 --> 01:36:57,620
money.

1448
01:36:57,620 --> 01:37:03,469
One of the investigations we had on me during
the Shuttle Program, because my cost was growing,

1449
01:37:03,469 --> 01:37:07,969
a very famous man came in to be head of the
program named General Abrahamson.

1450
01:37:07,969 --> 01:37:11,150
Many of you may have heard of General Abrahamson.

1451
01:37:11,150 --> 01:37:16,510
While he did the investigation, he said, Aaron,
you need a large reserve.

1452
01:37:16,510 --> 01:37:19,320
He didn't know he was going to become manager
of the program.

1453
01:37:19,320 --> 01:37:23,889
When he became manager of the program, I had
this large reserve in there, he was reviewing

1454
01:37:23,889 --> 01:37:28,690
and said you cannot have that much reserve
so he wacked it down.

1455
01:37:28,690 --> 01:37:34,330
But you do need a reserve because you're not
going to make it without a reserve.

1456
01:37:34,330 --> 01:37:41,330
And, I don't know if you noticed or not but
in the papers I gave you on the cost of the

1457
01:37:43,320 --> 01:37:47,360
Shuttle system, it had about a 30% reserve
in there.

1458
01:37:47,360 --> 01:37:53,910
When I did the study for the 90-day study,
I had a 50% reserve in there because I felt

1459
01:37:53,910 --> 01:38:00,770
that was a pretty far out program and you
needed a reserve.

1460
01:38:00,770 --> 01:38:03,790
You need to fight for your reserve, but I
guaranty you're not going to get everything

1461
01:38:03,790 --> 01:38:05,070
you think you need.

1462
01:38:05,070 --> 01:38:05,550
Yes, sir.

1463
01:38:05,550 --> 01:38:10,280
For something like aerospace, every time you
do a cost estimate, they talk about it, look

1464
01:38:10,280 --> 01:38:15,400
at it and say, well, let's increase that by
50% or double it.

1465
01:38:15,400 --> 01:38:19,040
And often, even once you've done that, you
still don't come under budget.

1466
01:38:19,040 --> 01:38:26,040
I mean look at the ISS or the Airbus 380 that
is going out.

1467
01:38:26,340 --> 01:38:26,590
Why?

1468
01:38:26,530 --> 01:38:32,670
Well, again, it is really because -- First
of all, it is primarily for the inefficiencies.

1469
01:38:32,670 --> 01:38:34,969
You just cannot do a high-tech program.

1470
01:38:34,969 --> 01:38:41,130
Look at what it cost to do the Big Dig, to
drive it home.

1471
01:38:41,130 --> 01:38:48,130
It is very difficult to do a good cost estimate,
but that is the problem that Congress sees

1472
01:38:50,760 --> 01:38:56,420
with government programs today, both the Department
of Defense and NASA, that they cannot do a

1473
01:38:56,420 --> 01:39:00,530
good cost estimate.

1474
01:39:00,530 --> 01:39:06,710
The current program says they are going to
do the CEVing and go to the Moon for $104

1475
01:39:06,710 --> 01:39:13,710
billion, which is 55% of what it cost to do
the Apollo Program in current year dollars.

1476
01:39:15,610 --> 01:39:17,469
And that is going to be interesting to see
if they can do it.

1477
01:39:17,469 --> 01:39:18,969
It is tough to do.

1478
01:39:18,969 --> 01:39:21,650
But really I think the biggest problem is
your inefficiencies.

1479
01:39:21,650 --> 01:39:28,650
Of course there are changes and I forgots.

1480
01:39:30,170 --> 01:39:34,690
Interesting enough, though, you may or may
not believe this, and you probably won't but

1481
01:39:34,690 --> 01:39:40,450
I will say it anyway, it turns out that the
studies done for the Shuttle Program, for

1482
01:39:40,450 --> 01:39:47,450
the Orbiter actually, in real year dollars,
met the cost if we would have gotten the inflation

1483
01:39:48,990 --> 01:39:50,370
rate.

1484
01:39:50,370 --> 01:39:55,520
We lost the inflation for two years, which
Dale Myers fought for vigorously with OMB.

1485
01:39:55,520 --> 01:40:01,660
If we would have gotten the inflation for
those two years, we would have met cost for

1486
01:40:01,660 --> 01:40:02,680
the development cost.

1487
01:40:02,680 --> 01:40:05,530
Not for the operational cost, but for the
development cost we would have come pretty

1488
01:40:05,530 --> 01:40:06,889
close to it.

1489
01:40:06,889 --> 01:40:08,930
We didn't really miss it that far.

1490
01:40:08,930 --> 01:40:13,260
In Apollo, I'm not sure, you would need to
ask Dr.

1491
01:40:13,260 --> 01:40:18,100
Siemens, but the story goes is that NASA came
in with an estimate.

1492
01:40:18,100 --> 01:40:22,320
And at that time the administrator, Webb,
looked at it and said that is fine and he

1493
01:40:22,320 --> 01:40:25,540
doubled it, and so Apollo did very well.

1494
01:40:25,540 --> 01:40:28,750
I think Bob Siemens has told that.

1495
01:40:28,750 --> 01:40:35,600
The story we heard was it was on their way
over to the White House that he said our final

1496
01:40:35,600 --> 01:40:40,309
estimate, we've put all the factors and everything
in, is $10 billion.

1497
01:40:40,309 --> 01:40:43,580
And Webb said, OK, we will tell the President
$20 billion.

1498
01:40:43,580 --> 01:40:46,520
I am sure Bob Siemens would know.

1499
01:40:46,520 --> 01:40:46,830
Yes, sir.

1500
01:40:46,830 --> 01:40:52,130
Aaron, I wanted to ask you about the location
of the reserves.

1501
01:40:52,130 --> 01:40:59,130
Not just dollar reserves, but my experience
as a payload [NOISE OBSCURES].

1502
01:41:00,440 --> 01:41:07,440
The model I have been faced with is that the
mission manager has a reserve for crew time

1503
01:41:11,920 --> 01:41:16,120
or for weight of something but, down on the
project level, you would really like to go

1504
01:41:16,120 --> 01:41:17,780
and have your own.

1505
01:41:17,780 --> 01:41:22,740
And so there was a tendency for everybody
but the lowest level, from the individual

1506
01:41:22,740 --> 01:41:27,250
experiment up to say, well, I am going to
put in a little bit of pad rather than count

1507
01:41:27,250 --> 01:41:32,150
on the fact that I can go up to the mission
manager and ask him for some reserve later.

1508
01:41:32,150 --> 01:41:33,610
Well, that is the danger with that.

1509
01:41:33,610 --> 01:41:34,660
You are absolutely right.

1510
01:41:34,660 --> 01:41:36,120
Everybody would like to put their own reserve
in.

1511
01:41:36,120 --> 01:41:42,710
Of course, being a project manager, I thought
the reserve ought to be at my level.

1512
01:41:42,710 --> 01:41:49,710
But actually, John Yardley was my boss in
Washington, and he thought he should handle

1513
01:41:50,800 --> 01:41:52,270
the reserve.

1514
01:41:52,270 --> 01:41:56,840
And the subsystems managers wanted their reserve,
right?

1515
01:41:56,840 --> 01:41:58,260
Wanted their reserve.

1516
01:41:58,260 --> 01:42:02,760
And John Yardley was another person I should
have mentioned.

1517
01:42:02,760 --> 01:42:07,630
John actually basically pulled the reserve
up and handled it in Washington.

1518
01:42:07,630 --> 01:42:13,710
There is a certain efficiency, I assume, in
having all the reserves centralized so that

1519
01:42:13,710 --> 01:42:17,800
there is not a lot of waste down to lower
levels, but you have to have the confidence

1520
01:42:17,800 --> 01:42:20,070
that you are going to be able to draw on it.

1521
01:42:20,070 --> 01:42:20,320
Right.

1522
01:42:20,280 --> 01:42:24,809
And with that John would partial out some.

1523
01:42:24,809 --> 01:42:31,340
What he would do is be sure that you weren't
double reserving him or triple reserving him.

1524
01:42:31,340 --> 01:42:33,010
And then he would allocate it to me.

1525
01:42:33,010 --> 01:42:34,880
I know he had a rule of thumb.

1526
01:42:34,880 --> 01:42:41,880
When JR Thompson went up, you heard JR talk,
he always asked for more than he needed, so

1527
01:42:45,740 --> 01:42:46,740
John would give him less.

1528
01:42:46,740 --> 01:42:49,530
When I went up, I always asked for less than
I needed, so he would give me more.

1529
01:42:49,530 --> 01:42:52,260
He had a good feel for his managers.

1530
01:42:52,260 --> 01:42:59,100
That is what he told JR one day.

1531
01:42:59,100 --> 01:43:01,020
Normally, you wind up handling it.

1532
01:43:01,020 --> 01:43:04,980
If you have a good program manager at the
top you normally wind up handling it there

1533
01:43:04,980 --> 01:43:08,120
and he parcels it out based on the fact there
is no double bookkeeping.

1534
01:43:08,120 --> 01:43:08,719
Yes.

1535
01:43:08,719 --> 01:43:15,719
My experience at Draper is typically there
are two different ways of estimating.

1536
01:43:15,990 --> 01:43:18,670
One was called top down and the other was
bottoms up.

1537
01:43:18,670 --> 01:43:25,670
And traditionally bottoms up would always
stick their reserve in some way and top down

1538
01:43:26,100 --> 01:43:30,260
would always say I want to know what your
reserves are because I want to bring it up

1539
01:43:30,260 --> 01:43:30,950
to the top.

1540
01:43:30,950 --> 01:43:33,559
And that was a constant argument.

1541
01:43:33,559 --> 01:43:40,260
Then when you got to submitting your proposal
often the customer didn't want to know about

1542
01:43:40,260 --> 01:43:46,639
this extra money and then you would have to
shove it back in and hide it, in a sense,

1543
01:43:46,639 --> 01:43:47,510
by padding things.

1544
01:43:47,510 --> 01:43:54,290
Handling costing in reserves is, to some extent,
at least in my experience, a psychological

1545
01:43:54,290 --> 01:43:57,820
thing dealing with customers and your management.

1546
01:43:57,820 --> 01:44:02,920
And it has always been one where you shuffle
that reserve around.

1547
01:44:02,920 --> 01:44:09,920
But this gentleman asked the question about
why does it appear that NASA never can stay

1548
01:44:12,030 --> 01:44:12,740
on their budget?

1549
01:44:12,740 --> 01:44:15,469
That is really a tough question to answer.

1550
01:44:15,469 --> 01:44:16,700
I don't know.

1551
01:44:16,700 --> 01:44:23,700
I mean I have a personal feeling on it, but
I think the basic problem you have is that

1552
01:44:26,320 --> 01:44:29,590
it is a high technology program.

1553
01:44:29,590 --> 01:44:35,880
It is one of a kind and it is a lot of inefficiencies.

1554
01:44:35,880 --> 01:44:41,080
When you come to work every day, you don't
know exactly what you are going to do.

1555
01:44:41,080 --> 01:44:42,969
You have a technology development that costs.

1556
01:44:42,969 --> 01:44:43,930
You have I forgots.

1557
01:44:43,930 --> 01:44:46,240
You have changing requirements.

1558
01:44:46,240 --> 01:44:51,010
Changing requirements are something that systems
engineering ought to be able to control.

1559
01:44:51,010 --> 01:44:55,520
I forgots are something systems engineering
ought to be able to control.

1560
01:44:55,520 --> 01:44:58,900
Inefficiencies, to a certain extent, systems
engineering ought to be able to control.

1561
01:44:58,900 --> 01:45:04,700
That is one of the purposes, in all honesty,
that are concentrating on system engineering

1562
01:45:04,700 --> 01:45:11,360
at such places at MIT and other schools to
allow the future generation of designers and

1563
01:45:11,360 --> 01:45:17,400
developers in working that they can have a
budget and then stay on the budget.

1564
01:45:17,400 --> 01:45:23,639
Maybe in your generation you will be smarter
and able to control it, but it is not going

1565
01:45:23,639 --> 01:45:24,610
to be an easy job.

1566
01:45:24,610 --> 01:45:27,150
That is one thing I would say.

1567
01:45:27,150 --> 01:45:30,280
Let me just make one other comment.

1568
01:45:30,280 --> 01:45:32,389
We cannot estimate too much.

1569
01:45:32,389 --> 01:45:39,389
That you've got to always keep in mind this
triangle of cost, performance, schedule.

1570
01:45:39,639 --> 01:45:46,639
If you are working on a project where the
schedule is fixed like Apollo then that better

1571
01:45:46,880 --> 01:45:51,580
ring the bells to say that I need more reserves.

1572
01:45:51,580 --> 01:45:56,120
And maybe that is what was going on in Jim
Webb's mind when he said if we really have

1573
01:45:56,120 --> 01:46:02,880
to get this done by the end of the decade,
I better make sure that I am not going to

1574
01:46:02,880 --> 01:46:07,300
be constrained by cost and so I will double
it.

1575
01:46:07,300 --> 01:46:14,190
The other thing that you have to take into
account with the performance is how much new

1576
01:46:14,190 --> 01:46:16,500
technology is this program going to require?

1577
01:46:16,500 --> 01:46:21,510
Because the new technology, that is the hardest
thing to estimate the cost.

1578
01:46:21,510 --> 01:46:22,290
That's right.

1579
01:46:22,290 --> 01:46:28,850
And so, if you have a new technology program
then your margins better be big enough to

1580
01:46:28,850 --> 01:46:29,740
accommodate that.

1581
01:46:29,740 --> 01:46:36,740
That is why, for the CEV, they are really
going out of their way to saying, to the maximum

1582
01:46:37,680 --> 01:46:42,889
extent possible, we don't want to develop
new technology for the CEV.

1583
01:46:42,889 --> 01:46:48,770
We want to use Shuttle parts wherever we can,
Apollo heritage.

1584
01:46:48,770 --> 01:46:54,520
We want to build it out of things that we
know and we understand so that we hopefully

1585
01:46:54,520 --> 01:46:59,130
can keep the cost as low as possible.

1586
01:46:59,130 --> 01:47:00,610
And we will see how it all works out.

1587
01:47:00,610 --> 01:47:00,860
Yes, sir.

1588
01:47:00,790 --> 01:47:02,510
It seems to me that you often times have the
pressure to give a cost estimate on the low

1589
01:47:02,510 --> 01:47:04,150
end of what you believe it to actual be just
so the project might be more appealing [NOISE

1590
01:47:04,150 --> 01:47:04,400
OBSCURES].

1591
01:47:04,240 --> 01:47:11,240
That is certainly another part of the

1592
01:47:15,860 --> 01:47:17,010
whole political thing.

1593
01:47:17,010 --> 01:47:22,880
And not just from the political point of view
of presenting your estimate when NASA presents

1594
01:47:22,880 --> 01:47:29,380
the estimate to Congress, but then you have
the contractors who want to get the contract

1595
01:47:29,380 --> 01:47:29,800
from NASA.

1596
01:47:29,800 --> 01:47:31,820
And you have certainly had experience with
this.

1597
01:47:31,820 --> 01:47:36,930
How do you deal with the contractor who low-balls
and then assumes that they will be able to

1598
01:47:36,930 --> 01:47:38,650
make it up later on?

1599
01:47:38,650 --> 01:47:43,530
Low-balling really should not be tolerated.

1600
01:47:43,530 --> 01:47:47,780
And they try to do that, not tolerating it,
but you are absolutely right.

1601
01:47:47,780 --> 01:47:52,550
When I did this 90-day study, I really felt
they wanted to know what it was going to cost,

1602
01:47:52,550 --> 01:47:56,990
so I had a 55% reserve in there because I
really felt that there were so many unknowns

1603
01:47:56,990 --> 01:47:57,469
with it.

1604
01:47:57,469 --> 01:47:58,860
And, of course, that didn't go over very well.

1605
01:47:58,860 --> 01:48:02,510
They said my study actually killed the projects.

1606
01:48:02,510 --> 01:48:05,410
But I told them what it was going to cost.

1607
01:48:05,410 --> 01:48:12,410
And then this is a subject for another lecture
entirely, which I am not sure that we are

1608
01:48:12,860 --> 01:48:18,630
going to do in this course, but there are
many different ways to let out contracts,

1609
01:48:18,630 --> 01:48:23,480
fixed price contracts, cost plus contracts,
award contracts.

1610
01:48:23,480 --> 01:48:30,480
And all of those can have an impact on the
ultimate cost of your project.

1611
01:48:30,900 --> 01:48:37,900
And they all have their utility depending
on how well the technology is known and what

1612
01:48:40,300 --> 01:48:42,130
is the time pressure.

1613
01:48:42,130 --> 01:48:45,270
Again, cost, performance, schedule.

1614
01:48:45,270 --> 01:48:49,910
And you asked about Airbus.

1615
01:48:49,910 --> 01:48:54,170
I don't know the specifics of Airbus contracting,
but I am familiar with ESA.

1616
01:48:54,170 --> 01:48:59,219
And, because of some of the peculiarities
of the European system, they tend to use fixed

1617
01:48:59,219 --> 01:49:04,530
price contracts a lot more than we do here.

1618
01:49:04,530 --> 01:49:10,389
And sometimes it works but sometimes you get
really burned on it because contractors are

1619
01:49:10,389 --> 01:49:16,389
not going to go into bankruptcy because they
cannot deliver on their fixed price contracts.

1620
01:49:16,389 --> 01:49:19,719
You try to go fixed price on the CEV and it
would cost you a fortune.

1621
01:49:19,719 --> 01:49:23,400
Those are very good questions.

1622
01:49:23,400 --> 01:49:27,200
Unfortunately, we don't give you very firm
answers on cost.

1623
01:49:27,200 --> 01:49:30,990
But at least you are thinking and I appreciate
your thought process.

1624
01:49:30,990 --> 01:49:35,760
It is very encouraging to hear you ask those
types of questions.

1625
01:49:35,760 --> 01:49:36,450
Have a good weekend.

1626
01:49:36,450 --> 01:49:37,700
We will see you on Tuesday.