1
00:00:04,890 --> 00:00:08,960
The question of how far
we can go when we enhance,

2
00:00:08,960 --> 00:00:14,860
or adjust, or touch up an image
in science is a critical one.

3
00:00:14,860 --> 00:00:16,760
You are all familiar,
I'm sure, with

4
00:00:16,760 --> 00:00:19,270
the stunningly
beautiful Hubble images

5
00:00:19,270 --> 00:00:21,340
published all over the world.

6
00:00:21,340 --> 00:00:24,390
But most of the world is
not familiar with the fact

7
00:00:24,390 --> 00:00:26,650
that the colors
we're seeing were

8
00:00:26,650 --> 00:00:32,619
artificially created, decided
and implemented by humans.

9
00:00:32,619 --> 00:00:34,430
You can read a
conversation I had

10
00:00:34,430 --> 00:00:36,820
with some of these
researchers in American

11
00:00:36,820 --> 00:00:39,710
Scientist, in the
resource section,

12
00:00:39,710 --> 00:00:41,270
look for this thumbnail.

13
00:00:41,270 --> 00:00:43,760
I think you'll find the
article interesting,

14
00:00:43,760 --> 00:00:47,340
about the decisions the
researchers made about coloring

15
00:00:47,340 --> 00:00:49,500
the detail of nebula.

16
00:00:49,500 --> 00:00:53,060
So in essence, the
coloring, or enhancement,

17
00:00:53,060 --> 00:00:57,380
was mostly done for the purpose
of communicating structure,

18
00:00:57,380 --> 00:01:00,610
and I would say also for
helping bring attention

19
00:01:00,610 --> 00:01:03,400
to these amazing images.

20
00:01:03,400 --> 00:01:06,190
Now remember, these
pictures of the universe,

21
00:01:06,190 --> 00:01:11,200
are representations, that
is re- presentations,

22
00:01:11,200 --> 00:01:14,330
they are photographs
of that universe,

23
00:01:14,330 --> 00:01:17,230
they are not the universe.

24
00:01:17,230 --> 00:01:19,350
All the photographs
in this course,

25
00:01:19,350 --> 00:01:22,130
and the ones you are
making and will make,

26
00:01:22,130 --> 00:01:26,920
re- present various structures
and phenomena, in order

27
00:01:26,920 --> 00:01:28,980
to communicate research.

28
00:01:28,980 --> 00:01:32,570
You, the image maker,
has to make decisions

29
00:01:32,570 --> 00:01:37,550
in order to make these re-
presentations, as I do.

30
00:01:37,550 --> 00:01:41,120
Should I make an
image of this or that,

31
00:01:41,120 --> 00:01:45,400
shall I compose it
this way or that?

32
00:01:45,400 --> 00:01:51,900
When shall I set up my camera,
in the winter or in the summer?

33
00:01:51,900 --> 00:01:58,130
At five minutes of the
phenomena, or three days later?

34
00:01:58,130 --> 00:02:01,830
What tool shall I use,
should I use a scanner

35
00:02:01,830 --> 00:02:04,630
or a stereo microscope?

36
00:02:04,630 --> 00:02:07,900
What lighting set
up should I use?

37
00:02:07,900 --> 00:02:10,900
What background is best?

38
00:02:10,900 --> 00:02:15,270
So, in a way, the whole
process of photography

39
00:02:15,270 --> 00:02:20,130
requires us to make decisions
about making adjustments,

40
00:02:20,130 --> 00:02:22,680
for the purpose
of communication.

41
00:02:22,680 --> 00:02:25,410
The question we
must ask ourselves

42
00:02:25,410 --> 00:02:30,880
is, do any of these choices
misrepresent reality.

43
00:02:30,880 --> 00:02:35,170
So that process of selecting
colors in the Eagle Nebula

44
00:02:35,170 --> 00:02:39,310
is similar to my process when
I colored this image, which

45
00:02:39,310 --> 00:02:41,750
was originally grayscale.

46
00:02:41,750 --> 00:02:45,490
I made the image with electrons
on a scanning electron

47
00:02:45,490 --> 00:02:47,870
microscope, an SEM.

48
00:02:47,870 --> 00:02:51,100
We needed electrons
to see the nanowires,

49
00:02:51,100 --> 00:02:53,550
they were too small
to use photons,

50
00:02:53,550 --> 00:02:55,490
as we do in photography.

51
00:02:55,490 --> 00:02:59,310
Once again, the purpose
here is to show more clearly

52
00:02:59,310 --> 00:03:01,320
the structure.

53
00:03:01,320 --> 00:03:03,400
The difference
between my adjustment

54
00:03:03,400 --> 00:03:05,180
and the astronomers'
adjustment is

55
00:03:05,180 --> 00:03:08,410
that the color choices of the
researchers in the article

56
00:03:08,410 --> 00:03:11,980
were informed by
chemical characteristics

57
00:03:11,980 --> 00:03:15,100
of the various
areas of the image.

58
00:03:15,100 --> 00:03:19,550
My choices in this particular
case was purely aesthetic.

59
00:03:19,550 --> 00:03:23,010
But first and foremost
with any work in science,

60
00:03:23,010 --> 00:03:26,920
whenever we color any
SEM or other image,

61
00:03:26,920 --> 00:03:32,400
we must always indicate
what we have done, period.

62
00:03:32,400 --> 00:03:36,120
Somewhere near the image,
not on another page,

63
00:03:36,120 --> 00:03:40,180
we have to dig deeply to
see what's going on here,

64
00:03:40,180 --> 00:03:43,680
somewhere near the
image we must say,

65
00:03:43,680 --> 00:03:47,079
"this image was color enhanced."

66
00:03:47,079 --> 00:03:50,190
The subject of enhancement
is not discussed enough

67
00:03:50,190 --> 00:03:51,290
in your training.

68
00:03:51,290 --> 00:03:55,160
Many of you just assume that
certain adjustments can be made

69
00:03:55,160 --> 00:03:57,740
in software on your pictures.

70
00:03:57,740 --> 00:03:59,900
That is not the case.

71
00:03:59,900 --> 00:04:03,570
I can't possibly go through
every example of adjustments

72
00:04:03,570 --> 00:04:06,130
at this point, but
I'd like to encourage

73
00:04:06,130 --> 00:04:09,720
you to start thinking about it,
from the very beginning when

74
00:04:09,720 --> 00:04:11,965
capturing your
work, when you start

75
00:04:11,965 --> 00:04:15,220
a particular investigation,
which you will be submitting

76
00:04:15,220 --> 00:04:17,750
for a publication,
when you start

77
00:04:17,750 --> 00:04:22,050
documenting the work as
evidence of your research.

78
00:04:22,050 --> 00:04:25,100
Just as Important, I'm
encouraging you to have this

79
00:04:25,100 --> 00:04:28,230
kind of conversation
with your colleagues,

80
00:04:28,230 --> 00:04:32,740
to create a mindset in your
lab of do's and dont's.

81
00:04:32,740 --> 00:04:35,400
We just don't talk
about it enough.

82
00:04:35,400 --> 00:04:37,820
So let's even start
with these videos you've

83
00:04:37,820 --> 00:04:39,700
been watching for our course.

84
00:04:39,700 --> 00:04:42,170
I mentioned to you
in the welcome video

85
00:04:42,170 --> 00:04:44,710
that I digitally cleaned
some of the images

86
00:04:44,710 --> 00:04:46,180
that you're looking at.

87
00:04:46,180 --> 00:04:50,110
The reason I gave you for
getting rid of the specks,

88
00:04:50,110 --> 00:04:52,370
was so that you would
not be distracted

89
00:04:52,370 --> 00:04:56,230
by imperfections in the various
samples that I photographed.

90
00:04:56,230 --> 00:05:00,130
Well, that was a decision I
made for this particular purpose

91
00:05:00,130 --> 00:05:04,310
for this course, to
teach you about process.

92
00:05:04,310 --> 00:05:07,390
And most important is the
fact that I informed you

93
00:05:07,390 --> 00:05:09,040
of my adjustment.

94
00:05:09,040 --> 00:05:12,320
However, if you are submitting
an image for a published

95
00:05:12,320 --> 00:05:16,290
figure to a journal, well
that's a whole different story.

96
00:05:16,290 --> 00:05:20,000
I was privileged to have a long
conversation with a few editors

97
00:05:20,000 --> 00:05:22,230
and the art director at Nature.

98
00:05:22,230 --> 00:05:25,910
We discussed a number of
examples you'll be seeing here.

99
00:05:25,910 --> 00:05:29,500
We talked about what is
permitted and what is not.

100
00:05:29,500 --> 00:05:33,140
First, we concluded that
the rules for cover images

101
00:05:33,140 --> 00:05:36,409
are different from
images in figures.

102
00:05:36,409 --> 00:05:41,409
Nature regards cover submissions
more as illustration, or "art,"

103
00:05:41,409 --> 00:05:43,090
and in that case,
one is permitted

104
00:05:43,090 --> 00:05:47,380
to clean some specks and
distracting material,

105
00:05:47,380 --> 00:05:50,820
and to add color for example,
if the image is initially

106
00:05:50,820 --> 00:05:52,650
in black and white.

107
00:05:52,650 --> 00:05:54,630
Here's a picture
I made, starting

108
00:05:54,630 --> 00:05:58,300
with the unadjusted image
of the yeast colony,

109
00:05:58,300 --> 00:06:02,470
and here I digitally
deleted the Petri dish,

110
00:06:02,470 --> 00:06:06,760
because I wanted the viewer
to see the stunning morphology

111
00:06:06,760 --> 00:06:08,660
of the colony.

112
00:06:08,660 --> 00:06:12,800
The data in the image is the
morphology of the colony.

113
00:06:12,800 --> 00:06:16,380
I'm not adjusting
that data, I am simply

114
00:06:16,380 --> 00:06:19,570
removing the part of the
image that in my opinion

115
00:06:19,570 --> 00:06:23,780
is unnecessary or distracting,
although not everybody agrees

116
00:06:23,780 --> 00:06:24,640
with me.

117
00:06:24,640 --> 00:06:27,760
Science did permit that
adjustment for the cover,

118
00:06:27,760 --> 00:06:30,940
and once again I
described the adjustment

119
00:06:30,940 --> 00:06:33,130
in the cover captions.

120
00:06:33,130 --> 00:06:37,140
Here, I made this image
of a device on film,

121
00:06:37,140 --> 00:06:40,840
so because the lighting of
the lab created a green cast,

122
00:06:40,840 --> 00:06:44,080
it's how the film
read the scene.

123
00:06:44,080 --> 00:06:46,350
My adjustment of
the color was trying

124
00:06:46,350 --> 00:06:50,120
to get the image to
what my eyes saw,

125
00:06:50,120 --> 00:06:52,930
and in my opinion is acceptable.

126
00:06:52,930 --> 00:06:55,980
And this one taken under
a stereo microscope,

127
00:06:55,980 --> 00:06:59,159
we see an artifact of
the process, a reflection

128
00:06:59,159 --> 00:07:01,630
of the lens on the wafer.

129
00:07:01,630 --> 00:07:05,670
So here I remove the reflection,
again always indicating

130
00:07:05,670 --> 00:07:07,500
what I've done.

131
00:07:07,500 --> 00:07:10,920
In this one, which you've
seen before, I first

132
00:07:10,920 --> 00:07:14,490
use the stereo microscope,
but because with this tool

133
00:07:14,490 --> 00:07:18,250
I didn't have control over
the direction of the light,

134
00:07:18,250 --> 00:07:21,330
we're seeing all
sorts of distractions.

135
00:07:21,330 --> 00:07:25,050
In the second image, which I
made with a camera and a lens,

136
00:07:25,050 --> 00:07:26,780
the image is cleaner.

137
00:07:26,780 --> 00:07:29,220
I did not digitally
clean the specks ,

138
00:07:29,220 --> 00:07:32,970
it just looked better with this
equipment and this lighting.

139
00:07:32,970 --> 00:07:35,480
Or let's take this
device one step further,

140
00:07:35,480 --> 00:07:39,120
where I made this detail
with a microscope.

141
00:07:39,120 --> 00:07:41,900
Using a technique
in microscopy called

142
00:07:41,900 --> 00:07:44,540
Nomarski Differential
Contrast, which

143
00:07:44,540 --> 00:07:46,770
emphasizes surface
structure, based

144
00:07:46,770 --> 00:07:49,010
on the index of refraction.

145
00:07:49,010 --> 00:07:51,790
That's why you're
seeing all these colors.

146
00:07:51,790 --> 00:07:55,250
So all of these decisions
address which tool shall

147
00:07:55,250 --> 00:08:01,940
I use, basically adjusting my
point of view, which translates

148
00:08:01,940 --> 00:08:06,200
into adjusting the final image.

149
00:08:06,200 --> 00:08:08,560
This one is a more
complicated question,

150
00:08:08,560 --> 00:08:11,830
which the editors and
I discussed at length.

151
00:08:11,830 --> 00:08:14,710
In effect, we started
having a conversation

152
00:08:14,710 --> 00:08:18,560
about what is reality, which is
a subject I promise we're not

153
00:08:18,560 --> 00:08:21,520
going to pursue here,
but in essence this

154
00:08:21,520 --> 00:08:23,890
is a question that
should be somewhere

155
00:08:23,890 --> 00:08:25,830
in the back of your thinking.

156
00:08:25,830 --> 00:08:31,100
After all, you are making
re- presentations of reality,

157
00:08:31,100 --> 00:08:32,260
aren't you?

158
00:08:32,260 --> 00:08:34,820
So here's the
image, which I hope

159
00:08:34,820 --> 00:08:36,530
gets you to think a little bit.

160
00:08:36,530 --> 00:08:40,510
These are rods, measuring about
two to three centimeters long,

161
00:08:40,510 --> 00:08:43,950
all of which have absorbed
fluorescing material.

162
00:08:43,950 --> 00:08:48,070
This is the image I captured
on film under UV light,

163
00:08:48,070 --> 00:08:53,680
however my eye saw this
image with orange rods.

164
00:08:53,680 --> 00:08:58,060
It turns out the film did not
capture the orange wavelength,

165
00:08:58,060 --> 00:09:01,620
so I digitally adjusted
those rods that

166
00:09:01,620 --> 00:09:04,250
were supposed to appear orange.

167
00:09:04,250 --> 00:09:07,660
That adjustment was not
acceptable to the editors

168
00:09:07,660 --> 00:09:08,890
it turns out.

169
00:09:08,890 --> 00:09:13,290
In my opinion, the adjusted
image was closer to reality

170
00:09:13,290 --> 00:09:15,610
than the unadjusted image.

171
00:09:15,610 --> 00:09:18,390
What do you think?

172
00:09:18,390 --> 00:09:21,330
And take this one, here
is the original image

173
00:09:21,330 --> 00:09:23,980
of these amazing
Proteus colonies

174
00:09:23,980 --> 00:09:25,910
growing in a Petri dish.

175
00:09:25,910 --> 00:09:30,350
As it happens, the agar is
cracking in a few places.

176
00:09:30,350 --> 00:09:33,990
I made the image for the book
On The Surface of Things,

177
00:09:33,990 --> 00:09:37,410
the point was to show
how the colonies grow.

178
00:09:37,410 --> 00:09:39,960
And I didn't want to
detract the reader

179
00:09:39,960 --> 00:09:44,790
from the point of the image, so
I digitally deleted the cracks.

180
00:09:44,790 --> 00:09:47,680
Now the Nature editors
all agree this would never

181
00:09:47,680 --> 00:09:50,640
be accepted for a figure
in their journal, which

182
00:09:50,640 --> 00:09:52,050
makes sense.

183
00:09:52,050 --> 00:09:57,250
But what about a book for the
public, what do you think?

184
00:09:57,250 --> 00:10:00,190
Here's an image from another
of my articles in American

185
00:10:00,190 --> 00:10:01,250
Scientist.

186
00:10:01,250 --> 00:10:04,800
Look for the thumbnail
again under the resources.

187
00:10:04,800 --> 00:10:09,450
I very mistakenly deleted
a small speck in the middle

188
00:10:09,450 --> 00:10:12,020
of the glowing circle here.

189
00:10:12,020 --> 00:10:15,720
I did so after asking the
researcher if I could do so,

190
00:10:15,720 --> 00:10:18,620
and he very politely
said it was fine,

191
00:10:18,620 --> 00:10:21,210
but years later when
we discuss the article,

192
00:10:21,210 --> 00:10:25,330
I realize that that
deletion was very improper.

193
00:10:25,330 --> 00:10:27,540
It turns out that
that small speck

194
00:10:27,540 --> 00:10:31,700
had an important effect on the
final outcome of the image.

195
00:10:31,700 --> 00:10:34,170
I guess at the time he
was trying to be easy,

196
00:10:34,170 --> 00:10:37,760
and the article was not part
of a journal submission.

197
00:10:37,760 --> 00:10:41,770
But now, and we both agree,
the mistake was clearly mine.

198
00:10:41,770 --> 00:10:44,050
I had no right to
adjust that image,

199
00:10:44,050 --> 00:10:47,120
even after asking
for permission.

200
00:10:47,120 --> 00:10:48,680
Here's that
adjustment, by the way,

201
00:10:48,680 --> 00:10:52,250
you saw before, reverting
a grayscale image,

202
00:10:52,250 --> 00:10:55,770
and the editors did
accept that at Nature.

203
00:10:55,770 --> 00:10:57,560
Now how about this one?

204
00:10:57,560 --> 00:10:59,890
It's true that these
are microscopic images,

205
00:10:59,890 --> 00:11:01,372
and beyond the
realm of the course,

206
00:11:01,372 --> 00:11:02,830
but I thought it
would be important

207
00:11:02,830 --> 00:11:04,770
for you to think about it.

208
00:11:04,770 --> 00:11:09,110
The researchers supplied me with
two of their original images.

209
00:11:09,110 --> 00:11:12,110
The colored areas
gave highly detailed

210
00:11:12,110 --> 00:11:15,140
morphological information
about the cell,

211
00:11:15,140 --> 00:11:18,120
as specific quantum
dots attached

212
00:11:18,120 --> 00:11:20,250
to specific structures.

213
00:11:20,250 --> 00:11:22,620
When the dots are
excited with UV light

214
00:11:22,620 --> 00:11:26,640
they fluoresce, and so
we see where they attach.

215
00:11:26,640 --> 00:11:30,590
In standard views of
fluorescently labeled material,

216
00:11:30,590 --> 00:11:34,430
the colored areas are set
against a black background.

217
00:11:34,430 --> 00:11:36,010
I thought it would
be interesting

218
00:11:36,010 --> 00:11:38,500
and maybe more
communicative to see

219
00:11:38,500 --> 00:11:42,160
the information differently,
as long as I maintain

220
00:11:42,160 --> 00:11:44,330
the integrity of the science.

221
00:11:44,330 --> 00:11:47,360
So I inverted the two
images in Photoshop

222
00:11:47,360 --> 00:11:49,350
to give a white background.

223
00:11:49,350 --> 00:11:52,380
But because it was important
to keep the original colors

224
00:11:52,380 --> 00:11:56,420
of the structures themselves,
I changed the colors back

225
00:11:56,420 --> 00:11:58,680
to the red and green.

226
00:11:58,680 --> 00:12:00,940
I then layered
both of the images

227
00:12:00,940 --> 00:12:05,800
over a third image taken with
standard microscopy, again

228
00:12:05,800 --> 00:12:07,950
supplied by the laboratory.

229
00:12:07,950 --> 00:12:11,560
The three layered results
shows the same information

230
00:12:11,560 --> 00:12:15,760
as the original separate
images, with the addition

231
00:12:15,760 --> 00:12:18,430
of a sense of the whole.

232
00:12:18,430 --> 00:12:21,410
I'm showing you where
the structures are,

233
00:12:21,410 --> 00:12:26,840
I'm not quantifying the amount
of quantum dots present.

234
00:12:26,840 --> 00:12:30,210
Another form of adjustment,
changing histograms,

235
00:12:30,210 --> 00:12:33,280
can also be part of
our conversation.

236
00:12:33,280 --> 00:12:37,420
Here's an old gel run
created a while ago,

237
00:12:37,420 --> 00:12:40,750
when the runs were
still captured on film.

238
00:12:40,750 --> 00:12:44,130
The point I want to make
here, it's still relevant.

239
00:12:44,130 --> 00:12:47,690
So here I am zooming
into a portion,

240
00:12:47,690 --> 00:12:52,150
after I change the
histogram for the whole run.

241
00:12:52,150 --> 00:12:56,380
Now that's important, I
didn't take a portion.

242
00:12:56,380 --> 00:13:00,280
In this one, I adjusted
the contrast even more,

243
00:13:00,280 --> 00:13:05,330
and on this one even more,
to better see the bands.

244
00:13:05,330 --> 00:13:09,080
Now what do you think about
this, is this permitted?

245
00:13:09,080 --> 00:13:11,360
The editors at
Nature were initially

246
00:13:11,360 --> 00:13:13,740
comfortable with
the enhancements,

247
00:13:13,740 --> 00:13:16,040
since the adjustment
was universally

248
00:13:16,040 --> 00:13:18,570
made to the entire image.

249
00:13:18,570 --> 00:13:20,370
And later in the
conversation, they

250
00:13:20,370 --> 00:13:23,340
emphasize the
requirement to indicate

251
00:13:23,340 --> 00:13:26,380
how that image was enhanced.

252
00:13:26,380 --> 00:13:28,040
OK, so what about this one?

253
00:13:28,040 --> 00:13:30,610
The tank you're
looking at was used

254
00:13:30,610 --> 00:13:33,420
to study sediment rates of sand.

255
00:13:33,420 --> 00:13:36,630
And it was too large to
photograph in one image,

256
00:13:36,630 --> 00:13:39,680
and so I made the
couple of images

257
00:13:39,680 --> 00:13:43,170
and superimposed them
for a journal article.

258
00:13:43,170 --> 00:13:46,590
You'll notice I stitch
the two images together,

259
00:13:46,590 --> 00:13:49,670
showing the fact that
there are two images,

260
00:13:49,670 --> 00:13:52,140
you're seeing the overlays.

261
00:13:52,140 --> 00:13:56,500
I did the same for this image,
this time microscopic, again

262
00:13:56,500 --> 00:14:00,770
since it wasn't possible to show
the whole device in one image.

263
00:14:00,770 --> 00:14:03,640
And here again, I decided
to include the boundaries

264
00:14:03,640 --> 00:14:06,360
to each image, letting
the viewer know

265
00:14:06,360 --> 00:14:09,570
the full image is a collage.

266
00:14:09,570 --> 00:14:14,100
And for this one, the
researcher made single SEMs

267
00:14:14,100 --> 00:14:18,860
and used Microsoft software
to stitch the pieces together

268
00:14:18,860 --> 00:14:19,840
for the article.

269
00:14:19,840 --> 00:14:25,250
Here we cleaned it a
bit, what do you think?

270
00:14:25,250 --> 00:14:28,280
Once again, I want
to say there are so

271
00:14:28,280 --> 00:14:31,410
many possibilities
in this conversation,

272
00:14:31,410 --> 00:14:35,900
but it's critical that you keep
all of these questions in mind

273
00:14:35,900 --> 00:14:39,000
when you talk to each
other, and certainly

274
00:14:39,000 --> 00:14:43,370
when you start thinking about
communicating your work.