Video 21: Image Enhancement

• Download English-US transcript (PDF)

The question of how far we can go when we enhance, or adjust, or touch up an image in science is a critical one.

You are all familiar, I'm sure, with the stunningly beautiful Hubble images published all over the world.

But most of the world is not familiar with the fact that the colors we're seeing were artificially created, decided and implemented by humans.

You can read a conversation I had with some of these researchers in American Scientist, in the resource section, look for this thumbnail.

I think you'll find the article interesting, about the decisions the researchers made about coloring the detail of nebula.

So in essence, the coloring, or enhancement, was mostly done for the purpose of communicating structure, and I would say also for helping bring attention to these amazing images.

Now remember, these pictures of the universe, are representations, that is re- presentations, they are photographs of that universe, they are not the universe.

All the photographs in this course, and the ones you are making and will make, re- present various structures and phenomena, in order to communicate research.

You, the image maker, has to make decisions in order to make these re- presentations, as I do.

Should I make an image of this or that, shall I compose it this way or that?

When shall I set up my camera, in the winter or in the summer?

At five minutes of the phenomena, or three days later?

What tool shall I use, should I use a scanner or a stereo microscope?

What lighting set up should I use?

What background is best?

So, in a way, the whole process of photography requires us to make decisions about making adjustments, for the purpose of communication.

The question we must ask ourselves is, do any of these choices misrepresent reality.

So that process of selecting colors in the Eagle Nebula is similar to my process when I colored this image, which was originally grayscale.

I made the image with electrons on a scanning electron microscope, an SEM.

We needed electrons to see the nanowires, they were too small to use photons, as we do in photography.

Once again, the purpose here is to show more clearly the structure.

The difference between my adjustment and the astronomers' adjustment is that the color choices of the researchers in the article were informed by chemical characteristics of the various areas of the image.

My choices in this particular case was purely aesthetic.

But first and foremost with any work in science, whenever we color any SEM or other image, we must always indicate what we have done, period.

Somewhere near the image, not on another page, we have to dig deeply to see what's going on here, somewhere near the image we must say, "this image was color enhanced." The subject of enhancement is not discussed enough in your training.

Many of you just assume that certain adjustments can be made in software on your pictures.

That is not the case.

I can't possibly go through every example of adjustments at this point, but I'd like to encourage you to start thinking about it, from the very beginning when capturing your work, when you start a particular investigation, which you will be submitting for a publication, when you start documenting the work as evidence of your research.

Just as Important, I'm encouraging you to have this kind of conversation with your colleagues, to create a mindset in your lab of do's and dont's.

We just don't talk about it enough.

So let's even start with these videos you've been watching for our course.

I mentioned to you in the welcome video that I digitally cleaned some of the images that you're looking at.

The reason I gave you for getting rid of the specks, was so that you would not be distracted by imperfections in the various samples that I photographed.

Well, that was a decision I made for this particular purpose for this course, to teach you about process.

And most important is the fact that I informed you of my adjustment.

However, if you are submitting an image for a published figure to a journal, well that's a whole different story.

I was privileged to have a long conversation with a few editors and the art director at Nature.

We discussed a number of examples you'll be seeing here.

We talked about what is permitted and what is not.

First, we concluded that the rules for cover images are different from images in figures.

Nature regards cover submissions more as illustration, or "art," and in that case, one is permitted to clean some specks and distracting material, and to add color for example, if the image is initially in black and white.

Here's a picture I made, starting with the unadjusted image of the yeast colony, and here I digitally deleted the Petri dish, because I wanted the viewer to see the stunning morphology of the colony.

The data in the image is the morphology of the colony.

I'm not adjusting that data, I am simply removing the part of the image that in my opinion is unnecessary or distracting, although not everybody agrees with me.

Science did permit that adjustment for the cover, and once again I described the adjustment in the cover captions.

Here, I made this image of a device on film, so because the lighting of the lab created a green cast, it's how the film read the scene.

My adjustment of the color was trying to get the image to what my eyes saw, and in my opinion is acceptable.

And this one taken under a stereo microscope, we see an artifact of the process, a reflection of the lens on the wafer.

So here I remove the reflection, again always indicating what I've done.

In this one, which you've seen before, I first use the stereo microscope, but because with this tool I didn't have control over the direction of the light, we're seeing all sorts of distractions.

In the second image, which I made with a camera and a lens, the image is cleaner.

I did not digitally clean the specks , it just looked better with this equipment and this lighting.

Or let's take this device one step further, where I made this detail with a microscope.

Using a technique in microscopy called Nomarski Differential Contrast, which emphasizes surface structure, based on the index of refraction.

That's why you're seeing all these colors.

So all of these decisions address which tool shall I use, basically adjusting my point of view, which translates into adjusting the final image.

This one is a more complicated question, which the editors and I discussed at length.

In effect, we started having a conversation about what is reality, which is a subject I promise we're not going to pursue here, but in essence this is a question that should be somewhere in the back of your thinking.

After all, you are making re- presentations of reality, aren't you?

So here's the image, which I hope gets you to think a little bit.

These are rods, measuring about two to three centimeters long, all of which have absorbed fluorescing material.

This is the image I captured on film under UV light, however my eye saw this image with orange rods.

It turns out the film did not capture the orange wavelength, so I digitally adjusted those rods that were supposed to appear orange.

That adjustment was not acceptable to the editors it turns out.

In my opinion, the adjusted image was closer to reality than the unadjusted image.

What do you think?

And take this one, here is the original image of these amazing Proteus colonies growing in a Petri dish.

As it happens, the agar is cracking in a few places.

I made the image for the book On The Surface of Things, the point was to show how the colonies grow.

And I didn't want to detract the reader from the point of the image, so I digitally deleted the cracks.

Now the Nature editors all agree this would never be accepted for a figure in their journal, which makes sense.

But what about a book for the public, what do you think?

Here's an image from another of my articles in American Scientist.

Look for the thumbnail again under the resources.

I very mistakenly deleted a small speck in the middle of the glowing circle here.

I did so after asking the researcher if I could do so, and he very politely said it was fine, but years later when we discuss the article, I realize that that deletion was very improper.

It turns out that that small speck had an important effect on the final outcome of the image.

I guess at the time he was trying to be easy, and the article was not part of a journal submission.

But now, and we both agree, the mistake was clearly mine.

I had no right to adjust that image, even after asking for permission.

Here's that adjustment, by the way, you saw before, reverting a grayscale image, and the editors did accept that at Nature.

Now how about this one?

It's true that these are microscopic images, and beyond the realm of the course, but I thought it would be important for you to think about it.

The researchers supplied me with two of their original images.

The colored areas gave highly detailed morphological information about the cell, as specific quantum dots attached to specific structures.

When the dots are excited with UV light they fluoresce, and so we see where they attach.

In standard views of fluorescently labeled material, the colored areas are set against a black background.

I thought it would be interesting and maybe more communicative to see the information differently, as long as I maintain the integrity of the science.

So I inverted the two images in Photoshop to give a white background.

But because it was important to keep the original colors of the structures themselves, I changed the colors back to the red and green.

I then layered both of the images over a third image taken with standard microscopy, again supplied by the laboratory.

The three layered results shows the same information as the original separate images, with the addition of a sense of the whole.

I'm showing you where the structures are, I'm not quantifying the amount of quantum dots present.

Another form of adjustment, changing histograms, can also be part of our conversation.

Here's an old gel run created a while ago, when the runs were still captured on film.

The point I want to make here, it's still relevant.

So here I am zooming into a portion, after I change the histogram for the whole run.

Now that's important, I didn't take a portion.

In this one, I adjusted the contrast even more, and on this one even more, to better see the bands.

Now what do you think about this, is this permitted?

The editors at Nature were initially comfortable with the enhancements, since the adjustment was universally made to the entire image.

And later in the conversation, they emphasize the requirement to indicate how that image was enhanced.

OK, so what about this one?

The tank you're looking at was used to study sediment rates of sand.

And it was too large to photograph in one image, and so I made the couple of images and superimposed them for a journal article.

You'll notice I stitch the two images together, showing the fact that there are two images, you're seeing the overlays.

I did the same for this image, this time microscopic, again since it wasn't possible to show the whole device in one image.

And here again, I decided to include the boundaries to each image, letting the viewer know the full image is a collage.

And for this one, the researcher made single SEMs and used Microsoft software to stitch the pieces together for the article.

Here we cleaned it a bit, what do you think?

Once again, I want to say there are so many possibilities in this conversation, but it's critical that you keep all of these questions in mind when you talk to each other, and certainly when you start thinking about communicating your work.

 

Free Downloads

Video

• iTunes U (MP4 - 34MB)
• Internet Archive (MP4 - 34MB)

Caption

• English-US (SRT)