Lesson 4—Image Processing

Purpose

This lesson reinforces your understanding of matrices in Matlab, and introduces you to some basic ideas behind digital images and image processing.

Background

This lesson is built around implementing a “green screen” (more technically known as “chroma key”) effect, in which a uniformly colored background in one image is replaced with another image to create a picture showing someone or something in a place they never really were. In the following, I will refer to the image whose background is being replaced as the “subject” image, and the replacement background as the “background” image.

Basic information about how Matlab programs represent, input, and output images is in section 13.2 of Attaway’s text. This lesson will also require you to use a number of other new Matlab features, including functions (section 3.7) and a number of matrix construction and access operations (chapter 2).

You will almost certainly need to do some amount of ancillary image processing outside of Matlab in this lesson. At the very least, you will have to scale your subject and background images to be the same size, and you might find it helpful to have an independent source of image statistics against which to test the statistics function I ask you to write. I can do some of this for you, but there are also many free image processing programs that you can download to your own computer. For example, ImageJ, available at http://imagej.nih.gov/ij/, is well respected and very capable.

Activity

Write two Matlab functions, as follows…

Function 1—Image Statistics

Write a Matlab function that takes an image (i.e., an n-by-m-by-3 array of color intensities) as its parameter, and that prints the following statistics for each of the image’s color channels:

• The minimum intensity of that color in any pixel in the image
• The maximum intensity of that color in any pixel in the image
• The average intensity of that color over all pixels in the image
• The standard deviation of that color’s intensity over all pixels in the image.

This function will be a useful tool for writing and testing the next function, since it will help you decide what range of colors in a subject image to replace with background image pixels. I suggest that you take a photograph consisting solely of the blue-green screen, pass it to this image statistics function, and use the resulting statistics to decide what colors to replace in future subject images. Exactly how you use image statistics to make this decision is up to you though.

Note that this function has no return value. It is one of the relatively rare cases in which a function is most useful for displaying output rather than returning a value for the caller to use however it wishes.

Function 2—Chroma Key

Write a Matlab function that takes two images, a subject and a background, and a description of the key color(s) in the subject to replace with background as its parameters. The two image parameters should simply be n-by-m-by-3 arrays. The description of the key color(s) should consist of one or more arrays or other values that specify a range of colors. This function should return a chroma-keyed composite of the subject and background images.

Note that this function returns, but does not display, the composite image. The function’s caller can do whatever it likes with that composite—quite likely display it, but maybe do further processing on it, or write it to a file, etc.

Follow-Up

I will grade this exercise in a face-to-face meeting with you. During this meeting I will look at your solution, ask you any questions I have about it, answer questions you have, etc. Please bring a written solution to the exercise to your meeting, as that will speed the process along.

Sign up for a meeting via Google calendar. If you worked in a group on this exercise, the whole group should schedule a single meeting with me. Please make the meeting 15 minutes long, and schedule it to finish before the end of the “Grade By” date above.