| SOCR ≫ | DSPA ≫ | DSPA2 Topics ≫ |
DSPA2: Data Science and Predictive Analytics (UMich HS650)
Appendix 6: Image Convolution, Filtering, Fourier Transform using
EBImage
Appendix 6: Image Convolution, Filtering, Fourier Transform using EBImage
SOCR/MIDAS (Ivo Dinov)
SOCR/MIDAS (Ivo Dinov)
June 2024
In this DSPA Appendix, we present examples of image processing, spectral manipulation, and filtering.
1 Getting started
r Biocpkg(“EBImage”) is an R package
distributed as part of the Bioconductor project. To install the
package, start R and enter:
Once the R package(“EBImage”) is installed, it can be
loaded by the following command.
- Note: If you can’t install the EBImage package for any reason (e.g., operating system or version incompatibility, this is not a problem as you should be able to use the jpeg package instead.
2 Reading, displaying and writing images
Basic R package(“EBImage”) functionality includes
reading, writing, and displaying of images. Images are read using the
function readImage, which takes as input a filename or an
URL. To start off, let us load a sample picture distributed with the
package.
The R package (“EBImage”) currently supports three image
file formats: jpeg, png and tiff.
Additional image formats can be imported using the R
GitHubPkg (“aoles/RBioFormats”), which adds support for a wide range of
file formats including proprietary microscopy image data and
metadata.
Imported images can be visualized by the function
display().
In interactive R sessions, display opens
the image in a JavaScript viewer in a web browser tab. Mouse or keyboard
shortcuts allow zooming in and out of the image, panning, and cycling
through multiple image frames. Images can also be displayed using core
R plotting methods, which allows for combining images with
other plotting functionality, e.g., adding text labels on top of the
image.
display(img, method="raster")
text(x = 20, y = 20, label = "Colorful Parrots", adj = c(0,1), col = "orange", cex = 2)
The graphics displayed in an R device can be saved using
R package(“base”) R functions
dev.print or dev.copy. For example, let’s save
our annotated image as a JPEG file and verify its size on disk.
filename = "parrots.jpg"
dev.print(jpeg, filename = filename , width = dim(img)[1], height = dim(img)[2])png
2 [1] 34379The default behavior of display can be globally changed
by setting the
"options("EBImage.display") to either“browser”or“raster”`.
This is useful, for example, to preview images inside RStudio.
It is also possible to read and view color images,
Images may contain multiple frames, in which case they can be
displayed all at once in a grid arrangement by specifying the function
argument all = TRUE.
nuc = readImage(system.file("images", "nuclei.tif", package="EBImage"))
display(nuc, method = "raster", all = TRUE)
Alternatively, single frames can be displayed.
In addition to importing images, images can be exported (saved) to
files using the EBImage::writeImage(). The image that we
loaded was a PNG file. To save this image as a JPEG file, the JPEG
format allows setting a quality value between 1 and 100 to reflect the
desired level of image compression. The default value of the
quality argument of writeImage is 100. Smaller
values yield images of smaller size, but worse resolution (less
detail).
Similarly, we could have saved the image as a TIFF file and set which
compression algorithm we want to use. For a complete list of available
parameters see ?writeImage.
3 Image data representation
EBImage uses a package-specific class Image to
store and process images. It extends the R base class
array, and all R package(“EBImage”) functions
can also be called directly on matrices and arrays. You can find out
more about this class by typing ?Image. Let us peek into
the internal structure of an Image object.
Formal class 'Image' [package "EBImage"] with 2 slots
..@ .Data : num [1:768, 1:512] 0.447 0.451 0.463 0.455 0.463 ...
..@ colormode: int 0
..$ dim: int [1:2] 768 512The .Data slot contains a numeric array of pixel
intensities. We see that in this case the array is two-dimensional, with
768 times 512 elements, and corresponds to the pixel width and height of
the image. These dimensions can be accessed using the dim
function, just like for regular arrays.
[1] 768 512Image data can be accessed as a plain R
array using the imageData accessor.
[,1] [,2] [,3] [,4] [,5] [,6]
[1,] 0.4470588 0.4627451 0.4784314 0.4980392 0.5137255 0.5294118
[2,] 0.4509804 0.4627451 0.4784314 0.4823529 0.5058824 0.5215686
[3,] 0.4627451 0.4666667 0.4823529 0.4980392 0.5137255 0.5137255The as.array() method can be used to coerce an
Image to an array.
[1] FALSEThe distribution of pixel intensities can be plotted in a histogram,
and their range inspected using the range function.
[1] 0 1A useful summary of Image objects is also provided by
the show method, which is invoked if we simply type the
object’s name.
Image
colorMode : Grayscale
storage.mode : double
dim : 768 512
frames.total : 1
frames.render: 1
imageData(object)[1:5,1:6]
[,1] [,2] [,3] [,4] [,5] [,6]
[1,] 0.4470588 0.4627451 0.4784314 0.4980392 0.5137255 0.5294118
[2,] 0.4509804 0.4627451 0.4784314 0.4823529 0.5058824 0.5215686
[3,] 0.4627451 0.4666667 0.4823529 0.4980392 0.5137255 0.5137255
[4,] 0.4549020 0.4666667 0.4862745 0.4980392 0.5176471 0.5411765
[5,] 0.4627451 0.4627451 0.4823529 0.4980392 0.5137255 0.5411765For a more compact representation without the preview of the
intensities array use the print method with the argument
short set to TRUE.
Image
colorMode : Grayscale
storage.mode : double
dim : 768 512
frames.total : 1
frames.render: 1 Color images are based on 3 channels (RBG).
Image
colorMode : Color
storage.mode : double
dim : 768 512 3
frames.total : 3
frames.render: 1 They differ from gray-scale images by the property
colorMode and the number of dimensions, 3 (B&W) vs. 4
(color). The colorMode slot turns out to be convenient when
dealing with stacks of images. If it is set to gray-scale,
then the third and all higher dimensions of the array are considered as
separate image frames corresponding, for instance, to different
z-positions, time points, replicates, etc. On the other hand, if
colorMode is Color, then the third dimension
is assumed to hold different color channels, and only the fourth and
higher dimensions are used for multiple image frames.
imgcol contains three color channels, which correspond to
the red, green and blue intensities of the photograph. However, this
does not necessarily need to be the case, and the number of color
channels is arbitrary.
The “frames.total” and “frames.render” fields shown by the object
summary correspond to the total number of frames contained in the image,
and to the number of rendered frames. These numbers can be accessed
using the function numberOfFrames by specifying the
type argument.
[1] 1[1] 3Image frames can be extracted using getFrame and
getFrames. getFrame returns the i-th frame
contained in the image y. If type is "total",
the function is unaware of the color mode and returns an xy-plane. For
type="render" the function returns the i-th image as shown
by the display function. While getFrame returns just a
single frame, getFrames retrieves a list of frames which
can serve as input to lapply-family functions. See the
“Global thresholding” section for an illustration of this approach.
Let’s look at nuclear/cellular imaging data, which contains 4 total frames that correspond to the 4 separate gray-scale images, as indicated by “frames.render”.
Image
colorMode : Grayscale
storage.mode : double
dim : 510 510 4
frames.total : 4
frames.render: 4
imageData(object)[1:5,1:6,1]
[,1] [,2] [,3] [,4] [,5] [,6]
[1,] 0.06274510 0.07450980 0.07058824 0.08235294 0.10588235 0.09803922
[2,] 0.06274510 0.05882353 0.07843137 0.09019608 0.09019608 0.10588235
[3,] 0.06666667 0.06666667 0.08235294 0.07843137 0.09411765 0.09411765
[4,] 0.06666667 0.06666667 0.07058824 0.08627451 0.08627451 0.09803922
[5,] 0.05882353 0.06666667 0.07058824 0.08235294 0.09411765 0.105882354 Color management
As described in the previous section, the class Image
extends the base class array and uses
colorMode to store how the color information of the
multi-dimensional data should be handled.
The function colorMode can be used to access and change
this property, modifying the rendering mode of an image. For example, if
we take a Color image and change its mode to
gray-scale, then the image won’t display as a single color
image anymore but rather as three separate gray-scale frames
corresponding to the red, green and blue channels. The function
colorMode does not change the actual content of the image
but only changes the way the image is rendered by R
package(“EBImage”).
Color space conversions between gray-scale and
Color images are performed using the function
channel. It has a flexible interface which allows to
convert either way between the modes, and can be used to extract color
channels. Unlike colorMode, channel changes
the pixel intensity values of the image.
Color to gray-scale conversion modes
include taking a uniform average across the RGB channels, and a weighted
luminescence preserving conversion mode better suited for display
purposes.
The asred, asgreen and asblue
modes convert a gray-scale image or array into a color image of the
specified hue.
The convenience function toRGB promotes a gray-scale
image to RGB color space by replicating it across the red, green and
blue channels, which is equivalent to calling channel with
mode set to rgb. When displayed, this image doesn’t look
different from its gray-scale origin, which is expected because the
information between the color channels is the same. To combine three
gray-scale images into a single RGB image use the function
rgbImage.
The function Image can be used to construct a color
image from a character vector or array of named R colors
(as listed by colors()) and/or hexadecimal strings of the
form “#rrggbb” or “#rrggbbaa”.
Warning in matrix(rep(c("red", "green", "#0000ff"), 25), 5, 5): data length
differs from size of matrix: [75 != 5 x 5]Image
colorMode : Color
storage.mode : double
dim : 5 5 3
frames.total : 3
frames.render: 1
imageData(object)[1:5,1:5,1]
[,1] [,2] [,3] [,4] [,5]
[1,] 1 0 0 1 0
[2,] 0 1 0 0 1
[3,] 0 0 1 0 0
[4,] 1 0 0 1 0
[5,] 0 1 0 0 1
5 Manipulating images
Being numeric arrays, images can be conveniently manipulated by any of R’s arithmetic operators. For example, we can produce a negative image by simply subtracting the image from its maximum value.
We can also increase the brightness of an image through addition, adjust the contrast through multiplication, and apply gamma correction through exponentiation.
In the example above we have used combine to merge
individual images into a single multi-frame image object.
Furthermore, we can crop and threshold images with standard matrix operations.
The thresholding operation returns an Image object with
binarized pixel values. The R data type used to store such
an image is logical.
Image
colorMode : Grayscale
storage.mode : logical
dim : 384 384
frames.total : 1
frames.render: 1
imageData(object)[1:5,1:6]
[,1] [,2] [,3] [,4] [,5] [,6]
[1,] FALSE FALSE FALSE FALSE FALSE FALSE
[2,] FALSE FALSE FALSE FALSE FALSE FALSE
[3,] FALSE FALSE FALSE FALSE FALSE FALSE
[4,] FALSE FALSE FALSE FALSE FALSE FALSE
[5,] FALSE FALSE FALSE FALSE FALSE FALSEFor image transposition, use transpose rather than R’s
R package(“base”) function t. This is because
the former one works also on color and multiframe images by swapping its
spatial dimensions.