Image Augmentation using Keras

Data preparation is required when working with neural network and deep learning models. Increasingly data augmentation is also required on more complex object recognition tasks.

Data augmentation means to increase the amount of data. In, other words, having a larger dataset means a more robust model. But acquiring more data cannot always be as easy and there may be a problem of storing them and feeding it to model.

To mitigate this problem, we can manually increase the data by doing some changes or we can make use of one of Keras image preprocessing class that can do this in just a few lines of codes.

In this post you we’ll see how to use data preparation and data augmentation with your image datasets when developing and evaluating deep learning models in Python with Keras.

After reading this post, you will get to know:

1. About the image augmentation API provide by Keras and how to use it with your models.How to perform feature standardization.
2. How to perform ZCA whitening of your images.
3. How to augment data with random rotations, shifts and flips.
4. How to save augmented image data to disk.

Keras Image Augmentation API: ImageDataGenerator

It generates batches of augmented data from the original batch. It takes the images, applies some random transformation on it, and produces a new batch for training.

Note: One thing to note here that ImageDataGenerator does not return the original images instead it just returns batches of augmented data which is a result of some transformation done on original data.

Loading Dataset

We will use MNIST handwritten digit recognition for data augmentation. Executing below code will load MNIST dataset from keras.datasets

Split data into Train & Test

Standardizing Features

The pixel standardization is supported at two levels: either per-image (called sample-wise) or per-dataset (called feature-wise). Specifically, the mean and/or mean and standard deviation statistics required to standardize pixel values can be calculated from the pixel values in each image only (sample-wise) or across the entire training dataset (feature-wise).

You can perform feature standardization by setting the featurewise_center and featurewise_std_normalization arguments on the ImageDataGenerator class.

Standardizing images across dataset, mean=0, stdev=1

Output Image: Standardised features

Running this example, you can see that the effect is different, seemingly darkening and lightening different digits.

ZCA Whitening

A whitening transform of an image is a linear algebra operation that reduces the redundancy in the matrix of pixel images. Typically, image whitening is performed using the Principal Component Analysis (PCA) technique. More recently, an alternative called ZCA shows better results and results in transformed images that keeps all of the original dimensions and unlike PCA, resulting transformed images still look like their originals.

ZCA Whitening

Output Image: ZCA

Running this example, you can see that the effect is different, seemingly darkening and lightening different digits.

Random Rotations

You can train your model to better handle rotations of images by artificially and randomly rotating images from your dataset during training. The example below creates random rotations of the MNIST digits up to 90 degrees by setting the rotation_range argument.

Rotation at 90 degrees

Output Image: Random Rotations

You can see that images have been rotated left and right up to a limit of 90 degrees. This is not helpful on this problem because the MNIST digits have a normalized orientation, but this transform might be of help when learning from photographs where the objects may have different orientations.

Random Shifts

Objects in your images may not be centered in the frame. They may be off-center in a variety of different ways. You can train your deep learning network to expect and currently handle off-center objects by artificially creating shifted versions of your training data. Keras supports separate horizontal and vertical random shifting of training data by the width_shift_range and height_shift_range arguments.

Shift whidth and height

Output Image: Random Shifts

Again, this is not required for MNIST as the handwritten digits are already centered, but you can see how this might be useful on more complex problem domains.

Random Flips

Another augmentation to your image data that can improve performance on large and complex problems is to create random flips of images in your training data. Keras supports random flipping along both the vertical and horizontal axes using the vertical_flip and horizontal_flip arguments.

Horizontal and Vertical Flips

Output Image: Random Flips

Flipping digits is not useful as they will always have the correct left and right orientation, but this may be useful for problems with photographs of objects in a scene that can have a varied orientation.

You can find code here: Github Link

Thanks for Reading!

Cheers!