Scientific Python and Image Processing

A substantial part of the human brain is dedicated to vision and processing of visual images. Social sites are full of visual images which 'friends' share with each other. The sheer number of images from various sites is so huge aggregation sites like Pixable (http://pixable.com/) hope to consolidate them for you and help you find the ones you want among all the clutter and noise. Another site Scalado (http://scalado.com/) created an application which allows you to remove unwanted people from your photograph among other things.

These examples illustrate that algorithms for better classification or improvement of images and their content is a hot area. Python is a superb tool for exploring ideas and algorithms very quickly. While Python may seem unsuitable for computationally expensive tasks, the scientific python community has built tools for fast numerical computation while retaining the power, versatility and flexibility of Python.

A common option for image processing in Python has been the Python Imaging Library(PIL). However, implementing and exploring image processing algorithms is better done in Scipy or Numpy. There are utility functions which allow conversion of a PIL image into a Scipy array and the reverse:

im = Image.open('image.png')

A = scipy.array(im)

im2 = Image.fromarray(A)

But, there is a ndimage module in scipy which makes it unnecessary to move between the two environments. The advantage of scipy is that you can manipulate arrays in elegant ways and not manipulate each element in a loop. Getting the sine of each element in an array, A, is as simple as:

sinA = scipy.sin(A)

As you explore the possibilities with an image, you may find numerous uses for it other areas as well, including financial modelling (http://www.logilab.org/blogentry/81704 http://augmentedtrader.wordpress.com/2011/07/02/which-programming-language-is-best-for-quants/ ) !

Fancy Indexing

The fancy indexing of Numpy/Scipy arrays provides a powerful abstraction tool. You can think about implementation of algorithms at a higher level than manipulating elements of an array. The best approach is that you can actually try these in some simple, but realistic examples and judge for yourself.

You may want to try implementing a threshold on an image by using a reference image included in scipy.

lena = scipy.lena()

# create an array with zeros of the same size

A = scipy.zeros(lena.shape, dtype='uint8')

# find the elements in lena above the threshold, e.g. 120

mask = lena > 120

# set these to white (255)

A[mask] = 255

The use of a mask as an index creates a very flexible and powerful way to manipulate arrays.

You may try a trivial example of finding a gradient in an image. A trivial , and not used, method would be to take the difference with the adjacent pixel in the same axis, e.g. for a row:

deltaA = A[1: , :] - A[:-1, :]

Indices are 0 relative; so, the first term means a slice of the array starting with the 2nd row and all columns. The second term represents a slice of the array starting with 1st row till last but one row and all columns. Both slices are of the same size and can thus be subtracted. You get speed as well because the element-wise operations are done by the c-library.

You wish to convolute an image with a 3x3 mask, M. This operation corresponds to element by element multiplication of the values in a 3x3 window around a pixel with corresponding mask elements and then summing the result. For simplicity, you may ignore the border elements.

NR, NC = A.shape

Result = [ [ (M*A[i-1:i+2, j-1:j+2]).sum()

for j in range(1,NC-1)] 

for i in range(1, NR-1)]

You can use the Laplacian mask to get another image of the edges.

You will find a rich set of routines for filtering, morphology, segmentation and feature extraction in the ndimage module. To know more, just see the section on image processing and analysis and face recognition(an example of machine learning) at http://scipy-lectures.github.com/advanced/.

New applications in processing of images are waiting to be created. One wishes that Matlab is replaced by Scipy in colleges. Then it is very likely that a clever student will create a killer app.

Comments