Fun with Image Smoothing in Python

December 07, 2007

Smoothing an image is helpful for a few different reasons. The big one that I've been using image smoothing for is to remove noise from an image. Laplacian edge detection is highly susceptible to noise due to the Laplacian operator being a second derivative operator. The Python Imaging Library (PIL) provides a few different ways to produce smooth images. The most obvious would be to use the ImageFilter.SMOOTH class.

The image I'll be working on throughout this post is this greyscale picture of me:

Standard smoothing in PIL is super easy. For example:

import sys, Image, ImageFilter 
 
def pil_smooth(filename, outfile): 
      img = Image.open(filename) 
      img = img.filter(ImageFilter.SMOOTH) 
      img.save(outfile) 
 
if __name__ == '__main__': 
    pil_smooth(sys.argv[1], sys.argv[2])

Using the image of me above, here's the result of this smoothing operation:

That's a pretty good result for just using stock PIL image filters. The Python Imaging Library also offers a SMOOTH_MORE filter. Replacing the ImageFilter.SMOOTH above with ImageFilter.SMOOTH_MORE, we get:
Digging into the PIL source gives a really good indication of how it produces these results. For example, the BuiltinFilter classes (such as ImageFilter.SMOOTH) use filter arguments to produce different results. These filter arguments are a size tuple, which is the width and height of the kernel, the convolution kernel itself as a sequence containing weighted values, the scale which is used to divide the result of each pixel, and finally the offset which is added to the result after it has been divided by the scale factor.

For ImageFilter.SMOOTH, these filter arguments are:

size = (3,3) 
scale = 13 
offset = 0 
kernel = ( 
    1, 1, 1, 
    1, 5, 1, 
    1, 1, 1 
)

I'm going to present a way to implement image smoothing so you can have a better idea of what's really going on when you smooth an image. One small note: the scale part of the process defaults to the sum of the weights in the kernel. So if it isn't present, you can calculate the default scale by doing this:

# assumes size = (kernel_width, kernel_height) 
scale = 0 
for i in xrange(size[0]): 
    for j in xrange(size[1]): 
        scale += kernel[j+i*size[0]]

With that out of the way, here's an implementation of image smoothing that is equivalent to calling ImageFilter.SMOOTH like the code above:

import Image 
import sys 
 
# img is an Image object 
# size is width, height tuple of kernel 
# scale is the value by which the sum of the convolution operation is divided 
# offset is added to the result of sum / scale 
# This smooth() function can only operate on 3x3 kernels for now. Adding support for 5x5 
# is similar to 3x3, but it's something I'll cover in a later post. 
def smooth(img, size, kernel, scale=0, offset=0): 
      # note that converting an image to greyscale isn't required. 
      # it's just something that I do and you can leave it out if you want. 
      if img.mode != 'L': img = img.convert('L') 
      pixels = list(img.getdata()) 
      width, height = img.size 
 
      outimg = Image.new('L', (width, height)) 
      outpixels = list(outimg.getdata()) 
 
      if scale == 0: 
          # calculate it from the sum of the weights in the kernel 
          for i in xrange(size[0]): 
              for j in xrange(size[1]): 
                  scale += kernel[j+i*size[0]] 
 
      for x in xrange(width): 
          # copy top row of pixels to eliminate top border 
          outpixels[x] = pixels[x] 
 
      for y in xrange(1, height-1): 
          # copy left column of pixels to eliminate left border 
          outpixels[y*width] = pixels[y*width] 
          for x in xrange(1, width-1): 
              result = 0 
          # PIL's C implementation unrolls these loops, but the idea is the same. 
          for i in xrange(-1, 2): 
              for j in xrange(-1, 2): 
                  result += pixels[(y+j)*width+x+i] * kernel[(j+1)*size[0]+i+1] 
 
          result = result / scale + offset 
          if result <= 0: outpixels[y*width+x] = 0 
          elif result >= 255: outpixels[y*width+x] = 255 
          else: outpixels[y*width+x] = result 
 
          # copy right column of pixels to eliminate right border. 
          # note the x+1: x at this point is width-2, so increment it once 
          # to get the far right column. 
          outpixels[y*width+x+1] = pixels[y*width+x+1] 
 
      for x in xrange(width): 
          # copy bottom row of pixels to eliminate bottom border. 
          # note the y+1: same reason as above for x+1. 
          outpixels[(y+1)*width+x] = pixels[(y+1)*width+x] 
 
      outimg.putdata(outpixels) 
      return outimg 
 
if __name__ == '__main__': 
    img = Image.open(sys.argv[1]) 
 
    kernel = ( 
        1, 1, 1, 
        1, 5, 1, 
        1, 1, 1 
    ) 
 
    outimg = smooth(img, (3,3), kernel, 13, 0) 
    outimg.save(sys.argv[2])

Notice how it loops through 1 <= y < height-1 and 1 <= x < width? That's because doing this processing through the entire image produces weird borders. The code above compensates for this and eliminates the dark borders by copying the original border pixels to the borders of the output image.

So what happens when I run the code above on the original image? Check it out:

Being able to code this kind of stuff is really cool, but for these really basic examples of standard smoothing, using PIL's built-in filters is definitely the way to go. You're also not limited to using PIL's built-in filters. If you require different kernels and even different scaling and offset attributes, PIL provides ways for you to do that.

I was hoping to cover the ways you can implement your own filters in PIL, but it's getting late so I will try covering them tomorrow if I have time.