Pims Labs

This article is about a pure JavaScript implementation of the famous image resizing algorithm known under the names of Content Aware Image Resizing, Seam Carving or Liquid Rescale. A live demo is available here (Fireox 3.1/3.5 required : download it here).
Update : since this post the demo has been improved and a new post has been written : A JavaScript implementation of the content aware image resizing algorithm

Short story

Last year an incredible buzz around the new image resizing algorithm exposed by Shai Avidan and Ariel Shamir happened.

For those on vacation in 2008 I suggest to give a look at this demo : http://www.seamcarving.com/.

As I saw it I started to think about implementations. There are already some here and there. Some are fairly fast, some are plugins, some are web based. None of them (from what I saw) are browser native.

Next gen browsers and HTML5

I started some work with Flash and ActionsScript 3 and then… several demos of the new HTML5 native features of next Firefox release (3.5 formerly 3.1) by Paul Rouget (video canvas special effects and video tag and subtitles) and some great articles by Vladimir Vukićević arose.

I decided to try to implement the algorithm in pure JavaScript, and this article presents the early results. The C based CAIR implementation, and associated article, was really helpful to fully understand the algorithm thus I used the same photo from the NASA, downscaled though, for performance reason.

JavaScript - Content Aware Image Resizing Algorithm

Below you can see three pictures :

1. First is the original, its size is 256px / 170px
2. Second is resized using Content Aware Image Resizing algorithm, its size is 206px / 170px
3. Third is resized with CSS, I put it here as a reference and its size is also 206px / 170px

Original picture

50px resized - Content Aware Image Resizing algorithm

50px resized - CSS Resizing reference

Compare how the angles and the cosmonauts are reproduced on both resized images. Content Aware Image Resizing algorithm manage to get rid of parts of the image with less energy (short for : less important parts of the image) and leave intact the most important parts.

A JavaScript implementation

The current implementation I made is really a first shout. It only makes use of JavaScript and the HTML5 <canvas /> tag. It uses the Sobel convolution filter for edge detection. You’ll need Firefox 3.1/3.5 to run it (merely tested). The code is still a bit awful and I may refactor and optimize it when I have time.

Next steps could be :

• Add expansion capabilities
• Add vertical reduction and expansion
• Test some other edge detection filters
• Add some features as image portion removing
• Improve performances in order to make it real time on page resizing (is it useful ?)
• Improve performances just for the beauty of art
• Include it in a JavaScript image manipulation widget from any toolkit like Pixstatic.

Any comments are welcome !