Make use of Imagecache for external files

Posted on December 29, 2011


Drupal module Imagecache is so good in doing online image manipulation that it makes sense to use it in web services, also on projects where Drupal is not used. This time I had to work with product images, we are talking about some thousands of JPGs and more to come. There are two systems using the same photos: company website (Drupal) and product engineering web service (custom extranet application). JPGs are laying in between these two systems.

Earlier, I had made my own image manipulation scripts to work things out in this project. All the images can’t be sent through Drupal anyway, I was reasoning, so Imagecache can’t be used. Later problems started to arise, related to image manipulation scripts I had wrote. Clearly they had to be revisioned, but then I got an idea to try to trick Imagecache to do the work for me. It has neat graphic setup page in Drupal, too, so making future changes would be super-easy.

First tryout: symlinks

Symlinks are working with everything, so nothing would be easier than just to link the material folder under Imagecache. That is, to trick stuff under Imagecache-active /styles/ folder, stuff that is not really there.

ln -s /path/to/material /path/to/www/sites/default/files/material

Now, lets try it with browser. I want to get file testphoto.jpg, normally found from local path /path/to/material/test/testphoto.jpg, so I’m going to All I get is “Error generating image.”. Inspecting from shell, Imagecache creates the path correctly, so new folders /styles/thumbnail/material/ and /styles/thumbnail/material/test/ have appeared. No files are generated though, also no error messages are found in any logs.

Browsing for this suggests two key points: file permissions and memory problems. I worked on for these for a while, with no luck. I verified that Imagecache is working normally with other files. That left one matter open: does Imagecache work with symlinks? Apparently not, although they are saying that since Drupal 7 it would. Well it didn’t work for me.

This is the first time ever I have seen that something is not following a symbolic link.

Second tryout: mounting

The number one alternative for symlink, mounting, didn’t work for me either. There are two reasons: 1. I’m not clever enough to use it and 2. this clients server is some improper CentOS which I’m not too familiar with. Anyway I tried this (as su):

mount /path/to/material /path/to/www/sites/default/files/material –bind –ro

But apparently CentOS is giving troubles when joining stuff from somewhere to itself. After some search I had again tried several choices with no luck, in the end giving me hard time to unmount the testing folder. “Device is busy” or “Trying to umount, but this resource is not mounted”. When I finally got rid of it with umount -fn and rm -rf I had also removed the original source, which I just can’t understand how that happend. It must’ve been abovementioned reason 1.

Third tryout: rsync and it works!

After restoring original files from backups (always remember to back up your stuff!) I had only one more choice to use: to actually copy everything for /styles/. That makes little sense but to make it work I did it anyway. There is a massive update coming to this client’s extranet, so I’m looking forward to change file system for product images the other way around: to upload everything under Drupal and to symlink them forward from there. The more I think it the more it makes sense, as Internet is available from everywhere so using linked stuff from there is guaranteed to work with all future applications.

So but now it is done with rsync, which is great because it can take care of the transferring files now and also to automatically keep them up to date. Also doing a recursive, selective copy is easy with rsync, as we only need JPGs from the material folder.

rsync -av –include “*/” –include “*.JPG” –include “*.jpg” –exclude “*” ~/public_html/material/. ~/public_html/sites/default/files/styles/material/.

Just add this to cron job table to make new files copied automatically. Note that rsync only rewrites new or changed files, keeping load put on your server minimal.

7 10 * * * rsync -av –include “*/” –include “*.JPG” –include “*.jpg” –exclude “*” ~/public_html/material/. ~/public_html/sites/default/files/material/.

Now thumbnails or whatever modified files you set up in Drupal Configure -> Image styles are easily accessible through path/to/public_html/ sites/default/files/styles/your_style_name_here/material/test/testphoto.jpg.