News, ideas and randomness

Versioned Media and Expires Headers in Django

Posted: December 18th, 2008 | Author: Andrew Gleave | Filed under: Django | No Comments »

We try to make our sites as responsive as possible, and as part of our testing, we realised that we should do the right thing and add Expires Headers to our static media. Our web servers are configured so when a client requests an image, stylesheet or JavaScript file, it is returned along with a far-future expires header. This tells the client not to ask for that file again, but to cache it for a month or more.

Encouraging Caching with Expires Headers

Without an expires header, the client will request media files each time it loads a page. Using if-modified-since and etag headers, the server usually doesn’t return the media files, but instead returns a 304 Not Modified response. Not resending the data is good. Not having to deal with the request at all is even better – that’s what expires headers offer.

Of course, if you tell clients not to request your stylesheet again for a month, what happens when you change your stylesheet? The client won’t know and won’t get the changes. That’s pretty disastrous. What we need is a way of changing the URL when our media changes so that clients will pick up the new version.

Changing URLs when Content Changes

There are a number of ways to serve your media so you can specify far-future dates in the expires header, but still have the client pick up new versions. We refer to this as versioned media.

One common scheme is to put the modification date of the file in its URL. When the file date changes, the URL changes and clients request the new version. The URL might look something like /media/main.css?200812180930 or /media/main-2008-12-18-0930.css. The former is easier because the querystring is ignored by the web server and the file returned as normal.

Using the date is good if you want more granular per-file versioning, but it seems a little messy. We decided to use a version number in the URL instead, e.g. /media/v123/main.css. To make this work we need to put a version number in the templates and have our web server ignore the version number and just serve the file.

Versioned Media Context Processor

Typically, Django-based sites use the MEDIA_URL context processor to include external resources such as Javascript and images in to their templates. We expanded on this idea by having VERSIONED_MEDIA_URL which puts the version number in as well.

Remembering to update a version number would be error prone, so we wanted to transparently support any updates to media. We use Subversion for version control, and figured out that we could use the versioning metadata of our media directory to help us generate a unique path, which would change as the media was updated. That’s exactly what we needed, and would allow us to specify expires headers on all paths which include a version number but still ensure that users would receive new copies of files if any changed.

from django.utils.version import get_svn_revision
from django.conf import settings

VERSIONED_MEDIA_URL = None

def get_versioned_media_url():
    if hasattr(settings, 'MEDIA_VERSION') and settings.MEDIA_VERSION is not None:
       version = 'v%s' % settings.MEDIA_VERSION
    else:
        revision = get_svn_revision(settings.MEDIA_ROOT)
        version = revision.replace('SVN-', 'v')
    return u'%s%s/' % (settings.MEDIA_URL, version)

def versioned_media(request):
    """Adds versioned media url to the context."""
    global VERSIONED_MEDIA_URL
    if not VERSIONED_MEDIA_URL:
        VERSIONED_MEDIA_URL = get_versioned_media_url()
    return {'VERSIONED_MEDIA_URL': VERSIONED_MEDIA_URL}

You can see from the code that we’ve added a MEDIA_VERSION setting which is either manually set, or can be set by a deployment script. We make use of django’s get_svn_revision method to pick up the version number from our MEDIA_ROOT and we then append our version number to our MEDIA_URL, adding it to context as VERSIONED_MEDIA_URL.

It’s convenient to update code on your server with a simple svn up, but serving from a working copy may have security issues. Instead, we have a deployment script which updates a working copy then copies the files (excluding .svn directories) to the directories used by the web server. It finds the revision number and writes that to the settings file so our version number is still updated automatically.

Configuring Expires Headers in Nginx

Now that we have a version-specifc URL for all our media, we need to configure the webserver to add an Expires Header to any requests which are destined for a versioned URL. We use Nginx, but the theory applies to any webserver.

location /versioned-media {
    internal;
    expires 90d;
    alias   /srv/www/live/thebarbershop/site/media;
}

location /media {
    rewrite /v(?:\d+)/(.*) /versioned-media/$1;
    rewrite /vunknown/(.*) /media/$1;
    root   /srv/www/live/thebarbershop/site;
}

We configure our /media URL so that any request which matches the version string created by our context processor is forwarded to the /versioned-media path, which then applies the expires header and sets the expiry date to 90 days in the future. Any request path without a version number simply gets served without the expires header.

One drawback: committing a change means that all versioned media URLs change, not just the one for the file that changed. However, we feel this is only a small drawback given the advantages this gives for the common case of a high-traffic site with relatively infrequent changes to the base media.

When you couple adding Expires Headers with other techniques like:

you can dramatically reduce both the number and size of requests to your application, and give users a more responsive experience.



Leave a Reply