Weblog

February archive

Django 1.3 release schedule - Update 4

February 16, 2011

According to our most recent release schedule, we should have published our release candidate this week. However, this hasn't happened.

This is mostly due to last week's security release. Preparing a security release has diverted resources that would have otherwise been used to work on the release candidate. In addition, the act of putting out a release candidate has increased the number of eyeballs on the code, which resulted in a number of new release-blocking issue reports. These issues have (with one exception) been resolved, but this does mean that we're not as far along as we had hoped.

Therefore, we're going to push the release by another week. This should allow us to resolve the blocker issues, and clear some more of the Ready For Checkin tickets. We are now aiming to produce a release candidate late in the week of February 21, with a final release late in the week of February 28.

As always, any and all assistance is most welcome. Test out the backwards compatibility of the trunk code, try out new features, review patches, edit documentation -- the more assistance we get, the better 1.3 will be.

Errata for yesterday's security release announcement

February 10, 2011

Yesterday, we announced a security release to address a number of issues that had been brought to our attention. However, that announcement contained one error, and one omission.

Firstly, the error: the example given in the blog entry was misleading. The sample JavaScript code will work for users of Django 1.1.4, but not for users of Django 1.2.5 or trunk. If you are using trunk or Django 1.2.5, you will need to use slightly different Javascript code to extract the CSRF token from a cookie. Django's documentation for CSRF handling contains the correct examples (see the 1.1, 1.2 and trunk documentation respectively).

Secondly, the omission: we neglected to mention that the 1.2.5 release contains three other minor backwards incompatibilities. These incompatibilities relate to the process of deleting files stored in a FileField, the interpretation of initial SQL during testing, and a change to the internals of admin list_filter validation introduced in Django 1.2.4. We generally go to extraordinary lengths to avoid backwards incompatibilities of any kind, but in the case of these three changes, it was not possible to resolve bugs while preserving full backwards compatibility. For more details, see the 1.2.5 release notes.

We deeply apologize for any inconvenience caused by these errors.

Security releases issued

February 8, 2011

Today the Django team is issuing multiple releases -- Django 1.2.5 and Django 1.1.4 -- to remedy three security issues reported to us. All users of affected versions of Django are urged to upgrade immediately.

Flaw in CSRF handling

Django includes a CSRF-protection mechanism, which makes use of a token inserted into outgoing forms. Middleware then checks for the token's presence on form submission, and validates it.

Previously, however, our CSRF protection made an exception for AJAX requests, on the following basis:

  1. Many AJAX toolkits add an X-Requested-With header when using XMLHttpRequest.
  2. Browsers have strict same-origin policies regarding XMLHttpRequest.
  3. In the context of a browser, the only way that a custom header of this nature can be added is with XMLHttpRequest.

Therefore, for ease of use, we did not apply CSRF checks to requests that appeared to be AJAX on the basis of the X-Requested-With header. The Ruby on Rails web framework had a similar exemption.

Recently, engineers at Google made members of the Ruby on Rails development team aware of a combination of browser plugins and redirects which can allow an attacker to provide custom HTTP headers on a request to any website. This can allow a forged request to appear to be an AJAX request, thereby defeating CSRF protection which trusts the same-origin nature of AJAX requests.

Michael Koziarski of the Rails team brought this to our attention, and we were able to produce a proof-of-concept demonstrating the same vulnerability in Django's CSRF handling.

To remedy this, Django will now apply full CSRF validation to all requests, regardless of apparent AJAX origin. This is technically backwards-incompatible, but the security risks have been judged to outweigh the compatibility concerns in this case.

Additionally, Django will now accept the CSRF token in the custom HTTP header X-CSRFTOKEN, as well as in the form submission itself, for ease of use with popular JavaScript toolkits which allow insertion of custom headers into all AJAX requests.

The following example using the jQuery JavaScript toolkit demonstrates this; the call to jQuery's ajaxSetup will cause all AJAX requests to send back the CSRF token in the custom X-CSRFTOKEN header:

$.ajaxSetup({
        beforeSend: function(xhr, settings) {
            if (!(/^http:.*/.test(settings.url) || /^https:.*/.test(settings.url))) {
                // Only send the token to relative URLs i.e. locally.
                xhr.setRequestHeader("X-CSRFToken",
                                     $("#csrfmiddlewaretoken").val());
            }
        }
    });

For reference, the Ruby on Rails team's announcement regarding this issue may be viewed here.

Potential XSS in file field rendering

Django's form system includes form fields and widgets for performing file uploads; in rendering these fields, the name of the file currently stored in the field is displayed. In the process of rendering, the filename is displayed without being escaped, as reported by Trac user "e.generalov".

In many cases this does not result in a cross-site-scripting vulnerability, as file-storage backends can and are encouraged to (and the default backends provided with Django do) sanitize the supplied filename according to their requirements. However, the risk of a vulnerability appearing in a backend which does not sanitize, or which performs insufficient sanitization, is such that Django will now automatically escape filenames in form rendering.

Directory-traversal vulnerability on Windows

Django's file-based session-storage backend stores session data in a file named with the session key. This backend validates that the key submitted in the session cookie does not contain the filesystem path separator character, as specified by Python's os.path.sep, and if that character is found in the key an exception is raised.

As mentioned in a report to the Django team by Paul McMillan, however, this is insufficient on Windows, as filesystem operations on Windows accept multiple potential path separators (os.path.sep for Windows is the backslash character; the forward slash character is also accepted). Due to other verification mechanisms -- sessions are also stored with a hash of their contents, and the hash is verified on session deserialization -- this does not automatically allow arbitrary loading or execution of files. However, due to the possibility of session replays, or more serious vulnerabilities if an attacker manages to defeat the session verification, the file-based session backend will now use an explicit whitelist of characters which may appear in session keys. This whitelist excludes all valid path separators.

Affected versions

All three of the issues described above are present in the following currently-supported Django versions:

  • Django development trunk
  • Django 1.2
  • Django 1.1

Resolution

Patches have been applied to Django trunk, and to the 1.2 and 1.1 release branches, which resolve the issues described above. The patches may be obtained directly from the following changesets:

  • Django trunk: 15464 for the CSRF issue, 15470 for the file field issue and 15467 for the file-storage issue.
  • Django 1.2: 15465 for the CSRF issue, 15471 for the file field issue and 15468 for the file-storage issue.
  • Django 1.1: 15466 for the CSRF issue, 15472 for the file field issue and 15469 for the file-storage issue.

The following new releases have been issued:

As Django trunk is currently in a beta state, users are strongly advised not to be running production deployments from it; if you are currently doing so, however, you are urged to upgrade immediately to the latest trunk, which contains patches for these issues.

General notes regarding security

As always, we ask that potential security issues be reported via private email to security@djangoproject.com, and not via Django's Trac instance or the django-developers list.

If you are or represent a third-party distributor of Django and did not receive a notification email regarding this announcement from the Django release manager, please contact james@b-list.org.

Django 1.3 release schedule - Update 3

February 2, 2011

According to our release schedule, this week should have seen the release of Django 1.3 final. Obviously, this hasn't happened.

The good news is that at this time, there are no tickets blocking a release. To the best of our knowledge, Django's trunk repository is currently free of regressions, there are no bugs causing serious data loss, and there are no major problems with new features. There is one ticket (#15149) that might turn into a release blocker, but is still undergoing triage. This means we're almost in a position to produce a final release.

However, in order to make Django 1.3 as good as it can be, we're going to keep the release open for a couple more weeks. This will allow us to clear out the backlog of patches that have been reviewed and are marked Ready For Checkin. There are currently 29 tickets in the Ready For Checkin state. The aim is to commit as many of these patches as time allows. If you have a ticket that is on Milestone 1.3 and you want to see it actually get committed to 1.3, then you need to get it reviewed by someone so it can progress to Ready For Checkin. If it isn't Ready For Checkin, it won't be checked in!

We have also just completed migrating Django's translation infrastructure to Transifex. This requires the existing translation teams to register on the new service. If you are involved in a translation team, make sure you have registered on Transifex so you are ready when the release candidate string freeze occurs.

With these factors in mind, we have revised the release schedule: we are now aiming to produce a release candidate in the week of February 14, with a final release in the week of February 21.

As always, any and all assistance is most welcome. Test out the backwards compatibility of the trunk code, try out new features, review patches, edit documentation -- the more assistance we get, the better 1.3 will be.