peach³

I use South whenever a model needs to be changed. One thing that is not really documented in the South documentation is how to convert data. After some trial and error I found a way to use Django QuerySets to access both the old and new models to convert data.

For example, I used to have two models defined like this:

class Assignment(models.Model):
    course = models.ForeignKey(Course)
    name = models.CharField(max_length=100)

class AssignmentEdition(models.Model):
    assignment = models.ForeignKey(Assignment)
    # ... other fields

The way the assignments were defined, each edition of an assignment has the same name. I needed to change this into:

class Assignment(models.Model):
    course = models.ForeignKey(Course)

class AssignmentEdition(models.Model):
    assignment = models.ForeignKey(Assignment)
    name = models.CharField(max_length=100)
    # ... other fields

Each edition has it’s own name. It’s still possible for editions to have the same name, but it is also possible to have a different name and still be an edition of the same assignment.

During the migration I want each assignment edition to get the name of it’s assignment. One way to do this is to use SQL queries to get the names of the assignments and update the assignment edition names, but that’s not very Django-like. I want to use QuerySets. I solved it in the following way (this works with Django 1.0.2 and South 0.4):

from south.db import db
from django.db import models

class Assignment(models.Model):
    name = models.CharField(max_length=100)

    class Meta:
        db_table = 'assignment_assignment'
        app_label = 'assignment_migrations_0002_assignment_names'

class AssignmentEdition(models.Model):
    assignment = models.ForeignKey(Assignment)
    name = models.CharField(max_length=100)

    class Meta:
        db_table = 'assignment_assignmentedition'
        app_label = 'assignment_migrations_0002_assignment_names'

class Migration:
    def forwards(self):
        db.start_transaction()

        # Add the new name column to AssignmentEdition
        db.add_column('assignment_assignmentedition', 'name',
                      models.CharField(max_length=100))

        # Convert the existing data
        if not db.dry_run:
            db.commit_transaction()
            db.start_transaction()

            for ae in AssignmentEdition.objects.all():
                ae.name = ae.assignment.name
                ae.save()

        # Delete the name column from Assignment
        db.delete_column('assignment_assignment', 'name')

        db.commit_transaction()

    def backwards(self):
        raise NotImplementedError

First, I need to create the models as they are when I want to convert the data. I can’t rely on definitions in models.py because first of all Assignment won’t have any name defined there, and secondly a future migration might remove the name field from AssignmentEdition as well. Also I only include the fields that are needed for the conversion here, so eg. the course field in Assignment is not included.

The Meta options are required for Django to be able to find the correct database tables. The db_table meta option is the table name (as created by Django), the app_label meta option is required because Django caches it’s models, and if I were to define another AssignmentEdition in another migration, Django will use the AssignmentEdition defined here instead of the one for that migration. It uses app_label as the key for this caching, so app_label needs to be a unique label for this migration.

The rest is pretty straightforward: after adding the new name column and saving the transaction, I can loop over all existing assignment editions and update the name field using a standard Django QuerySet. After the conversion the name column in Assignment can be deleted.

Tags: Django, migration, south

This entry was posted on Tuesday, February 10th, 2009 at 11:08 and is filed under Django. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.