A Django Snippet to Refresh Your Database

Django doesn’t have anything remotely close to migrations in Rails. It does have the syncdb management command, but that’s only good for initial table creation. If you need to add or remove a model attribute at a later point, you’re pretty much stuck with manual alter table commands to your database.

I’ve found that I have to do this entirely too often in development, and it becomes increasingly frustrating with frequency. As a result, I’ve written a little python script to take care of all the grunt work for me.

Disclaimer

Now, before I post the code, let me make clear that although I’ve used this a number of times already, I cannot guarantee that it’s 100% bug-free. It should definitely only be used on a development database, if for no other reason than should the catastrophic happen, you’ve only lost dummy data and not real user accounts and such. Also, I don’t claim to be a python expert. In fact, part of the reason I’m posting this is that hopefully a few python experts might stumble by and critique my code. Still, all that said, I think it still has benefit for the community, and hopefully it will be a help to some of you.

Code

#!/usr/bin/env python

print "Setting Up Environment... ",
try:
    import settings # Assumed to be in the same directory.
except ImportError:
    import sys
    sys.stderr.write("Error: Can't find the file 'settings.py' in the directory containing %r.\n(If the file settings.py does indeed exist, it's causing an ImportError somehow.)\n" % __file__)
    sys.exit(1)

from django.core.management import setup_environ, call_command
setup_environ(settings)
print "Done"

import sys

print "Dumping Data... ",
sys.stdout = open('dumped_data.json', 'w')
call_command('dumpdata', format='json', indent=4)
sys.stdout.close()
sys.stdout = sys.__stdout__
print "Done"

print "Deleting Tables... ",
import StringIO
from django.db.models import get_apps
app_labels = [app.__name__.split('.')[-2] for app in get_apps()]
sys.stdout = buffer = StringIO.StringIO()
call_command('sqlclear', *app_labels)
sys.stdout = sys.__stdout__

queries = buffer.getvalue().split(';')[1:-2]

from django.db import connection
cursor = connection.cursor()
for query in queries:
    cursor.execute(query.strip())
print "Done"

print "Synching Database... "
#disable the "create a super user" question
from django.contrib.auth.management import create_superuser
from django.contrib.auth import models as auth_app
from django.db.models import signals
from django.dispatch import dispatcher
dispatcher.disconnect(create_superuser, sender=auth_app, signal=signals.post_syncdb)

call_command('syncdb')
print "Done"

print "Loading Back Data... "
call_command('loaddata', 'dumped_data.json')
import os
os.remove('dumped_data.json')
print "Done"

What Does it Do?

First, the script loads in your project’s settings.py and sets up the Django environment. If you’ve ever looked at the code behind manage.py, you’ll notice I basically ripped that part off. Second, the script uses Django’s dumpdata management command to save all the current data in the database in a JSON object.

Third, it gets a list of all the currently install apps, and uses Django’s sqlclear management command to get the database specific delete statements. Essentially, it’s just emptying out the database. I debated about simply dropping the database and creating a new one, but I wanted this script to be as database agnostic as possible.

Next, it runs Django’s syncdb management command to recreate the database based on the current state of your models. However, before is does that, it disables the signal responsible for the “create a superuser” prompt that Django’s Auth module gives. Finally, it reloads the data we dumped previously back into the database.

At this point, you should have an exact duplicate of your old database, only with the changes you made to your models implemented.

How to Use it?

Save that code in an file in the root of your project (same level as settings.py). I called mine ‘refresh_db.py’. If you’re on a *nix based platform you’ll want to give the script execute priveleges, as well (chmod 755). Then, just run it from the command line, whenever you see fit.

Tags: code, database, django, python, scripts

Trackback URL

4 Responses to “A Django Snippet to Refresh Your Database”

1

Keats Says:
March 11th, 2008 at 6:30 am

you could have done it with a shell script
like this :
#!/bin/sh
./manage.py dumpdata >> blabla.json
./manage.py flush
./manage.py loadata blabla.json

or simply by adding a setting
FIXTURE_DIRS = (
‘/home/keats/fixtures’,
)

then ./manage.py dumpdata >> /home/keats/fixtures/initial_data.json

now each time you call :
./manage.py flush
it will flush the database and automatically reload the data in the initial_data.json
the name of the file is important

you can also add sql directory in application directory
the syntax is sql/model_name.json or .sql
and it will be loaded each time you flush your database

++
2

chrisdpratt Says:
March 18th, 2008 at 3:03 pm

Well, I feel a little stupid ;). But that’s why I posted this. Thanks for the tips.
3

stfu pls! » Refresh a django database Says:
June 20th, 2008 at 11:18 am

[…] just ran across this post on Chris Platt’s blog and don’t want to lose the code. There are a few ways to refresh your django database (that […]
4

Slava Says:
June 27th, 2008 at 3:50 am

Slight correction to first comment (from Keats) - there’s a typo - it should be “loaddata” (double d).