Django Migrations to the Rescue

Django Migrations to the Rescue

I have been programming professionally for 19 years now and it was always a pain to maintain the changes you do in your database prepared to be replicated in all the installations your application needed. Remember it was desktop software before so we needed to create programs that check the version of the database they were using and somehow upgrade the database to the new fields, tables, triggers, views, stored procedures, user defined functions etc, so the new software will be able to run without errors.

It was difficult to maintain because of different versions of SQL your application was using, because the application really depends on this and the user didn't even know that we needed to do this.

Now with Django Migrations since Django 1.7(South was first) the whole process is a breeze. 

Here you have what the documentation said at its very beginning

Migrations

Migrations are Django’s way of propagating changes you make to your models (adding a field, deleting a model, etc.) into your database schema. They’re designed to be mostly automatic, but you’ll need to know when to make migrations, when to run them, and the common problems you might run into.

The Commands

There are several commands which you will use to interact with migrations and Django’s handling of database schema:

  • migrate, which is responsible for applying migrations, as well as unapplying and listing their status.
  • makemigrations, which is responsible for creating new migrations based on the changes you have made to your models.
  • sqlmigrate, which displays the SQL statements for a migration.
  • showmigrations, which lists a project’s migrations.

You should think of migrations as a version control system for your database schema. makemigrations is responsible for packaging up your model changes into individual migration files - analogous to commits - andmigrate is responsible for applying those to your database.

The migration files for each app live in a “migrations” directory inside of that app, and are designed to be committed to, and distributed as part of, its codebase. You should be making them once on your development machine and then running the same migrations on your colleagues’ machines, your staging machines, and eventually your production machines.

You can read and play with every single piece of it but here I will show you how to use data migrations to insert all US cities into our city project.

The first thing we need to do is find a csv file with the information we need. I did it sometime ago and found a txt file with the info. I put this file into my migrations folder inside cities app. You can see the whole project in github.

In order to read the csv we will use Python standard library and it beautiful concept of batteries included.

The name of the file is zbp13totals.txt so we are ready to start.

./manage.py makemigrations --empty cities

This command will create an empty migration in cities app. The name starts with a number that is good to keep so it is easy to follow the order. When I create a data migration I change the file name to something more meaningful because you may want in the future to delete all the other migrations and ask Django to recreate those migrations. The more migrations you have the slower the software will create new migrations or actually migrate the changes.

In my case now the name of the migrations was 

0003_auto_20160426_1141.py

So I will renamed it to: 0003_insert_cities.py

mv 0003_auto_20160426_1141.py 0003_insert_cities.py

I am creating this post using a Mac but I am pretty sure it will be easy for you to follow in any other operating system.

Let's go to the meat.

Here is the sample code Django has created.

# -*- coding: utf-8 -*-
# Generated by Django 1.9.5 on 2016-04-26 11:41
from __future__ import unicode_literals

from django.db import migrations


class Migration(migrations.Migration):

dependencies = [
('cities', '0002_auto_20160425_1119'),
]

operations = [
]

So as you can see it is only saying that this migrations depends on a migration called  0002_auto_20160425_1119 in cities app. Now we go to Django documentation or usually I go to my previous migration to get an example of how to create a data migration.

The example they have uses a function called combine_names

def combine_names(apps, schema_editor):
    # We can't import the Person model directly as it may be a newer
    # version than this migration expects. We use the historical version.
    Person = apps.get_model("yourappname", "Person")
    for person in Person.objects.all():
        person.name = "%s %s" % (person.first_name, person.last_name)
        person.save()

And they just appended a new operations into the operations list. 

operations = [
migrations.RunPython(combine_names),
]

 

We rewrite our function to insert_cities and the code will look like this:

# -*- coding: utf-8 -*-
# Generated by Django 1.9.5 on 2016-04-26 11:41
from __future__ import unicode_literals

import csv

from django.db import migrations


def insert_cities(apps, schema_editor):
# We can't import the Person model directly as it may be a newer
# version than this migration expects. We use the historical version.
City = apps.get_model("cities", "City")
path = 'citiesprj/apps/cities/migrations/'
with open(path + 'zbp13totals.txt') as csvfile:
reader = csv.DictReader(csvfile)
for row in reader:
if row['city']:
City.objects.create(city=row['city'], state=row['stabbr'], zip_code=row['zip'])



class Migration(migrations.Migration):

dependencies = [
('cities', '0002_auto_20160425_1119'),
]

operations = [
migrations.RunPython(insert_cities),
]

Ready now to run:

./manage.py migrate

And you can check using the admin or your beatiful API or Swagger documentations that your are ready to roll.

Current rating: 5