Django Models

In Chapter 2, we covered the fundamentals of building dynamic web sites with Django: setting up views and URLconfs. As I explained, a view is responsible for doing some arbitrary logic, and then returning a response. In one of the examples, our arbitrary logic was to calculate the current date and time.

In modern web applications, the arbitrary logic often involves interacting with a database. Behind the scenes, a database-driven web site connects to a database server, retrieves some data out of it, and displays that data on a web page. The site might also provide ways for site visitors to populate the database on their own.

Many complex web sites provide some combination of the two. Amazon.com, for instance, is a great example of a database-driven site. Each product page is essentially a query into Amazon’s product database formatted as HTML, and when you post a customer review, it gets inserted into the database of reviews.

Django is well suited for making database-driven web sites, because it comes with easy yet powerful tools for performing database queries using Python. This chapter explains that functionality: Django’s database layer.

The “Dumb” Way to Do Database Queries in Views

Just as Chapter 2 detailed a “dumb” way to produce output within a view (by hard-coding the text directly within the view), there’s a “dumb” way to retrieve data from a database in a view. It’s simple: just use any existing Python library to execute an SQL query and do something with the results. In this example view, we use the MySQLdb library to connect to a MySQL database, retrieve some records, and feed them to a template for display as a web page:

from django.shortcuts import render
import MySQLdb

def book_list(request):
    db = MySQLdb.connect(user='me', db='mydb',  passwd='secret', host='localhost')
    cursor = db.cursor()
    cursor.execute('SELECT name FROM books ORDER BY name')
    names = [row[0] for row in cursor.fetchall()]
    db.close()
    return render(request, 'book_list.html', {'names': names})

This approach works, but some problems should jump out at you immediately:

  • We’re hard-coding the database connection parameters. Ideally, these parameters would be stored in the Django configuration.
  • We’re having to write a fair bit of boilerplate code: creating a connection, creating a cursor, executing a statement, and closing the connection. Ideally, all we’d have to do is specify which results we wanted.
  • It ties us to MySQL. If, down the road, we switch from MySQL to PostgreSQL, we’ll most likely have to rewrite a large amount of our code. Ideally, the database server we’re using would be abstracted, so that a database server change could be made in a single place. (This feature is particularly relevant if you’re building an open-source Django application that you want to be used by as many people as possible.)

As you might expect, Django’s database layer solves these problems.

Configuring the Database

With all of that philosophy in mind, let’s start exploring Django’s database layer. First, let’s explore the initial configuration that was added to settings.py when we created the application:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    }
}

The default setup is pretty simple. Here’s a rundown of each setting:

  • ENGINE tells Django which database engine to use. As we are using SQLite in the examples in this book, we will leave it to the default django.db.backends.sqlite3.
  • NAME tells Django the name of your database. For example: 'NAME': 'mydb',

Since we’re using SQLite, startproject created a full filesystem path to the database file for us.

This is it for the default setup – you don’t need to change anything to run the code in this book, I have included this simply to give you an idea of how simple it is to configure databases in Django. For a detailed description on how to set up the various databases supported by Django, see Chapter 21.

Your First App

Now that you’ve verified the connection is working, it’s time to create a Django app – a bundle of Django code, including models and views, that live together in a single Python package and represent a full Django application. It’s worth explaining the terminology here, because this tends to trip up beginners. We’ve already created a project, in Chapter 1, so what’s the difference between a project and an app? The difference is that of configuration vs. code:

  • A project is an instance of a certain set of Django apps, plus the configuration for those apps. Technically, the only requirement of a project is that it supplies a settings file, which defines the database connection information, the list of installed apps, the DIRS, and so forth.
  • An app is a portable set of Django functionality, usually including models and views, that live together in a single Python package. For example, Django comes with a number of apps, such as the automatic admin interface. A key thing to note about these apps is that they’re portable and reusable across multiple projects.

There are very few hard-and-fast rules about how you fit your Django code into this scheme. If you’re building a simple web site, you may use only a single app. If you’re building a complex web site with several unrelated pieces such as an e-commerce system and a message board, you’ll probably want to split those into separate apps so that you’ll be able to reuse them individually in the future.

Indeed, you don’t necessarily need to create apps at all, as evidenced by the example view functions we’ve created so far in this book. In those cases, we simply created a file called views.py, filled it with view functions, and pointed our URLconf at those functions. No apps were needed.

However, there’s one requirement regarding the app convention: if you’re using Django’s database layer (models), you must create a Django app. Models must live within apps. Thus, in order to start writing our models, we’ll need to create a new app.

Within the mysite project directory (this is the directory where your manage.py file is, not the mysite app directory), type this command to create a books app:

python manage.py startapp books 

This command does not produce any output, but it does create a books directory within the mysite directory. Let’s look at the contents of that directory (showing the project folder tree so you can see where the app should be located):

\mysite_project
    \books
        \migrations
        __init__.py
        admin.py
        apps.py
        models.py
        tests.py
        views.py
    \mysite
    \templates
    manage.py

These files will contain the models and views for this app. Have a look at models.py and views.py in your favorite text editor. Both files are empty, except for comments and an import in models.py. This is the blank slate for your Django app.

Defining Django Models in Python

As we discussed earlier in Chapter 1, the “M” in “MTV” stands for “Model.” A Django model is a description of the data in your database, represented as Python code. It’s your data layout – the equivalent of your SQL CREATE TABLE statements – except it’s in Python instead of SQL, and it includes more than just database column definitions.

Django uses a model to execute SQL code behind the scenes and return convenient Python data structures representing the rows in your database tables. Django also uses models to represent higher-level concepts that SQL can’t necessarily handle.

If you’re familiar with databases, your immediate thought might be, “Isn’t it redundant to define data models in Python instead of in SQL?” Django works the way it does for several reasons:

  • Introspection requires overhead and is imperfect. In order to provide convenient data-access APIs, Django needs to know the database layout somehow, and there are two ways of accomplishing this. The first way would be to explicitly describe the data in Python, and the second way would be to introspect the database at runtime to determine the data models. As introspection adds an unacceptable level of overhead and can be inaccurate, Django’s developers decided the first option was the best.
  • Writing Python is fun, and keeping everything in Python limits the number of times your brain has to do a context switch. It helps productivity if you keep yourself in a single programming environment/mentality for as long as possible. Having to write SQL, then Python, and then SQL again is disruptive.
  • Having data models stored as code rather than in your database makes it easier to keep your models under version control. This way, you can easily keep track of changes to your data layouts.
  • SQL allows for only a certain level of metadata about a data layout. Most database systems, for example, do not provide a specialized data type for representing email addresses or URLs. Django models do. The advantage of higher-level data types is higher productivity and more reusable code.
  • SQL is inconsistent across database platforms. If you’re distributing a web application, for example, it’s much more pragmatic to distribute a Python module that describes your data layout than separate sets of CREATE TABLE statements for MySQL, PostgreSQL, and SQLite.

A drawback of this approach, however, is that it’s possible for the Python code to get out of sync with what’s actually in the database. If you make changes to a Django model, you’ll need to make the same changes inside your database to keep your database consistent with the model. I’ll show you how to handle this problem when we discuss migrations later in this chapter.

Finally, you should note that Django includes a utility that can generate models by introspecting an existing database. This is useful for quickly getting up and running with legacy data. I’ll cover this in Chapter 21.

Your First Model

As an ongoing example in this chapter and the next chapter, I’ll focus on a basic book/author/publisher data layout. I use this as our example because the conceptual relationships between books, authors, and publishers are well known, and this is a common data layout used in introductory SQL textbooks. You’re also reading a book that was written by an author and produced by a publisher!

I’ll suppose the following concepts, fields, and relationships:

  • An author has a first name, a last name and an email address.
  • A publisher has a name, a street address, a city, a state/province, a country, and a web site.
  • A book has a title and a publication date. It also has one or more authors (a many-to-many relationship with authors) and a single publisher (a one-to-many relationship – aka foreign key – to publishers).

The first step in using this database layout with Django is to express it as Python code. In the models.py file that was created by the startapp command, enter the following:

from django.db import models

class Publisher(models.Model):
    name = models.CharField(max_length=30)
    address = models.CharField(max_length=50)
    city = models.CharField(max_length=60)
    state_province = models.CharField(max_length=30)
    country = models.CharField(max_length=50)
    website = models.URLField()

class Author(models.Model):
    first_name = models.CharField(max_length=30)
    last_name = models.CharField(max_length=40)
    email = models.EmailField()

class Book(models.Model):
    title = models.CharField(max_length=100)
    authors = models.ManyToManyField(Author)
    publisher = models.ForeignKey(Publisher)
    publication_date = models.DateField()

Let’s quickly examine this code to cover the basics. The first thing to notice is that each model is represented by a Python class that is a subclass of django.db.models.Model. The parent class, Model, contains all the machinery necessary to make these objects capable of interacting with a database – and that leaves our models responsible solely for defining their fields, in a nice and compact syntax.

Believe it or not, this is all the code we need to write to have basic data access with Django. Each model generally corresponds to a single database table, and each attribute on a model generally corresponds to a column in that database table. The attribute name corresponds to the column’s name, and the type of field (e.g., CharField) corresponds to the database column type (e.g., varchar). For example, the Publisher model is equivalent to the following table (assuming PostgreSQL CREATE TABLE syntax):

CREATE TABLE "books_publisher" (
    "id" serial NOT NULL PRIMARY KEY,
    "name" varchar(30) NOT NULL,
    "address" varchar(50) NOT NULL,
    "city" varchar(60) NOT NULL,
    "state_province" varchar(30) NOT NULL,
    "country" varchar(50) NOT NULL,
    "website" varchar(200) NOT NULL
);

Indeed, Django can generate that CREATE TABLE statement automatically, as I’ll show you in a moment. The exception to the one-class-per-database-table rule is the case of many-to-many relationships. In our example models, Book has a ManyToManyField called authors. This designates that a book has one or many authors, but the Book database table doesn’t get an authors column. Rather, Django creates an additional table – a many-to-many join table – that handles the mapping of books to authors.

For a full list of field types and model syntax options, see Appendix B. Finally, note we haven’t explicitly defined a primary key in any of these models. Unless you instruct it otherwise, Django automatically gives every model an auto-incrementing integer primary key field called id. Each Django model is required to have a single-column primary key.

Installing the Model

We’ve written the code; now let’s create the tables in our database. In order to do that, the first step is to activate these models in our Django project. We do that by adding the books app to the list of installed apps in the settings file. Edit the settings.py file again, and look for the INSTALLED_APPS setting. INSTALLED_APPS tells Django which apps are activated for a given project. By default, it looks something like this:

INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
]

To register our “books” app, add 'books.apps.BooksConfig' to INSTALLED_APPS, so the setting ends up looking like this:

INSTALLED_APPS = [
'books.apps.BooksConfig',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
]

Each app in INSTALLED_APPS is represented by its full Python path – that is, the path of packages, separated by dots, leading to the app package. The dotted path in this case points to the BooksConfig class that Django created for you in the apps.py file. Well will look at app configurations in more detail later in the book.

Now that the Django app has been activated in the settings file, we can create the database tables in our database. First, let’s validate the models by running this command:

python manage.py check 

The check command runs the Django system check framework – a set of static checks for validating Django projects. If all is well, you’ll see the message System check identified no issues (0 silenced). If you don’t, make sure you typed in the model code correctly. The error output should give you helpful information about what was wrong with the code. Any time you think you have problems with your models, run python manage.py check. It tends to catch all the common model problems.

If your models are valid, run the following command to tell Django that you have made some changes to your models (in this case, you have made a new one):

python manage.py makemigrations books 

You should see something similar to the following:

Migrations for 'books':
  0001_initial.py:
    - Create model Author
    - Create model Book
    - Create model Publisher
    - Add field publisher to book 

Migrations are how Django stores changes to your models (and thus your database schema) – they’re just files on disk. In this instance, you will find a file names 0001_initial.py in the \migrations folder of the books app. The migrate command will take your latest migration file and update your database schema automatically, but first, let’s see what SQL that migration would run. The sqlmigrate command takes migration names and returns their SQL:

python manage.py sqlmigrate books 0001

You should see something similar to the following (reformatted for readability):

BEGIN;

CREATE TABLE "books_author" (
    "id" integer NOT NULL PRIMARY KEY AUTOINCREMENT,
    "first_name" varchar(30) NOT NULL,
    "last_name" varchar(40) NOT NULL,
    "email" varchar(254) NOT NULL
);
CREATE TABLE "books_book" (
    "id" integer NOT NULL PRIMARY KEY AUTOINCREMENT,
    "title" varchar(100) NOT NULL,
    "publication_date" date NOT NULL
);
CREATE TABLE "books_book_authors" (
    "id" integer NOT NULL PRIMARY KEY AUTOINCREMENT,
    "book_id" integer NOT NULL REFERENCES "books_book" ("id"),
    "author_id" integer NOT NULL REFERENCES "books_author" ("id"),
    UNIQUE ("book_id", "author_id")
);
CREATE TABLE "books_publisher" (
    "id" integer NOT NULL PRIMARY KEY AUTOINCREMENT,
    "name" varchar(30) NOT NULL,
    "address" varchar(50) NOT NULL,
    "city" varchar(60) NOT NULL,
    "state_province" varchar(30) NOT NULL,
    "country" varchar(50) NOT NULL,
    "website" varchar(200) NOT NULL
);
CREATE TABLE "books_book__new" (
    "id" integer NOT NULL PRIMARY KEY AUTOINCREMENT,
    "title" varchar(100) NOT NULL,
    "publication_date" date NOT NULL,
    "publisher_id" integer NOT NULL REFERENCES
    "books_publisher" ("id")
);

INSERT INTO "books_book__new" ("id", "publisher_id", "title",
"publication_date") SELECT "id", NULL, "title", "publication_date" FROM
"books_book";

DROP TABLE "books_book";

ALTER TABLE "books_book__new" RENAME TO "books_book";

CREATE INDEX "books_book_2604cbea" ON "books_book" ("publisher_id");

COMMIT;

Note the following:

  • Table names are automatically generated by combining the name of the app (books) and the lowercase name of the model (publisher, book, and author). You can override this behavior, as detailed in Appendix B.
  • As we mentioned earlier, Django adds a primary key for each table automatically – the id fields. You can override this. By convention, Django appends "_id" to the foreign key field name. As you might have guessed, you can override this behavior, too.
  • The foreign key relationship is made explicit by a REFERENCES statement.

These CREATE TABLE statements are tailored to the database you’re using, so database-specific field types such as auto_increment (MySQL), serial (PostgreSQL), or integer primary key (SQLite) are handled for you automatically. The same goes for quoting of column names (e.g., using double quotes or single quotes). This example output is in PostgreSQL syntax.

The sqlmigrate command doesn’t actually create the tables or otherwise touch your database – it just prints output to the screen so you can see what SQL Django would execute if you asked it. If you wanted to, you could copy and paste this SQL into your database client, however Django provides an easier way of committing the SQL to the database: the migrate command:

python manage.py migrate 

Run that command, and you’ll see something like this:

Operations to perform:
Apply all migrations: admin, auth, books, contenttypes, sessions
Running migrations:
Applying books.0001_initial... OK

In case you get a bunch of extra migrations, this is most likely because you forgot to run the initial migrations in Chapter 1. The first time you run migrate, Django will also create all the system tables that Django needs for the inbuilt apps.

Migrations are Django’s way of propagating changes you make to your models (adding a field, deleting a model, etc.) into your database schema. They’re designed to be mostly automatic, however there are some caveats. For more information on migrations, see Chapter 21.

<<< Templates in Views | Table of Contents | Django Models: Basic Data Access >>>

Join the Djangobook Family

Updates, freebies, discounts and more!

No spam. Like, ever. Unsubscribe at any time. Powered by ConvertKit