In Chapter 2, we covered the fundamentals of building dynamic Web sites with Django: setting up views and URLconfs. As we explained, a view is responsible for doing some arbitrary logic, and then returning a response. In one of the examples, our arbitrary logic was to calculate the current date and time.
In modern Web applications, the arbitrary logic often involves interacting with a database. Behind the scenes, a database-driven Web site connects to a database server, retrieves some data out of it, and displays that data on a Web page. The site might also provide ways for site visitors to populate the database on their own.
Many complex Web sites provide some combination of the two. Amazon.com, for instance, is a great example of a database-driven site. Each product page is essentially a query into Amazon’s product database formatted as HTML, and when you post a customer review, it gets inserted into the database of reviews.
Django is well suited for making database-driven Web sites, because it comes with easy yet powerful tools for performing database queries using Python. This chapter explains that functionality: Django’s database layer.
The “Dumb” Way to Do Database Queries in Views
Just as Chapter 2 detailed a “dumb” way to produce output within a view (by hard-coding the text directly within the view), there’s a “dumb” way to retrieve data from a database in a view. It’s simple: just use any existing Python library to execute an SQL query and do something with the results. In this example view, we use the
MySQLdb library to connect to a MySQL database, retrieve some records, and feed them to a template for display as a Web page:
This approach works, but some problems should jump out at you immediately:
- We’re hard-coding the database connection parameters. Ideally, these parameters would be stored in the Django configuration.
- We’re having to write a fair bit of boilerplate code: creating a connection, creating a cursor, executing a statement, and closing the connection. Ideally, all we’d have to do is specify which results we wanted.
- It ties us to MySQL. If, down the road, we switch from MySQL to PostgreSQL, we’ll most likely have to rewrite a large amount of our code. Ideally, the database server we’re using would be abstracted, so that a database server change could be made in a single place. (This feature is particularly relevant if you’re building an open-source Django application that you want to be used by as many people as possible.)
As you might expect, Django’s database layer solves these problems.
Configuring the Database
With all of that philosophy in mind, let’s start exploring Django’s database layer. First, let’s explore the initial configuration that was added to
settings.py when we created the application:
The default setup is pretty simple. Here’s a rundown of each setting.
ENGINEtells Django which database engine to use. As we are using SQLite in the examples in this book, we will leave it to the default
NAMEtells Django the name of your database. For example:
Since we’re using SQLite,
startproject created a full filesystem path to the database file for us.
This is it for the default setup – you don’t need to change anything to run the code in this book, I have included this simply to give you an idea of how simple it is to configure databases in Django. For a detailed description on how to set up the various databases supported by Django, see Chapter 21.
Your First App
Now that you’ve verified the connection is working, it’s time to create a Django app – a bundle of Django code, including models and views, that live together in a single Python package and represent a full Django application. It’s worth explaining the terminology here, because this tends to trip up beginners. We’ve already created a project, in Chapter 1, so what’s the difference between a project and an app? The difference is that of configuration vs. code:
- A project is an instance of a certain set of Django apps, plus the configuration for those apps. Technically, the only requirement of a project is that it supplies a settings file, which defines the database connection information, the list of installed apps, the
DIRS, and so forth.
- An app is a portable set of Django functionality, usually including models and views, that live together in a single Python package. For example, Django comes with a number of apps, such as the automatic admin interface. A key thing to note about these apps is that they’re portable and reusable across multiple projects.
There are very few hard-and-fast rules about how you fit your Django code into this scheme. If you’re building a simple Web site, you may use only a single app. If you’re building a complex Web site with several unrelated pieces such as an e-commerce system and a message board, you’ll probably want to split those into separate apps so that you’ll be able to reuse them individually in the future.
Indeed, you don’t necessarily need to create apps at all, as evidenced by the example view functions we’ve created so far in this book. In those cases, we simply created a file called
views.py, filled it with view functions, and pointed our URLconf at those functions. No “apps” were needed.
However, there’s one requirement regarding the app convention: if you’re using Django’s database layer (models), you must create a Django app. Models must live within apps. Thus, in order to start writing our models, we’ll need to create a new app.
mysite project directory (this is the directory where your
manage.py file is, not the
mysite app directory), type this command to create a
This command does not produce any output, but it does create a
books directory within the
mysite directory. Let’s look at the contents of that directory:
These files will contain the models and views for this app. Have a look at
views.py in your favorite text editor. Both files are empty, except for comments and an import in
models.py. This is the blank slate for your Django app.
Defining Django Models in Python
As we discussed earlier in Chapter 1, the “M” in “MTV” stands for “Model.” A Django model is a description of the data in your database, represented as Python code. It’s your data layout – the equivalent of your SQL
CREATE TABLE statements – except it’s in Python instead of SQL, and it includes more than just database column definitions.
Django uses a model to execute SQL code behind the scenes and return convenient Python data structures representing the rows in your database tables. Django also uses models to represent higher-level concepts that SQL can’t necessarily handle.
If you’re familiar with databases, your immediate thought might be, “Isn’t it redundant to define data models in Python instead of in SQL?” Django works the way it does for several reasons:
- Introspection requires overhead and is imperfect. In order to provide convenient data-access APIs, Django needs to know the database layout somehow, and there are two ways of accomplishing this. The first way would be to explicitly describe the data in Python, and the second way would be to introspect the database at runtime to determine the data models.This second way seems cleaner, because the metadata about your tables lives in only one place, but it introduces a few problems. First, introspecting a database at runtime obviously requires overhead. If the framework had to introspect the database each time it processed a request, or even only when the Web server was initialized, this would incur an unacceptable level of overhead.(While some believe that level of overhead is acceptable, Django’s developers aim to trim as much framework overhead as possible.) Second, some databases, notably older versions of MySQL, do not store sufficient metadata for accurate and complete introspection.
- Writing Python is fun, and keeping everything in Python limits the number of times your brain has to do a “context switch.”
It helps productivity if you keep yourself in a single programming environment/mentality for as long as possible. Having to write SQL, then Python, and then SQL again is disruptive.
- Having data models stored as code rather than in your database makes it easier to keep your models under version control. This way, you can easily keep track of changes to your data layouts.
- SQL allows for only a certain level of metadata about a data layout. Most database systems, for example, do not provide a specialized data type for representing email addresses or URLs. Django models do. The advantage of higher-level data types is higher productivity and more reusable code.
- SQL is inconsistent across database platforms.
If you’re distributing a Web application, for example, it’s much more pragmatic to distribute a Python module that describes your data layout than separate sets of
CREATE TABLEstatements for MySQL,
PostgreSQL, and SQLite.
A drawback of this approach, however, is that it’s possible for the Python code to get out of sync with what’s actually in the database. If you make changes to a Django model, you’ll need to make the same changes inside your database to keep your database consistent with the model. I’ll show you how to handle this problem when we discuss migrations later in this chapter.
Finally, you should note that Django includes a utility that can generate models by introspecting an existing database. This is useful for quickly getting up and running with legacy data. We’ll cover this in Chapter 21.
Your First Model
As an ongoing example in this chapter and the next chapter, I’ll focus on a basic book/author/publisher data layout. I use this as our example because the conceptual relationships between books, authors, and publishers are well known, and this is a common data layout used in introductory SQL textbooks. You’re also reading a book that was written by authors and produced by a publisher!
I’ll suppose the following concepts, fields, and relationships:
- An author has a first name, a last name and an email address.
- A publisher has a name, a street address, a city, a state/province, a country, and a Web site.
- A book has a title and a publication date. It also has one or more authors (a many-to-many relationship with authors) and a single publisher (a one-to-many relationship – aka foreign key – to publishers).
The first step in using this database layout with Django is to express it as Python code. In the
models.py file that was created by the
startapp command, enter the following:
Let’s quickly examine this code to cover the basics. The first thing to notice is that each model is represented by a Python class that is a subclass of
django.db.models.Model. The parent class,
contains all the machinery necessary to make these objects capable of interacting with a database – and that leaves our models responsible solely for defining their fields, in a nice and compact syntax.
Believe it or not, this is all the code we need to write to have basic data access with Django. Each model generally corresponds to a single database table, and each attribute on a model generally corresponds to a column in that database table. The attribute name corresponds to the column’s name, and the type of field (e.g.,
CharField) corresponds to the database column type (e.g.,
varchar). For example, the
Publisher model is equivalent to the following table (assuming PostgreSQL
CREATE TABLE syntax):
Indeed, Django can generate that
CREATE TABLE statement automatically, as we’ll show you in a moment. The exception to the one-class-per-database-table rule is the case of many-to-many relationships. In our example models,
Book has a
authors. This designates that a book has one or many authors, but the
Book database table doesn’t get an
authors column. Rather, Django creates an additional table – a many-to-many “join table” – that handles the mapping of books to authors.
For a full list of field types and model syntax options, see Appendix B. Finally, note we haven’t explicitly defined a primary key in any of these models. Unless you instruct it otherwise, Django automatically gives every model an auto-incrementing integer primary key field called
id. Each Django model is required to have a single-column primary key.
Installing the Model
We’ve written the code; now let’s create the tables in our database. In order to do that, the first step is to activate these models in our Django project. We do that by adding the
books app to the list of “installed apps” in the settings file. Edit the
settings.py file again, and look for the
INSTALLED_APPS tells Django which apps are activated for a given project. By default, it looks something like this:
To register our “books” app, add
INSTALLED_APPS, so the setting ends up looking like this (
'books' refers to the “books” app we’re working on):
Each app in
INSTALLED_APPS is represented by its full Python path – that is, the path of packages, separated by dots, leading to the app package. Now that the Django app has been activated in the settings file, we can create the database tables in our database. First, let’s validate the models by running this command:
check command runs the Django system check framework – a set of static checks for validating Django projects. If all is well, you’ll see the message
System check identified no issues (0 silenced). If you don’t, make sure you typed in the model code correctly. The error output should give you helpful information about what was wrong with the code. Any time you think you have problems with your models, run
python manage.py check. It tends to catch all the common model problems.
If your models are valid, run the following command to tell Django that you have made some changes to your models (in this case, you have made a new one):
You should see something similar to the following:
Migrations are how Django stores changes to your models (and thus your database schema) – they’re just files on disk. In this instance, you will find a file names
0001_initial.py in the ‘migrations’ folder of the
books app. The
migrate command will take your latest migration file and update your database schema automatically, but first, let’s see what SQL that migration would run. The
sqlmigrate command takes migration names and returns their SQL:
python manage.py sqlmigrate books 0001
You should see something similar to the following (reformatted for readability):
Note the following:
- Table names are automatically generated by combining the name of the app (
books) and the lowercase name of the model (
author). You can override this behavior, as detailed in Appendix B.
- As we mentioned earlier, Django adds a primary key for each table automatically – the
idfields. You can override this. By convention, Django appends
"_id"to the foreign key field name. As you might have guessed, you can override this behavior, too.
- The foreign key relationship is made explicit by a
CREATE TABLE statements are tailored to the database you’re using, so database-specific field types such as
serial (PostgreSQL), or
integer primary key (SQLite) are handled for you automatically. The same goes for quoting of column names (e.g., using double quotes or single quotes). This example output is in PostgreSQL syntax.
sqlmigrate command doesn’t actually create the tables or otherwise touch your database – it just prints output to the screen so you can see what SQL Django would execute if you asked it. If you wanted to, you could copy and paste this SQL into your database client, however Django provides an easier way of committing the SQL to the database: the
Run that command, and you’ll see something like this:
In case you were wondering what all the extras are (commented out above), the first time you run migrate, Django will also create all the system tables that Django needs for the inbuilt apps.
Migrations are Django’s way of propagating changes you make to your models (adding a field, deleting a model, etc.) into your database schema. They’re designed to be mostly automatic, however there are some caveats. For more information on migrations, see Chapter 21.