Django Tip: Developing Without Projects

Fri, 9 Nov 2007, 22:03 +1100

As developers progress along the path from Django grasshopper to Django master and beyond, it slowly (or rapidly) dawns on them that Django's project concept is something that really only exists as a convenient kickstart mechanism. It's a great way to help you get your initial work started and arrive at the point where you can start writing code.

The usefulness of the project concept is reinforced by the fact that django-admin.py startproject exists and does all this work for you. Surely, we are meant to use projects all the time.

No! A project is certainly a convenient "getting started" concept. It's also a handy development aid (the meaning of which, I'll describe later). However, the core of Django-based software is the application, not the project. When you distribute Django software to other people, you are generally distributing one or more applications.

So, here are a few tips and tricks for using a collection of Django applications without needing to create a Django project.

Why Use Projects At All?

As I've already indicated and without intending to be patronising at all, projects are a great way to get started with Django. For your first few experiments with Django, run django-admin.py startproject and go to town. There is nothing wrong with this. You might want to keep the remainder of this article in mind, though, so that you aren't locking yourself into the project structure later on.

The second area where the project concept is useful is for testing and initial development work. I still often start up a quick project to try out something or get the groundwork laid for a larger project (I did this yesterday when I was starting work on something that's probably going to be a large number of applications by the time I'm finished). It creates a nice initial directory hierarchy and makes sure you don't forget anything in the rush to get going. So don't look down your nose at startproject even if you're a fan of writing purely standalone applications. It can save time for quick experiments.

If There Is No Project, What Am I Giving Up?

Letting the cat out of the bag right now: if you have a collection of applications you want to use, the only extra bits you need are a settings file and a root URL configuration file. You might also need a directory or two for storing templates and maybe some static data (CSS files, Javascript files, etc). But Django doesn't need those directories.

If you're version controlling a whole bunch of related applications (not every application is intended to be reusable in isolation, after all), it might be worth putting them all under a single parent directory. That's what I do; it means there's only one spot that needs a version control update when I'm synchronising various machines. Again, though, that's a matter of personal taste and code organisation preferences, rather than a requirements.

How Do I Set All This Up Without Projects To Guide Me?

One benefit of working with applications, rather than projects, is it actually leads to cleaner installation requirements.

  1. Write your code so that you are importing applications as though they live on the Python import path. This means if you have an application called newsletter, write import newsletter or from newsletter import ... in code that uses the newsletter application.
    Do not import things using the project directory's name. If you write from myproject import newsletter anywhere, you cannot move the newsletter application out of the myproject project directory. Even if you use projects, avoid mentioning the project's name in your code.

  2. As a corollary to the previous point, install your applications somewhere that is on your system's Python path. Usually this works in reverse. You install your applications somewhere (say, in /usr/local/django-apps/) and then configure your Python path to include that directory.

  3. Create your settings file and root URL configuration file (described below). Both of these files are also going to live somewhere on your Python path, since they are imported by lots of different pieces of code.

After you complete steps 2 and 3, if you put everything in the same directory, it's going to look a lot like a project directory. That's fine. Again, there's nothing wrong with the project concept per se. The problems start happening when you think of the project directory as the driver for the setup, rather than just a directory that holds all your bits and pieces.

It's entirely up to you whether you want to put your applications in a common directory that contains all the Django applications you are using, or whether you want to put them under a directory specifically for this installation that will also contain the settings and URL files.

I tend to use a mix of both systems at the moment. Third-party applications and things I've written to be reusable live in a common location. Applications that are mostly for a single software project (as opposed to a Django project), will be installed under a directory all together, along with the settings and URL files.

The advantage of thinking this way (steps 1, 2, 3), is that it makes configuring your webserver a lot easier. At some point in the installation process, you need to tell your webserver how to set up Python's import path — say, using the PythonPath directive for mod_python or the WSGIPythonPath directive for mod_wsgi. You set these to include all the directories containing the applications, settings and URL files you need. There might be multiple such directories, but it's easy to work out what they are. People using projects all the time often run into trouble setting their Python paths because they end up needing both the project directory and the parent of the project directory, but might not realise that (despite it being clearly documented), because testing with manage.py runserver set up that path for them.

URL configuration file

Your root URL configuration file — traditionally called urls.py in Django, but it can be called anything — should be written just as you normally would. Again, import applications directly (don't use projectname.appname). Keep the project name out of things. This URL file is going to be imported by your settings file, so it needs to be somewhere the settings file can import. Unless you have some good reasons, I would keep it right alongside the settings file.

Settings file

This is where most people hesitate, I would guess, when they try to work only with applications. The default settings file that startproject creates looks like a very complicated animal. How are you to be expected to get all that right?

The secret is, you don't need to. There are very few settings that are actually required to be set by you, the developer. Most of the time, the defaults are fine.

Starting from a completely blank file, here are the settings I would recommend starting from:

  • Set the DATABASE_ENGINE and DATABASE_NAME and, if required, the other database settings.

  • Nobody really lives in Chicago, so the default timezone probably isn't going to suit you. You'll need to set TIME_ZONE for your own region.

  • Create a SECRET_KEY for yourself and set this. There is no default here and failing to set this config option will cause things to break. You can generate the key the same way Django does. Run the following at a Python prompt:

    >>> from random import choice
    >>> ''.join([choice('abcdefghijklmnopqrstuvwxyz0123456789!@#$%^&*(-_=+)')
     ... for i in range(50)])
    

    Copy and paste the string that is displayed as your SECRET_KEY value.

  • Set ROOT_URLCONF to be the string that is imported to get your URL configuration file. If the URL config file is in the same directory as your settings file, this will just be the name of the URL file, without the .py extension.

  • INSTALLED_APPS will need to be populated. By default, it's empty. At a minimum, you probably need

    • 'django.contrib.auth'
    • 'django.contrib.contenttypes'
    • 'django.contrib.sessions'
    • 'django.contrib.admin'

    But even that list could be trimmed down if you didn't need, for example, the admin application, or weren't interested in using sessions. Don't forget to add your own applications to the list, too.

  • You might want to look at what MIDDLEWARE_CLASSES you need. The default settings aren't bad, although I can usually live without the XViewMiddleware.

  • Set up ADMINS, DEFAULT_FROM_EMAIL, SERVER_EMAIL, EMAIL_HOST (and possibly other email settings) and change EMAIL_SUBJECT_PREFIX to something that matches your website (seeing email with a prefix of [Django] gets confusing if you've got more than one site.

That's all you need! Optionally, you might also want to consider:

  • If you're writing for a non-English locale by default, set LANGUAGE_CODE to your default. This is the locale that everybody without a preference will see.
  • If you don't need the internationalisation support at all (your site only supports English), set USE_I18N = False for slightly better performance.
  • During development, it's obviously worth settings DEBUG=True and TEMPLATE_DEBUG=DEBUG, or something similar.
  • If your content will benefit from downstream HTTP caches storing it on your behalf at times, set USE_ETAGS=True.

Maybe a dozen settings all up for many simple cases. A lot of this stuff is common between all the sites on a particular machine, too (for example, the email configuration). So you might choose to put the common stuff into a site_settings.py file and then have from site_settings import * at the top of your settings file.

Naming Things

Keep in mind that your settings file does not have to be called settings.py and your URL configuration does not have to be called urls.py. Using descriptive names will make debugging installation problems easier and let you have files for multiple sites in the same directory, if you want to work that way.

Living Without manage.py

This is why I choose to use Django's project concept when I start on a new project. Until I've worked out how I'm going to lay things out, which always takes me a few attempts, being able to experiment with manage.py ... without worrying about Python paths or setting file imports or anything is a bonus. However, manage.py is an aid, not a crutch. All it does is import your settings file and then acts as a proxy for django-admin.py. You can do everything by calling django-admin.py directly.

Depending on your development environment and particular skills, there may not be anything that's quite as simple as manage.py. Fortunately, though, you can write your own. Or copy one from another directory. Replace the name of the file it imports at the top to match your settings file's name (if you haven't called it settings.py).

In my environment, being a comfortable Linux user and quite happy in the shell, I've got a collection of little shell scripts in $HOME/bin/ that look like this:

#!/bin/bash

export PYTHONPATH=/home/malcolm/Programming/website/
export DJANGO_SETTINGS_MODULE=website_settings
django-admin.py $@

If I save this as dj-website.sh, then I can run things like dj-website.sh runserver or dj-website.sh runtests.

For other projects where I don't work on them very often, I just set up my Python path correctly and do things like django-admin.py runserver --settings=project_settings.

Conclusions

Hopefully this has given you some ideas about how to write project-less collections of applications.

I would like to repeat what I said earlier, though: using the project support in Django isn't a bad thing when you're getting started or trying to do something quickly. Even so, write your code without referring to the project directory name as much as you can (which should be always). It will then be much easier to install things on a remote system or distribute applications in isolation, because you won't need overly complicated Python path configurations.

Also, don't be afraid to experiment. All of the things I'm writing about here work for me. I've only picked them up over time, though. Even 15 months ago I wans't working exactly like this. As I've written more code that I want to reuse or where I've had to write installation instructions for clients, though, it's become clear that I needed to be a lot more regimented in how I set things up. It also allows me to be a lot leaner on the configuration side as a bonus; my settings files are half the size or less, compared to what Django automatically generates. A bit of research, a bit of reading the documentation (or, sometimes, writing the documentation) and everything became fairly easy.

Good luck with your own experiments and hopefully the application/project split is now a little clearer.


Posted in:

  • software/django/tips