djangorecipe

Djangorecipe: easy install of Django with buildout

With djangorecipe you can manage your django site in a way that is familiar to buildout users. For example:

• bin/django to run django instead of bin/python manage.py.

• bin/test to run tests instead of bin/python manage.py test yourproject. (Including running coverage "around" your test).

• bin/django automatically uses the right django settings. So you can have a development.cfg buildout config and a production.cfg, each telling djangorecipe to use a different django settings module. bin/django will use the right setting automatically, no need to set an environment variable.

Djangorecipe is developed on github at https://github.com/rvanlaar/djangorecipe, you can submit bug reports there. It is tested with travis-ci and the code quality is checked via landscape.io:

.. image:: https://secure.travis-ci.org/rvanlaar/djangorecipe.png?branch=master :target: http://travis-ci.org/rvanlaar/djangorecipe/

.. image:: https://landscape.io/github/rvanlaar/djangorecipe/master/landscape.svg?style=flat :target: https://landscape.io/github/rvanlaar/djangorecipe/master :alt: Code Health

Setup

You can see an example of how to use the recipe below with some of the most common settings::

[buildout]
show-picked-versions = true
parts =
    django
eggs =
    yourproject
    gunicorn
develop = .
# ^^^ Assumption: the current directory is where you develop 'yourproject'.
versions = versions

[versions]
Django = 1.8.2
gunicorn = 19.3.0

[django]
recipe = djangorecipe
settings = development
eggs = ${buildout:eggs}
project = yourproject
test = yourproject
scripts-with-settings = gunicorn
# ^^^ This line generates a bin/gunicorn-with-settings script with
# the correct django environment settings variable already set.

Earlier versions of djangorecipe used to create a project structure for you, if you wanted it to. Django itself generates good project structures now. Just run bin/django startproject <projectname>. The main directory created is the one where you should place your buildout and probably a setup.py.

Startproject creates a manage.py script for you. You can remove it, as the bin/django script that djangorecipe creates is the (almost exact) replacement for it.

See django's documentation for startproject <https://docs.djangoproject.com/en/1.8/ref/django-admin/#django-admin-startproject>_.

You can also look at cookiecutter <https://cookiecutter.readthedocs.org/>_.

Supported options

The recipe supports the following options.

project This option sets the name for your project.

settings You can set the name of the settings file which is to be used with this option. This is useful if you want to have a different production setup from your development setup. It defaults to development.

test If you want a script in the bin folder to run all the tests for a specific set of apps this is the option you would use. Set this to the list of app labels which you want to be tested. Normally, it is recommended that you use this option and set it to your project's name.

scripts-with-settings Script names you add to here (like 'gunicorn') get a duplicate script created with '-with-settings' after it (so: bin/gunicorn-with-settings). They get the settings environment variable set. At the moment, it is mostly useful for gunicorn, which cannot be run from within the django process anymore. So the script must already be passed the correct settings environment variable.

Note: the package the script is in must be in the "eggs" option of your part. So if you use gunicorn, add it there (or add it as a dependency of your project).

eggs Like most buildout recipes, you can/must pass the eggs (=python packages) you want to be available here. Often you'll have a list in the [buildout] part and re-use it here by saying ${buildout:eggs}.

coverage If you set coverage = true, bin/test will start coverage recording before django starts. The coverage library must be importable. See the extra coverage notes further below.

The options below are for older projects or special cases mostly:

dotted-settings-path Use this option to specify a custom settings path to be used. By default, the project and settings option values are concatenated, so for instance myproject.development. dotted-settings-path = somewhere.else.production allows you to customize it.

extra-paths All paths specified here will be used to extend the default Python path for the bin/* scripts. Use this if you have code somewhere without a proper setup.py.

control-script The name of the script created in the bin folder. This script is the equivalent of the manage.py Django normally creates. By default it uses the name of the section (the part between the [ ]). Traditionally, the part is called [django].

initialization Specify some Python initialization code to be inserted into the control-script. This functionality is very limited. In particular, be aware that leading whitespace is stripped from the code given.

wsgi An extra script is generated in the bin folder when this is set to true. This is mostly only useful when deploying with apache's mod_wsgi. The name of the script is the same as the control script, but with .wsgi appended. So often it will be bin/django.wsgi.

wsgi-script Use this option if you need to overwrite the name of the script above.

deploy_script_extra In the wsgi deployment script, you sometimes need to wrap the application in a custom wrapper for some cloud providers. This setting allows extra content to be appended to the end of the wsgi script. For instance application = some_extra_wrapper(application). The limits described above for initialization also apply here.

testrunner This is the name of the testrunner which will be created. It defaults to test.

Coverage notes

Starting in django 1.7, you cannot use a custom test runner (like django-nose) anymore to automatically run your tests with coverage enabled. The new app initialization mechanism already loads your models.py, for instance, before the test runner gets called. So your models.py shows up as largely untested.

With coverage = true, bin/test starts coverage recording before django gets called. It also prints out a report and export xml results (for recording test results in Jenkins, for instance) and html results.

Behind the scenes, true is translated to a default of report xml_report html_report. These space-separated function names are called in turn on the coverage instance. See the coverage API docs <http://coverage.readthedocs.io/en/latest/api.html>_ for the available functions. If you only want a quick report and xml output, you can set coverage = report xml_report instead.

Note that you cannot pass options to these functions, like html output location. For that, add a .coveragerc next to your buildout.cfg. See the coverage configuration file docs <http://coverage.readthedocs.io/en/latest/config.html>_. Here is an example::

[run]
omit =
    */migrations/*
    *settings.py
source = your_app

[report]
show_missing = true

[html]
directory = htmlcov

[xml]
output = coverage.xml

Example configuration for mod_wsgi

If you want to deploy a project using mod_wsgi you could use this example as a starting point::

<Directory /path/to/buildout>
     Order deny,allow
     Allow from all
</Directory>
<VirtualHost 1.2.3.4:80>
     ServerName      my.rocking.server
     CustomLog       /var/log/apache2/my.rocking.server/access.log combined
     ErrorLog        /var/log/apache2/my.rocking.server/error.log
     WSGIScriptAlias / /path/to/buildout/bin/django.wsgi
</VirtualHost>

Corner case: there is a problem when several wsgi scripts are combined in a single virtual host instance of Apache. This is due to the fact that Django uses the environment variable DJANGO_SETTINGS_MODULE. This variable gets set once when the first wsgi script loads. The rest of the wsgi scripts will fail, because they need a different settings modules. However the environment variable DJANGO_SETTINGS_MODULE is only set once. The new initialization option that has been added to djangorecipe can be used to remedy this problem as shown below::

[django]
settings = acceptance
initialization =
    import os
    os.environ['DJANGO_SETTINGS_MODULE'] = '${django:project}.${django:settings}'

Generating a control script for PyDev

Running Django with auto-reload in PyDev requires adding a small snippet of code::

import pydevd pydevd.patch_django_autoreload(patch_remote_debugger=False, patch_show_console=True)

just before the if __name__ == "__main__": in the manage.py module (or in this case the control script, normally bin/django, that is generated). The following example buildout generates two control scripts: one for command-line usage and one for PyDev, with the required snippet, using the recipe's initialization option::

[buildout]
parts = django pydev
eggs =
    mock

[django]
recipe = djangorecipe
eggs = ${buildout:eggs}
project = dummyshop

[pydev]
<= django
initialization =
    import pydevd
    pydevd.patch_django_autoreload(patch_remote_debugger=False, patch_show_console=True)

Example usage of django-configurations

django-configurations (http://django-configurations.readthedocs.org/en/latest/) is an application that helps you organize your Django settings into classes. Using it requires modifying the manage.py file. This is done easily using the recipe's initialization option::

[buildout]
parts = django
eggs =
    hashlib

[django]
recipe = djangorecipe
eggs = ${buildout:eggs}
project = myproject
initialization =
    # Patch the manage file for django-configurations
    import os
    os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myproject.settings')
    os.environ.setdefault('DJANGO_CONFIGURATION', 'Development')
    from configurations.management import execute_from_command_line
    import django
    django.core.management.execute_from_command_line = execute_from_command_line

Changes

2.2.1 (2016-06-29)

• Bugfix for 2.2: bin/test was missing quotes around an option. [reinout]

2.2 (2016-06-29)

• Added optional coverage option. Set it to true to automatically run coverage around your django tests. Needed if you used to have a test runner like django-nose run your coverage automatically. Since django 1.7, this doesn't work anymore. With the new "coverage" option, bin/test does it for you. [reinout]

• Automated tests (travis-ci.org) test with django 1.4, 1.8 and 1.9 now. And pypi, python 2.7 and python 3.4. [reinout]

2.1.2 (2015-10-21)

• Fixed documentation bug: the readme mentioned script-with-settings instead of scripts-with-settings (note the missing s after script). The correct one is script-with-settings. [tzicatl]

2.1.1 (2015-06-15)

• Bugfix: script-entrypoints entry point finding now actually works.

2.1 (2015-06-15)

• Renamed script-entrypoints option to scripts-with-settings. It accepts script names that would otherwise get generated (like gunicorn) and generates a duplicate script named like bin/gunicorn-with-settings.

  Technical note: this depends on the scripts being setuptools "console_script entrypoint" scripts.

2.0 (2015-06-10)

• Removed project generation. Previously, djangorecipe would generate a directory for you from a template, but Django's own template is more than good enough now. Especially: it generates a subdirectory for your project now. Just run bin/django startproject <projectname>.

  See django's documentation for startproject <https://docs.djangoproject.com/en/1.8/ref/django-admin/#django-admin-startproject>_.

  You can also look at cookiecutter <https://cookiecutter.readthedocs.org/>_.

  This also means the projectegg option is now deprecated, it isn't needed anymore.

• We aim at django 1.7 and 1.8 now. Django 1.4 still works, (except that that one doesn't have a good startproject command).

• Gunicorn doesn't come with the django manage.py integration, so bin/django run_gunicorn doesn't work anymore. If you add script-entrypoints = gunicorn to the configuration, we generate a bin/django_env_gunicorn script that is identical to bin/gunicorn, only with the environment correctly set. Note: renamed in 2.1 to scripts-with-settings.

  This way, you can use the wsgi.py script in your project (copy it from the django docs if needed) with bin/django_env_gunicorn yourproject/wsgi.py just like suggested everywhere. This way you can adjust your wsgi file to your liking and run it with gunicorn.

  For other wsgi runners (or programs you want to use with the correct environment set), you can add a full entry point to script-entrypoints, like script-entrypoints = gunicorn=gunicorn.app.wsgiapp:run would be the full line for gunicorn. Look up the correct entrypoint in the relevant package's setup.py.

  Django's 1.8 wsgi.py file looks like this, see https://docs.djangoproject.com/en/1.8/howto/deployment/wsgi/::

  import os
  
  from django.core.wsgi import get_wsgi_application
  
  os.environ.setdefault("DJANGO_SETTINGS_MODULE", "yourproject.settings")
  
  application = get_wsgi_application()
  

• The wsgilog option has been deprecated, the old apache mod_wsgi script hasn't been used for a long time.

• Removed old pth option, previously used for pinax. Pinax uses proper python packages since a long time, so it isn't needed anymore.

1.11 (2014-11-21)

• The dotted-settings-path options was only used in management script. Now it is also used for the generated wsgi file and the test scripts.

1.10 (2014-06-16)

• Added dotted-settings-path option. Useful when you want to specify a custom settings path to be used by the manage.main() command.

• Renamed deploy_script_extra (with underscores) to deploy-script-extra (with dashes) for consistency with the other options. If the underscore version is found, an exception is raised.

1.9 (2014-05-27)

• bin/test now passes along command line arguments to the underlying management command. Previously, only the equivalent of manage.py test list_of_apps would be done. Now command line arguments are passed as-is after the list of apps.

• Added deploy_script_extra option. It is appended to the wsgi script. Useful for instance for a cloud hoster where you need to wrap your wsgi application object in a custom call.

1.8 (2014-05-27)

• Supporting buildout's relative-paths setting now.

1.7 (2013-12-11)

• Add option to change the wsgi script name. Thanks hedleyroos (Fixes pull #74)

1.6 (2013-10-28)

• Djangorecipe works with django 1.6 now.

• Tested with Django 1.4, 1.5 and 1.6. Pre-1.4 support is gone, now. Also tested on Python 2.6/2.7, 3.2/3.3.

• Moved to buildout 2 support only.

• Removed generation of fastcgi script. You can run it easily as bin/django runfcgi and it will be deprecated in Django 1.7 anyway.

1.5 (2013-01-25)

• Removed support for a different python version than the one you use to run buildout with. Previously, you could run your buildout with 2.6 but get Django to use 2.7 instead. zc.buildout 2.0 doesn't allow it anymore, so we removed it too.

1.4 (2013-01-15)

• Added initialization code support. Thanks to anshumanb, jjmurre. (Closes #58).

1.3 (2012-09-07)

• Removed deprecation warning in Django 1.4. Fixes #49, thanks Shagi.

• Added documentation for use with mr.developer. Thanks shagi (closes issue #45)

• Added Travis support.

1.2.1 (2012-05-15)

• Fixed broken 1.2 release (missing *.rst files due to a recent txt-to-rst rename action).

1.2 (2012-05-14)

• Removed location path from recipe. Thanks bleskes (fixes issue #50).

1.1.2

• Added correct url to the deprecation warning

1.1.1

• Fixed Python3 Trove classifiers

1.1

• Support python3.
• Changed buildout and the tests to run the tests under nose.
• Removed some old pre 0.99 unittests that dealt with download support.

1.0

• Stable release with a real 1.0 version.
• Made djangorecipe more pep08 compliant.

0.99

• Djangorecipe now depends on Django. The use of the version = statement is deprecated. Specify the django version in the [versions] section. Install django via mr.developer if you need to use an svn/git/hg repository. For other uses Versionpin djangorecipe to 0.23.1 if you don't want to upgrade. Thanks to Reinout van Rees for help with this release.

• Removed subversion and download support.

0.23.1

• Added a missing 'import os'

0.23

• Support for settings/urls boilerplate for django 1.2 and django 1.3. It defaults to 1.3 when the version isn't 1.2.

0.22

• Added support for svn urls with spaces. Thanks to Brad103 (fixes #537718).

• Updated code and buildout to use newest zc.recipe.egg, zc.recipe.testrunner and python-dateutil.

0.21

• The admin url is now configured for django 1.1 or higher. Thanks to Sam Charrington (fixes #672220).

• Bootstrap.py updated (fixes #501954).

0.20

• The recipe know makes the django package know to setuptools during install. This closes #397864. Thanks to Daniel Bruce and Dan Fairs for the patch.

• Fixed #451065 which fixes a problem with the WSGI log file option.

• Added the posibilty to configure more FCGI related settings. Thanks to Vasily Sulatskov for the patch.

0.19.2

• The generated WSGI & FCGI scripts are now properly removed when options change (fixes #328182). Thanks to Horst Gutmann for the patch.

• Scripts are now updated when dependencies change. This fixes #44658, thanks to Paul Carduner for the patch.

0.19.1

• Applied fix for the change in WSGI script generation. The previous release did not work properly.

0.19

• When running again with non-newest set the recipe will no longer update the Subversion checkout. Thanks to vinilios for the patch.

• The WSGI and FCGI scripts are now generated using Buildout's own system. This makes them more similar to the generated manage script with regard to the setup of paths. Thanks to Jannis Leidel for the patch.

0.18

• Paths from eggs and extra-paths now get precedence over the default system path (fixes #370420). Thanks to Horst Gutmann for the patch.

• The generated WSGI script now uses the python option if present. This fixes #361695.

0.17.4

• Fixed a problem when not running in verbose mode (fixes #375151).

0.17.3

• Removed dependency on setuptools_bzr since it does not seem to work like I expected.

0.17.2

• Changed the download code to use urllib2. This should make it work from behind proxies (fixes #362822). Thanks to pauld for the patch.

0.17.1

• Fixed a problem with the new WSGI logging option #348797. Thanks to Bertrand Mathieu for the patch.

• Disable generation of the WSGI log if "wsgilog" isn't set, thanks to Jacob Kaplan-Moss for the patch.

• Updated buildout.cfg and .bzrignore, thanks Jacob Kaplan-Moss.

0.17

• Added an option to specify a log file for output redirection from the WSGI script. Thanks to Guido Wesdorp for the patch.

0.16

• Subversion aliases are now supported (something like svn+mystuff://myjunk). Thanks to Remco for the patch.

0.15.2

• Update to move pth-files finder from the init method to the install method so it runs in buildout-order, else it looks for pth files in dirs that may not yet exist. Thanks to Chris Shenton for the update to his original patch.

0.15.1

• Update to make the previously added pth-files option better documented.

0.15

• Added "pth-files" option to add libraries to extra-paths from site .pth files. Thanks to Chris Shenton for the patch.

0.14

• The recipe now supports creating a FCGI script. Thanks to Jannis Leidel for the patch.

• When downloading a Django recipe for the first time the recipe now properly reports the url it is downloading from.

0.13

• Specifying a user name within a subversion url now works. The code that determined the revision has been updated. This fixes issue #274004. Thanks to Remco for the patch.

• Updated the template for creating new projects. It now uses the current admin system when generating it's urls.py file. This fixes issue #276255. Thanks to Roland for the patch.

0.12.1

• Re-upload since CHANGES.txt was missing from the release

0.12

• The recipe no longer executes subversion to determine whether the versions is to be downloaded using subversion. This fixes issue #271145. Thanks to Kapil Thangavelu for the patch.

• Changed the pythonpath option to extra-paths. This makes the recipe more consistent with other recipes (see issue #270908).

0.11

• Another go at fixing the updating problem (#250811) by making sure the update method is always called. It would not be called in the previous version since the recipe wrote a random secret (if it wasn't specified) to the options for use with a template. Buildout saw this as a change in options and therefore always decided to un-install & install.

• When both projectegg and wsgi=True are specified, the generated wsgi file did not have the correct settings file in it. This has been fixed with a patch from Dan Fairs.

• The recipe now has logging. All print statements have been replaced and a few extra logging calls have been added. This makes the recipe more informative about long running tasks. Thanks erny for the patch from issue #260628.

0.10

• The recipe no longer expects the top level directory name in a release tarball to be consistent with the version number. This fixes issue #260097. Thanks to erny for reporting this issue and suggesting a solution.

• Revision pinns for the svn checkout now stay pinned when re-running the buildout. This fixes issue #250811. Thanks to Remco for reporting this.

• Added an option to specify an egg to use as the project. This disables the code which creates the basic project structure. Thanks to Dan Fairs for the patch from issue #252647.

0.9.1

• Fixed the previous release which was broken due to a missing manifest file

0.9

• The settings option is fixed so that it supports arbitrary depth settings paths (example; conf.customer.development).

• The version argument now excepts a full svn url as well. You can use this to get a branch or fix any url to a specific revision with the standard svn @ syntax

• The wsgi script is no longer made executable and readable only by the user who ran buildout. This avoids problems with deployment.