Google App Engine and zc.buildout

Setting up a project is like laying out the fundamentals for the house you are going to build. It is just that important. Right project structure can save you from unneeded headaches (have no worries, you will have enough of those anyway :) ).

There is an excellent guide on how to set up a Django project with buildout here.

But I didn't find any such guide on how to set up a GAE project, and after struggling with it for a few hours, this is how I made it work.

First things first - directory structure

project/
   bootstrap.py  †
   buildout.cfg  †
   .installed.cfg
   README
   setup.py
   parts/
   develop-eggs/
   bin/
       buildout
   eggs/
   downloads/
   src/

† put this items under version control

For more info about each part you see read here.

So, first things first, right? I start by creating the project/ directory, with src/ directory under it. src/ is where all of the code resides. I will have another post about how to set up a GAE application, those reside in src/ directory.

$ cd path/to/project/
$ wget http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py
$ touch setup.py
$ touch buildout.cfg
$ cat  -> buildout.cfg
[buildout]
parts = 
^D

After this we will have a bootstrap.py, buildout.cfg and setup.py alongside src/ directory.

And here we go..

Next thing, let's get us a buildout directory structure:

$ python bootstrap.py

This will create all the other directories (aside form downloads/ which is created later) you can see in the project/ structure.

In next step we need a buildout.cfg and a proper setup.py, so let's create those now.

A word about zc.buildout. This tool allows you to create an isolated development environment, much like virtualenv. In that, you can install any needed python, and it will be installed for this project only. Same goes for other packages.

Open a buildout.cfg in an editor of your choice and add following:

[buildout]
parts = mygaeapp
develop = .

[mygaeapp]
recipe = rod.recipe.appengine
url = http://googleappengine.googlecode.com/files/google_appengine_1.7.2.zip
server-script = dev_appserver
src = ${buildout:directory}/src/mygaeapp
exclude = tests

In my case I use system python. If you want to use a local version of python you can add  it using various recipes on PyPi.

Next let's open setup.py and modify it:

from setuptools import setup, find_packages

install_requires = [
                    'setuptools',
                    'nose',
                    ]
setup(
    name='project',
    version='1.0',
    packages=find_packages('src'),
    package_dir = {'': 'src'},
    install_requires = install_requires,
    url='github link for example',
    license='LISENCE',
    author='Your Name',
    author_email='your@email.com',
    description='Describe your project here or link a readme file'
)

Next thing, run:

$ ./bin/buildout

This will download google app engine from the link you specified in buildout.cfg, install your packages from src/ into parts/ folder, as well as any other dependencies you specified in setup.py and buildout.cfg. This will also add a dev_appserver and appcf.py to bin/ that you need to run the server and upload your application to google app engine.

Attention!

I found out that it's better not to use a global zc.buildout. It caused me a lot of silly errors, and hours of headaches.

Summary

zc.buildout is a nice tools to distribute your code among other collaborators. This way you can give a simple setup and ensure a quick and easy entrance for other developers into your project.