Django deployment to AWS using Elastic Beanstalk

Today I wanted to finally deploy a project of mine (a non commercial one) to the cloud for staging purpose.
I had to choose between heroku (they are really cool) and AWS. At this point I won't explain why these were the only options nor why I made my choice like I did. However, as the title already implies I choose AWS and wanted to use Elastic Beanstalk (eb) as the matter of choice to solve the issue. Basically AWS docs are pretty rich and should explain it pretty straight forward, at least my guess.
During the deployment I faced some issues that were mainly related to the layout/setup of my project. All these
things were either directly based on or inspired by the best practices that are (thanks to @pydanny and @audreyr) official collected and released by the book two scoops of django

To summarize the things that were different to the common django project as the docs propagate it is and caused some troubles with eb:

DATABASE_URL environment variable ---

As already mentioned I use dj-database-url app for being able to setup the database as a url string. Nice approach that is maybe already known from other languages like java or .net. However eb does launch RDS servers on deployment time (if it is the first one) and the credentials and connection data are somehow randomized and served as environment variables

So my first idea was to use the .ebextensions/project.config file to provide a environment variable DATABASE_URL that is based off other environment variables. Something like:

option_settings:
  - option_name: DATABASE_URL
    value: mysql://$RDS_USERNAME:$RDS_PASSWORD@$RDS_HOSTNAME:$RDS_PORT/$RDS_DB_NAME

but u guess what happend. The $ is not a character for evaluating the content of the variables. It is simply just part of a value. After a long search journey I gave up and had to realize that is seems not to be possible to relay on environment variables at this config files (I did not ask the AWS guy for support with that). In the end I came with the AWS suggest way of reading the variables one by one at set it to the dictionary of DATABASES. But not without trapping in another fall.

At Configuration everything is a Value ---

The django-configurations app provides a convenient way of accessing values from the environment and hand them with some type safeness into the configuration. In addition it offers to use classes and inheritance for different environment configuration like Development inherits from Base and Production inherits from Base. So far so good. The issue is still about the Database configuration from above.

So I wrote the config by utilizing the Value classes that are intent to be used with django-configurations like:

# project_root/dj_project_root/config/settings.py
from configurations import Configuration, values

class Prod(Configuration):

    ########## DATABASE CONFIGURATION
    # See: https://docs.djangoproject.com/en/dev/ref/settings/#databases
    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.mysql',
            'NAME': values.Value(environ_name='DB_NAME', environ_prefix='RDS'),
            'USER': values.Value(environ_name='USERNAME', environ_prefix='RDS'),
            'PASSWORD': values.SecretValue(environ_name='PASSWORD', environ_prefix='RDS'),
            'HOST': values.Value(environ_name='HOSTNAME', environ_prefix='RDS'),
            'PORT': values.Value(environ_name='PORT', environ_prefix='RDS'),
        }
    }
    ########## END DATABASE CONFIGURATION

So each of the DATABASE values should be taken now from environment variables. In theory. Practically the django mysql client got the dictionary like this and was not able to work with a database NAME of type values.Value. Of course it was expecting a String. I was also expecting a lazy runtime evaluation of all Value instances but due to the fact that a Value can also be intitialized like this:

class FooConfig(Configuration):
    BAR = values.Value('foo bar')

Where the default behavior is to lookup a environment variable called 'DJANGO_BAR' with a default value of 'foo bar' if that environment variable is not set. So the name of the environment variables is not part of Value it is part of the FooConfig class. This implicitly means that each value is context depending and cannot be have a lazy loading mechanism just like that. So in the end the maintainer of django-configurations must decide weather that's a bug or a feature (request).

However finally it ended up with the solution that AWS already proposed:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': os.environ.get('RDS_DB_NAME'),
        'USER': os.environ.get('RDS_USERNAME'),
        'PASSWORD': os.environ.get('RDS_PASSWORD'),
        'HOST': os.environ.get('RDS_HOSTNAME'),
        'PORT': os.environ.get('RDS_PORT'),
    }
}

What a pity.

Apache WSGI and path issues

The WSGIPath that is used for the hosting setup can be configured with eb in .ebextensions/project.config this does
lead into the situation that apache is launched but during the inital loading of the WSGI application the django settings (see DJANGO_SETTINGS_MODULE) cannot be resolved because the django project root is not the project root.

To fix that the PYTHONPATH must be extended to contain the django project root. Then the settings can be found. The best place IMO to fix that is the wsgi.py

# project_root/dj_project_root/config/wsgi.py

import os
import sys

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'config.settings')
os.environ.setdefault('DJANGO_CONFIGURATION', 'Dev')

# this hack is because dj_project_root is not the top level of
# project directory, so elastic beanstalk cannot find config.settings for example
sys.path.append(os.path.dirname(os.path.dirname(__file__)))

from configurations.wsgi import get_wsgi_application
application = get_wsgi_application()

References

[1]Tutorial for AWS EB and Django Deployment http://grigory.ca/2012/09/getting-started-with-django-on-aws-elastic-beanstalk/
[2]Official Tutorial for AWS EB and Django Deployment http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_Python_django.html