Deploying Django on Nginx using Gunicorn, Postgres and Supervisor

So I finally built my blog on Django. I've been working on this for quite a while. My code is on a private repository on Bitbucket right now because I have my production settings file in the open. Once I clean that up I'll make this repository public.

So this time I am using Nginx instead of Apache as a web server. They both have their merits but I decided to go with Nginx because to me it seems much more easier to configure than the latter. I recently acquired a VPS on DigitalOcean and it's been great. It's cheap, fast and they have really good documentation. I am also using Gunicorn instead of mod_wsgi because I've had some really bad experiences with mod_wsgi on CentOS when I was working with IIT a year back. Gunicorn is directly integrated within Django so that makes things easier. Anyway, so this is how I did it.

Installing Postgres and configuration

So instead of using sqlite3 I am going to be using Postgres because of South compatibility. Since this is a low traffic website. I am hosting the DB Server on the same server instance. I simple followed the guide given on DigitalOcean's page. Here are the few steps:

#installing postgres on ubuntu server. It's different for every distro (obviously)
$ sudo apt-get install postgresql
#Then simply create the database
$ sudo su - postgres
$ createdb proddb
$ createuser -P
#you'll have to add the user information after this. Takes about 5 steps
#now switch over to psql and grant permissions
#my username is produser btw. 
$ psql
$ GRANT ALL PRIVILEGES ON DATABASE proddb TO produser;
There. Done, now include these values in the Django settings file.

Installing and configuring Gunicorn

So this one's pretty easy too. Django has a page dedicated to this part. Well it has the basic commands to get it running directly as a WSGI application. All I did was create a bash file that would run the command with the given parameters.

Lets try running the application first

#activate your virtual environment
$ source ../venv/bin/activate
#install gunicorn
$ pip install gunicorn
#run this command where you have manage.py so that PYTHON_PATH doesn't give you trouble
$ gunicorn appname.wsgi:application --bind your_ip:port
#In my case it was gunicorn dashblog.wsgi:application --bind my_ip:5001
You'll see it running and it will give you a bunch of lines stating PID etc. You can test it by going to your IP:PORT and testing it out. Now for bigger applications you'd like gunicorn to spawn more worker sub processes so you can add the --workers flag. I have 2 running at the moment. You can read about all the options available here. Now we know we just need to create a bash script that can run this command with a bunch of flags so that we don't have to type it out every time.

#!/bin/bash
NAME="appname"                                       # Name of the application
DJANGODIR=absolute_path_to_the_project_directory          # Django project directory
SOCKFILE=absolute_path_to_the_socket file_that_will_be_created  # we will communicte using this unix socket
USER=run_as_username                                          # the user to run as
GROUP=nogroup                                         # the group to run as
NUM_WORKERS=2                                         # how many worker processes should Gunicorn spawn
DJANGO_SETTINGS_MODULE=appname.settings_that_you_want_to_use         # which settings file should Django use
DJANGO_WSGI_MODULE=appname.wsgi                      # WSGI module name

echo "Starting $NAME as `whoami`"

# Activate the virtual environment
cd $DJANGODIR
source ../venv/bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH

# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR

# Start your Django Unicorn
# Programs meant to be run under supervisor should not daemonize themselves (do not use --daemon)
exec ../venv/bin/gunicorn ${DJANGO_WSGI_MODULE}:application \
  --name $NAME \
  --workers $NUM_WORKERS \
  --user=$USER --group=$GROUP \
  --log-level=debug \
  --bind=unix:$SOCKFILE

This script was blatantly copied from Michał Karzyński's blog post and modified. Now there's one interesting thing here. The --bind flag utilizes a socket file instead of an address. Now Nginx is a reverse proxy so what it does is, it listens on a port and passes requests from that port to another port that's being utilized by the application. So you can specify the binding address in the Nginx conf or you can simply do this transaction with a unix socket. I created a socket file in my run directory and it worked great. Thanks Michał.

Now that your gunicorn_deploy.sh is ready try it out and check if you get the same result as running the command I listed before. You can see the workers when you do

ps aux
You'll notice three instances instead of the 2 you specified. That's because one of them is the master. Now, an interesting tool that I came to know about is setproctitle that lists the name of the application with the processes. It's a great tool if you have multiple applications running on the same tool.
#You need python-dev for this
$ sudo apt-get install python-dev
$ pip install setproctitle
#once it's installed you'll be able to see the application name with the processes
$ ps aux
Awesome! Let's configure Supervisor so that you don't have to monitor Gunicorn all the time

Installing and configuring Supervisor

First things first, let's install it

sudo apt-get install supervisor
Then, lets' create a conf/ini file that will trigger Gunicorn. The conf file goes in /etc/supervisor/conf.d/
[program:appname]
command = absolute path to the gunicorn script or the command to execute it
user = pronoyc
stdout_logfile = 
redirect_stderr = true
Save it and let's run Supervisor
#First make supervisor aware of your process file
$ sudo supervisorctl reread
$ sudo supervisorctl update
Now you can check if it's working by simply doing this
$ sudo supervisorctl status appname
It will give you the status. If there's been a FATAL error check the log that you specified in the process conf. Beautiful! we got this working, now all we need to do is have nginx running and read from our socket file.

Installing and configuring Nginx

Let's install Nginx

$ sudo apt-get install nginx

It's installed. Now on Ubuntu, in order to run a configuration file all you need to do is write a conf file in the sites-available and symlink to it in sites-enables. Beautiful. So we simply write our conf file in sites-available

cd /etc/nginx/sites-available
touch server_name.conf

Now. You can directly get the default basic Nginx configuration for Gunicorn from here. Simply do this.

$ wget https://raw.github.com/benoitc/gunicorn/master/examples/nginx.conf && mv nginx.conf server_name.conf

Now this file is pretty elaborate. Since our Gunicorn deployment consisted of worker specifications etc. we don't need most of it. Here's a trimmed down version that you really need.

# This is example contains the bare mininum to get nginx going with
# Gunicornservers.  

  # this can be any application server, not just Unicorn/Rainbows!
  upstream app_server {

    server unix:absolute_path_to_socket_file fail_timeout=0;

  }

  server {
    listen 80 default;

    client_max_body_size 4G;
    server_name _;

    keepalive_timeout 5;

    # path for static files
    location /static/ { #STATIC_URL
      alias absolute_path_to_STATIC_ROOT;
    }

    location /media/ { #MEDIA_URL
      alias absolute_path_to_MEDIA_ROOT;
    }

    location / {
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

      proxy_set_header Host $http_host;

      proxy_redirect off;

      if (!-f $request_filename) {
        proxy_pass http://app_server;
        break;
      }
    }

    # Error pages
    error_page 500 502 503 504 /500.html;
    location = /500.html {
      root absolute_path_to_500.html #if you have one;
    }
  }
Now you have your .conf file ready. Simply link it in sites-enabled.
$ cd ../sites-enabled
$ ln -s ../sites-available/server_name.conf .
$ rm default 
$ mv server_name.conf default
$ sudo service nginx restart
If you don't get the status of Nginx as the output, this means something went wrong. Simply go to /var/log/nginx and take a look at the error.log file. There! It's all done it should be running now, go ahead and test it in the browser.

After note and considerations

Okay so this is a little shabby because I would have done it better using fab files. But at the time I was still trying to figure out what was what and didn't have time to write a fab file. But it'll be a lot more cleaner with fab file. There are certain things to take care about that I'd like to leave you with.

  • If you're using DEBUG=False in your Django settings file, make sure you have ALLOWED_HOSTS configured
  • Static files will be accessible only if you specify STATIC_ROOT and do a ./manage collectstatic
  • The wsgi.py file utilises a settings file, you need to specify that

That's it! Thanks for reading! Have a good one.