This is part 2 of a multiple part series. Read part 1.
To drive the multiple sites our guidelines were:
- Easily maintained. Adding a new site shouldn’t mean a ton of additional work for everyone.
- Easily understood by developers and producers.
- Allow for site-specific look-and-feel, without having to reimplement the entire look-and-feel.
Django has a module called the sites framework that allows for publishing content on multiple sites, hosted with the same code. We decided not to use this for a couple of reasons
- Different content. The content wasn’t the same; it was translated.
- Multiple independent teams. Django’s sites framework works best when there is overlap across the content producers. In our case, this was unlikely due to language differences.
- Required retrofitting existing code. Our existing code would have to be retrofitted to support this. Given the short time frame, this wasn’t ideal.
- Isolated content deemed better for producers. Django’s sites framework has all the data co-existing within the same database. This could either lead to confusion for producers when the see content they don’t manage, or extra work retrofitting existing code to support object-level privileges.
We decided to have one code base for all sites. I have been a part of efforts to unify code across multiple sites with disastrous results. The reason the efforts would typically fail is due to the very different needs of the sites and the types of content they manage. For example, on the surface it might seem that National Geographic Magazine and the National Geographic Education websites both simply manage articles, photos and videos, they actually are quite different. The education site, for example tracks lots of additional metadata for education needs, such as national standards compliance, age and grade levels and curriculum properties. What allowed us to use one code base in this instance was that each site was to function identically. They would manage the same data in the same ways.
Within our one code repository, each site would have its own set of configurations. This also meant that when the code was deployed on a server, it could serve one or more of the sites, depending on needs. That gave us flexibility in scaling.
We structured the code like this:
codebase ├─ apps │ └─ ... ├─ bootstrap.py ├─ fabfile.py ├─ manage.py ├─ requirements.txt ├─ settings │ ├─ __init__.py │ ├─ base.py │ ├─ dev.py │ └─ production.py ├─ sites │ ├─ __init__.py │ └─ site1 │ ├─ __init__.py │ ├─ conf │ │ ├─ nginx.conf │ │ ├─ gunicorn_conf.py │ │ └─ upstart.conf │ ├─ static │ │ └─ ... │ ├─ templates │ │ └─ ... │ ├─ site_settings.py │ ├─ start_gunicorn.sh │ └─ urls.py ├─ static │ └─ ... ├─ templates │ └─ ... └─ urls.py
|apps||Django applications or other python code that is so specific to the overall site that it is useless for any other site. And also legacy code that we are still stuck with.|
|bootstrap.py||A file that will set up the virtualenv and install the requirements. Typically used on server deployments, but could be used on a local machine.|
|fabfile.py||A file that contains scripts to run on servers|
|manage.py||The default Django command.|
|requirements.txt||All of the requirements for running the code. Can be installed with
The basic settings and derivations.
|sites||This module contains overrides for each site installed using this code base. Each site will have its own module under this.|
|sites/site1||The module for a site. It contains overrides specific to this site.|
Configurations for various parts. These files are symlinked to other locations if necessary
|sites/site1/static||New or modified static files go here. They are collected with the default static media, but over ride any items with the same name.|
|sites/site1/templates||New or modified templates go here.|
|sites/site1/site_settings.py||Settings for this site. It imports settings.production so it only needs to override specific things, such as static directories, template directories, site id, default language, database, etc. Gunicorn is configured to use these settings for this site.|
|sites/site1/start_gunicorn.sh||A shell script, run from Upstart to run the Gunicorn WSGI server at server start.|
|sites/site1/urls.py||Overridden urls, if necessary.|
One of Django’s core philosophies made this process much easier: discourage redundancy and don't repeat yourself. For example, when you specify a template to render, Django doesn’t just look in one place. It can look in several places and take the first one it finds. We would configure site 1 to look for templates in “/sites/site1/templates”, and then in “/templates”. So our base templates could be in “/templates” and then provide site-specific overrides to templates in “/sites/site1/templates”.
Configurations were also done this way. “/sites/site1/site_settings.py” first imports everything in “settings/production.py”, and then adds whatever differences exist for that site such as database, template and language configurations.
The final goal was prevent us from testing every change in every site. That made maintenance much easier and development much faster.