How to bootstrap a startup using Django

Talk Description

Based on the experiences building Gidsy.com this talk will give you valuable insights as to how your infrastructure will evolve and how to set up the basic components (load balancer, web servers, DB, caching, celery, CDN, …) of your site.

What is Gidsy?

Gidsy is a place where anyone can explore, book and offer things to do.

Why did they choose Django?

  • Big community
  • Network
  • Language
  • Many problems already solved
  • The Admin

Why Django is a good choice?

  • Proven technology by similar use cases
  • Stable APIs in a well-defined release process
  • Good documentation with focus on prose
  • Huge community of 3rd party components

Implementing API

Tastypie: http://tastypieapi.org/

  • Highly customizable Web API library
  • Hooks for auth, throttling, caching, etc
  • Backbone.js compatible
  • Not for 3rd parties, just to serve out system content. So no need yet for OAuth or other protocols

Task Queues

Celery: http://celeryproject.org/

If you have a user triggered process that will take a long time, use an asynchronous task queue to manage the task

  • Async code execution
  • Generate thumbnails, search index updates, caching, etc
  • Collect stats without blocking

Caching

Memcached

  • Periodic cache refreshing for high traffic sites
  • Fragment caching with dates and cache version
  • Cache warming during deployment
  • All their code is based off the Django cache module. They don’t use any libraries here because their data is too complex to rely on a caching framework.

Workflow

“Always check in a module cleaner then when you checked it out.” – Uncle Bob
  • Main branch is always deployable
  • Development happens in feature branches
  • Code reviews via pull requests
  • Shared responsibility

Testing

Releasing

Scaling up

  • TODO - get slide that I missed

Deploy System

  • Builds are virtualenvs
  • Atomic and orchestrated releases
  • Lots of notifications
  • Mix of Chef and Fabric
  • Trying to open source it
  • Using New Relic and Hipchat to track things
  • Operations made as easy as possible via knife and fabric

Operations

  • Log everything you can for debugging

  • If you deploy often you need immediate feedback

  • Use services if you can:

    • Mixpanel
    • NewRelic
    • Librato
    • Papertrail
    • Pageduty
  • Show the metrics on a screen in the office

  • django-app-metrics to get a log trail from the system

Things they learned

  • Only scale when you need to, but be prepared
  • Be pragmatic, use the best tool to do the job
  • Automate as much as you can
  • Continuous integration and deployment
  • Make routine tasks as easy as possible
  • Use services
  • Display metrics