May 17, 2012

Amazonian Fabric

Note

Good talk about patterns and techniques but I’m having trouble keeping up with switching to slides and code non-stop.

Basics

  • Why another management framework?

    • Wanted to be up and running quickly.
    • Wasn’t familiar with Chef, Puppet, or even cfengine.
    • Was very familiar with sysadmin, Python, boto, and EC2

Starting instances

Let’s start a backup database server:

$start_instance web web99

This will:

  • Create new instance of the “database” type
  • Give it the host name db10.memrise.com
  • Install MySQL 5.5 from source
  • Install our custom .cnf

Instance Types

EC2 Instance Types and AMIs are configured using the… (lost the slide content here)

Control Server

  • “Master” server for running other servers and tracking state.
  • Issue: Another system to maintain
  • Very good at getting out of sync with running systems.

Deployment in Parallel

The fabric command for deployments now can:

  • Fund the host types that should be deployed to
  • Run common global deployment code once
  • Run common deployment code across all servers in parallel
  • Update the load balancer

Building Role Lists

  • Create lists of each server type using instance_type metadata and instance state
  • Add ssh username and port
  • Cache results to speed repeated calls

Scaling Web Servers

  • Can’t use EC2 Elastic Cloud Service because of metrics
  • Will often know in advance when load spikes will occur
  • Want to make adding and removing capacity very easy

Load balancing

  • Site stack deployed to all active web servers
  • New active servers added to load balancer, DNS updated
  • Inactive servers removed from load balancer
  • When servers inactivated, also removed