jk's blog

A Small Web Dev Network

This is a quick description of my current web dev network that includes a caching proxy server for Debian and Ubuntu packages, and Ansible.

Due to general annoyance with Ubuntu, I started using Debian again, but when I started learning Ansible to set up a staging server, I had to switch back, because Ansible plays nice with Ubuntu. It works with Debian but you need to build from sources.

So, start out by firing up VirtualBox and creating two machines (512M to 1G RAM, 8G disk). Into one, I loaded Debian, but any distro is fine. Debian is just smaller. (I also tend to use Vagrant-style account: username vagrant, password vagrant; just in case I want to Vagrant-ize it.)


Then I followed these directions to set up an apt-cacher server

I’ve used Squid3 before, but I’m finding this use-specific cache performs better.

The reason for having this server is to avoid hitting the slow internet when downloading packages. In the next part, we bootstrap Ansible, and the point of using Ansible is to be able to “instantly” rebuild a server from scratch.


Devpi is a full-featured Python repository server. In addition to caching packages, it also lets you host your own repository. I set this up with Apache and daemontools, here.


This is a cache for node’s npm. I did a generic installation, but [made it run from daemontools]. Unfortunately, there seem to be some problems running local-npm across a network, so this is kind of moot.


Ansible is a tool to configure computers. It’s like installer scripts, except with a little intelligence and a lot more organization.

The “intelligence” part is that it installs packages only if the package isn’t installed. It does the same for settings and config files.

The organization part is that you don’t use scripts – you use YAML based config files. Ansible modules interpret the config files and act on the settings. The config files are broken out into a fairly rigid tree of directories.

Here’s an example of a config:

  - name: Install Node
    apt: name=nodejs

So instead of “apt-get install nodejs” you have that setting above.

These settings are called “playbooks”. The Ansible docs explain it well.

The idea is that the playbook contains a description of the final machine state. Execution of the playbook is sequential, from top to bottom (except for triggers), but you are expected to run the playbook more than once to resolve unmet dependencies. (Contrast with package managers that try to resolve dependencies.)

You run a playbook with ansible-playbook:

ansible-playbook playbook.yml

A complete machine state is broken down into modules called “roles”:

- hosts: localhost
     - mysql
     - apache2

A role is like a module. It’s stored in a subdirectory, roles/*. The directory tree is like this:


It takes a while to learn Ansible. It’s not hard – it’s just different from shell scripting, and it takes a while to read the docs and learn the modules. Executing playbooks is kind of slow, as well.

Setting up (self hosting) Ansible

The use-case described on the Ansible website pertains mainly to devops who run a network of machines. My personal use case is a development environment, and I don’t need the overhead of centralizing Ansible, so I will run Ansible within the virtual machines, to install itself.

First, set up a virtual machine with the Ubuntu server ISO. Again, I use the generic Vagrant setup: username vagrant, password vagrant.

This script will set up the computer to install itself, and also configure the local apt to use the apt-cacher.

# ansible-setup.sh
cat << EOF > /etc/apt/apt.conf.d/70proxy 
Acquire::http::Proxy "";

apt-get install -y software-properties-common
apt-add-repository ppa:ansible/ansible
apt-get update
apt-get install -y ansible
apt-get upgrade -y

This takes several minutes to run, then it’s pretty much ready to go. I also install the VirtualBox Guest Additions so I can get folder sharing, and do my dev through a shared folder. It’s not necessary, but it’s more convenient.

Folder Sharing on Boot

This section is wrong. There’s a symlink bug in VBox that makes using virtualenv and npm impossible in the shared folder. I’m going to switch to NFS sharing.

There’s a fix to deal with folder sharing not being available on boot. Basically, you add “vboxsf” to the /etc/modules in both the main boot and the initrd ramdisk boot.

There is one caveat with Ansible – it is a Python application, and it installed a lot of common packages, like Jinja2 and requests. So, for your deployment, you must use a virtualenv. (Hynek Schlawack says to always use it.) I’ll try to get to that point.

Attachment Size
ansible-setup.sh 442 bytes