Monthly Archives: October 2014

How I Docker, Part 2

Yesterday I laid out a high level overview of how I use Docker. I neglected to point out 1 thing — the environment variables. I exposed the POSTGRES_USER and POSTGRES_PASSWORD, but didn’t explicitly call out what to do with those in the context of the Rails app.


default: &default
  adapter: postgresql
  encoding: unicode
  pool: 5
  host: <%= ENV['POSTGRES_PORT_5432_TCP_ADDR'] || 'db' %>
  username: <%= ENV['POSTGRES_USER'] || 'developer' %>
  password: <%= ENV['POSTGRES_PASSWORD'] || 'password' %>

  <<: *default
  database: rails_4_1_6_development

  <<: *default
  database: rails_4_1_6_test

  <<: *default
  database: rails_4_1_6_production

Docker exposes some variables for you based on the containers and their names. You can also specify your own with that -e switch.

Alright. With that little bit cleared up, let’s talk about how I actually use all this now.

How I Actually Use All This Now

Remember the intent of this exercise was to come up with a way to easily share an app with it’s dependencies in some nice tidy package such that nobody else needs to understand how to line up all those dependencies and deal with the maintenance.

Docker is great, but one of the things that I liked about Vagrant was how easily provisioning and reprovisioning happens. But I don’t really need all the overhead of some big complicated thing that requires 3 separate licenses and all that. We have boot2docker, which handles the virtualization on OSX with VirtualBox. It’s maybe not the most amazing software, but it hasn’t left a bad taste in my mouth yet.

I like the simplicity of Docker, and I like the simplicity of just using the tooling that Docker provides us.

So to simplify the workflow (using Docker and Boot2Docker) I wrote a bunch of rake tasks. Feel free to comment or submit updates.

The Rakefile

So what does this buy us, this giant pile of rake tasks? This gives us a thin wrapper around some pretty basic functionality. Rake also has the ability to chain things together, so you can ask to clean up all the old instances and it will make sure they’re all stopped and that boot2docker is even running.

It makes is so that we can pass this around and have a simplified workflow.

Sometimes boot2docker loses it’s mind and quits on you, and you don’t even have to care about that anymore because the rake tasks will make sure boot2docker is running for you.

Here are all the rake tasks that I currently have in place:
$ rake -T
rake boot2docker:install_nsenter # Install nsenter
rake boot2docker:shellinit # Setup shell
rake boot2docker:ssh[container] # SSH into the given container
rake boot2docker:start # Start VM from any states
rake boot2docker:stop # Gracefully shutdown the VM
rake docker:fig:build # Use fig to build the Dockerfile image
rake docker:fig:up # Use fig to run the app as described in fig.yml
rake docker:info:hostname[container,port] # Give the hostname and port for a container
rake docker:info:images # List all the images on localhost (docker images)
rake docker:info:ps # List all the container (docker ps -a)
rake docker:maintenance:rm[container] # Remove one or more containers
rake docker:maintenance:rmi[container] # Remove one or more images
rake docker:maintenance:stop[container] # Stop running containers
rake git:pull[remote,branch] # Pull the freshest code
rake git:stash # Stash files

Generalized Yet Specific

So we have Dockerfiles and environment variables that we can pass into containers. We have boot2docker to handle the virtualization for us on OSX. We have rake to manage docker and boot2docker. But what about apps and their dependencies?


We need a DSL that is a little bit generalized, but not as generalized as Chef or Puppet. We just need to be able to tell Docker how to do all those things that were in my little container startup script. Enter Fig.

Whereas before we had

docker run -d \
  --name rails-basic-postgres \

docker build -t="barrettclark/rails-basic:devel" .

docker run -d -P \
  --name rails-basic-app \
  --link rails-basic-postgres:postgres \
  -v /Users/barrettclark/temp/rails_basic/project:/rails \

We now have fig.yml

  image: barrettclark/postgis
    - "5432"
    - POSTGRES_USER=docker
  build: .
    - POSTGRES_USER=docker
    - ./project:/rails
    - db
    - "3000"

You’ll note in the Rakefile that the default rake task is docker:fig:up, which will first run the docker:fig:build task (and also make sure boot2docker is running and the environment is set).

Putting It All Together

So let’s run this app, shall we?
$ rake
db uses an image, skipping
Building web...
---> d90ac1ef8a16
Step 1 : RUN apt-get update
---> Using cache
---> 8e44373a6c4e
Step 2 : RUN apt-get upgrade -y
---> Using cache
---> 7c8a817248f8
Step 3 : RUN apt-get -y --fix-missing install libpq-dev nodejs
---> Using cache
---> 0fd7db2ace6d
Step 4 : RUN gem install bundler --no-ri --no-rdoc
---> Using cache
---> d653711e99ef
Step 5 : ADD
---> Using cache
---> b5298535c564
Step 6 : RUN chmod +x /
---> Using cache
---> 51f55dde7927
Step 7 : ADD project/Gemfile Gemfile
---> Using cache
---> 49f2911395c8
Step 8 : ADD project/Gemfile Gemfile.lock
---> Using cache
---> f1f6b6cf4863
Step 9 : RUN bundle install
---> Using cache
---> 73271fb83da5
Step 10 : RUN rm /Gemfile*
---> Using cache
---> be0105be5b00
Step 11 : ADD project/ /rails
---> ff9da0621df5
Removing intermediate container b18ed235b2a5
Step 12 : WORKDIR /rails
---> Running in 045c488e22c0
---> 4c85897ee553
Removing intermediate container 045c488e22c0
Step 13 : EXPOSE 3000
---> Running in 0909d1f73082
---> 43df967060e7
Removing intermediate container 0909d1f73082
Step 14 : CMD /
---> Running in 8c515adc0285
---> 69d3e7832b58
Removing intermediate container 8c515adc0285
Successfully built 69d3e7832b58
Creating railsbasic_db_1...
Creating railsbasic_web_1...
Attaching to railsbasic_db_1, railsbasic_web_1
web_1 | *** STARTING RAILS APP ***
web_1 | rm: cannot remove /rails/tmp/pids/': No such file or directory
db_1 | 2014-10-28 15:00:14 UTC LOG: database system was shut down at 2014-10-17 16:06:21 UTC
db_1 | 2014-10-28 15:00:14 UTC LOG: autovacuum launcher started
db_1 | 2014-10-28 15:00:14 UTC LOG: database system is ready to accept connections
web_1 | Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
web_1 | installing your bundle as root will break this application for all non-root
web_1 | users on this machine.
web_1 | Fetching gem metadata from
web_1 | Using rake 10.3.2
web_1 | Using i18n 0.6.11
web_1 | Using json 1.8.1
web_1 | Using minitest 5.4.2
web_1 | Using thread_safe 0.3.4
web_1 | Using tzinfo 1.2.2
web_1 | Using activesupport 4.1.6
web_1 | Using builder 3.2.2
web_1 | Using erubis 2.7.0
web_1 | Using actionview 4.1.6
web_1 | Using rack 1.5.2
web_1 | Using rack-test 0.6.2
web_1 | Using actionpack 4.1.6
web_1 | Installing mime-types 2.3
web_1 | Using mail 2.6.1
web_1 | Using actionmailer 4.1.6
web_1 | Using activemodel 4.1.6
web_1 | Using arel
web_1 | Using activerecord 4.1.6
web_1 | Using coffee-script-source 1.8.0
web_1 | Installing execjs 2.2.1
web_1 | Using coffee-script 2.3.0
web_1 | Using thor 0.19.1
web_1 | Using railties 4.1.6
web_1 | Using coffee-rails 4.0.1
web_1 | Using hike 1.2.3
web_1 | Using multi_json 1.10.1
web_1 | Installing jbuilder 2.1.3
web_1 | Using jquery-rails 3.1.2
web_1 | Using pg 0.17.1
web_1 | Using bundler 1.7.4
web_1 | Using tilt 1.4.1
web_1 | Using sprockets 2.11.0
web_1 | Installing sprockets-rails 2.1.4
web_1 | Using rails 4.1.6
web_1 | Using rdoc 4.1.2
web_1 | Using sass 3.2.19
web_1 | Using sass-rails 4.0.3
web_1 | Using sdoc 0.4.1
web_1 | Using spring 1.1.3
web_1 | Using turbolinks 2.4.0
web_1 | Using uglifier 2.5.3
web_1 | Your bundle is complete!
web_1 | Use
bundle show [gemname]` to see where a bundled gem is installed.
web_1 | == 20141004114903 CreateGrilledThings: migrating ==============================
web_1 | -- create_table(:grilled_things)
web_1 | -> 0.0099s
web_1 | == 20141004114903 CreateGrilledThings: migrated (0.0100s) =====================
web_1 |
web_1 | [2014-10-28 15:00:43] INFO WEBrick 1.3.1
web_1 | [2014-10-28 15:00:43] INFO ruby 2.1.2 (2014-05-08) [x86_64-linux]
web_1 | [2014-10-28 15:00:43] INFO WEBrick::HTTPServer#start: pid=25 port=3000

Then you can ask docker about the instance:
$ rake docker:info:hostname[railsbasic_web_1,3000]

Even cooler, you can open that up in your browser with
open $(rake docker:info:hostname[railsbasic_web_1,3000])

And using a little bit on linux magic (nsenter) you can SSH into the running container:
$ rake boot2docker:ssh[railsbasic_web_1]
SSH into container: f52a7f422ec2


So that’s where my head is at the moment. Docker + boot2docker + rake + fig. That gives you a flexible system that is still specific enough. You can run multiple apps, and you can have multiple apps that have dependencies in each other. I’ve had a little more trouble getting this running with a Scala stack, so YMMV.

Again, I’m not saying that any of this is particularly correct. It’s just what has evolved for me in my particular team’s use case. I would love to hear feedback on what you think works best for you and your needs. I would also like to thank Clifton King, who gave me a couple of docker-related tips along the way.


Filed under programming

How I Docker

I have spent the past several months playing with Docker off and on. I have to say that I think I really like it.

I love the idea of really small, specific, containers. I also love the idea of building up a toolkit of things that you use.

Some Background

I work at Sabre Labs, and we get to explore and experiment with trends and technologies that we think could make a difference in the travel world. This means that we spin up a lot of projects in a lot of different technologies and languages. So it is really nice to have a toolkit of things that we’ve used and done. That also helps with knowledge sharing.

About a year ago we set upon a path to attempt to make DevOps easier in a cross-functional team. We have a designer, a front-end developer, and a couple of back-end or full-stack developers. DevOps can be very time consuming when you’re working in a Java stack one day and a Rails stack the next.


First we looked at Vagrant, using Chef to setup the dependencies. It worked, generally. It’s expensive. It’s messy.

The licenses are frustrating. You pay for Vagrant, which is fine. If you want to use VMWare, which we did, you pay for that. Also fine. If you want to use Vagrant with VMWare you pay for a third license. That feels weird.

Chef. I’ve used Chef before in previous engagements, and it’s worked ok. It’s better than nothing (or shell scripts) for sure. My frustration with Chef is how messy the recipes are. The dependencies in particular. We didn’t use Chef Server, so we would just copy in all the recipes that we needed, and then their dependencies. Look at the dependency chain for git (on Ubuntu). Before you’re done you’ve brought in Windows dependencies. And also the Yum package manager. Why? That makes me crazy.

I know that Vagrant supports Puppet, and I just never got around to looking at Puppet. I know that Vagrant also supports Docker now, and I can’t get it to dance. But I don’t really want to. I think that’s overly complicated.


What’s so great about Docker then? It’s complicated in different ways. Or can be complicated.

Like I said at the top, I love how simple and concise the containers are. I also like the DSL. It’s specific, and it lets me do what feels natural on that particular environment. I don’t need to overly generalize. I don’t need a DSL that can work on all *nix distros. I just want to say what I need and make it so.

Teach Me How To Docker

Look around at blog posts on how to Docker. I’ve noticed that a great many of them fall into the “install ALLTHETHINGS” camp. You build a single container that includes the database and the app server. Bleh. Why bother with containerization if you’re just going to stick everything in a single container?

Small. Simple. Concise. That’s how I Docker.

So, without all of that as the background, here is a simple Docker setup for a simple Rails app.


FROM barrettclark/ruby-2.1.2

RUN apt-get update
RUN apt-get upgrade -y
RUN apt-get -y --fix-missing install libpq-dev nodejs

RUN gem install bundler --no-ri --no-rdoc

RUN chmod +x /
ADD project/Gemfile Gemfile
ADD project/Gemfile Gemfile.lock
RUN bundle install
RUN rm /Gemfile*

ADD project/ /rails
WORKDIR /rails



You see here that I have a base container image called barrettclark/ruby-2.1.2. Docker containers can inherit from other containers. This is where you can end up with a bit of a Russian Doll situation because that container takes a base image and puts Ruby 2.1.2 on it. You see it then runs some basic apt-get commands. We are using Postgres for this Rails app, so we need those header files for the pg gem, hence libpq-dev.

Next we add some files. Docker caches each step to see if it needs to be rebuilt. That’s why we don’t just copy in the whole project yet. Once we’ve handled the gem dependencies, then we can copy over the app. Rails will run on port 3000, and because we know it’s the only thing running in that container, and also in this machine for that matter, we can just expose that port and call it good.

CMD / tells the container to run that file when the container is started (unless you tell it to do something else when you start it). You can see that it does some housekeeping, database setup, and fires up the app.

echo "*** STARTING RAILS APP ***"
rm /rails/tmp/pids/
bundle install
rake db:create
rake db:migrate
rake db:seed
bundle exec rake log:clear
rails s

Build the container: docker build -t="barrettclark/rails-basic:devel" . Great. Now you’ve got everything you need to run a simple Rails app in Docker.

Run All The Things

I usually write a little shell script to help with this step. We need 2 containers, and we want them eventually linked together.

docker run -d \
--name rails-basic-postgres \

docker run -d -P \
--name rails-basic-app \
--link rails-basic-postgres:postgres \
-v /Users/barrettclark/temp/rails_basic/project:/rails \
barrettclark/rails-basic:devel /bin/bash -l -c "/"

What’s all that? The first docker run spins up a Postgres container named ‘rails-basic-postgres’ and sets some environment variables. If you don’t have the given image on your localhost, Docker will go try to fetch it from Dockerhub. The ‘-d’ switch tells Docker to daemonize the container — run it in the background. Next we run the Rails container and link that Postgres container to it.

Remember how I said the whole point of all this was to be able to give this service (or whatever) to everyone else in a cross-functional team. I don’t want them to have to know how to line up all these dependencies. This allows me to handle that for them. The real kicker here is that you can mount a directory on the localhost inside the container. Yes! The caveat is that the localhost directory MUST live in /Users. A symlink is not sufficient. Now this allows a designer to play with the javascript or markup on his or her computer in his or her preferred editor, and nobody has to know where the code is actually running.

Well, that last piece isn’t entirely true. Remember how I sort of waved my hands at just exposing port 3000 and calling it good? The -P Docker option allows us to expose ports, and Docker will map them to a port on the localhost. If you want to be specific you can do that as well with a different switch.

Run docker ps -a to see all the instances running and what ports they’ve been mapped to.

I’m not saying that any of this is particularly correct. It’s just what has evolved and makes the most sense to me. The next blog post will go into a little more detail on how I actually use all of this, and some tooling that I’ve written to make it a little easier.


Filed under programming

Ruby Minitest

I had the privilege of attending DCamp this past weekend. It was awesome. While doing the Game Of Life pairing sessions, one of my pair partners wanted to explore the exercise using Minitest. Neither of us had ever used it, so we moved on with RSpec and I made a note to come back to it.

I try to be discerning about bringing additional dependencies (including gems) into a project. I’ve always just used test/unit, and I prefer that syntax over the RSpec syntax. Minitest is baked into the stdlib (starting with Ruby 1.9.3), so you’ve already got everything you need. You can also write tests using either the test/unit or spec style. Win-win right? If you want more there are also additional gems that can supplement functionality. Everybody wins!!!

Here is a basic class, taken from the Minitest gem documentation.


class Meme
  def i_can_has_cheezburger?

  def will_it_blend?

Unit tests, using Minitest:


require "minitest/autorun"
require_relative "../lib/meme"

class TestMeme < Minitest::Unit::TestCase
  def setup
    @meme =

  def test_that_kitty_can_eat
    assert_equal "OHAI!", @meme.i_can_has_cheezburger?

  def test_that_it_will_not_blend
    refute_match /^no/i, @meme.will_it_blend?

  def test_that_will_be_skipped
    skip "test this later"

And a spec testing the same things:


require 'minitest/autorun'
require_relative "../lib/meme"

describe Meme do
  before do
    @meme =

  describe "when asked about cheeseburgers" do
    it "must respond positively" do
      @meme.i_can_has_cheezburger?.must_equal "OHAI!"

  describe "when asked about blending possibilities" do
    it "won't say no" do
      @meme.will_it_blend?.wont_match /^no/i

Outstanding. You can run either of these from the command line:

$ ruby spec/meme_spec.rb
Run options: –seed 25656

# Running tests:


Finished tests in 0.001430s, 1398.6014 tests/s, 2097.9021 assertions/s.

2 tests, 3 assertions, 0 failures, 0 errors, 0 skips

You can also use rake to run the tests, and this is the preferred method.


require 'rake/testtask' do |t|
  t.warning    = false    # this is the default -- shown as example of options
  t.verbose    = false
  t.test_files = FileList['spec/*_spec.rb']
end do |t|
  t.test_files = FileList['test/test_*.rb']

desc "Run ALL tests"
task :default do

Now you can ask rake what tasks it has for you:

$ rake -T
rake default # Run ALL tests
rake spec # Run tests for spec
rake unit # Run tests for unit

Cool. How about we just run the default rake task (ALLTHETESTS):

$ rake
Run options: –seed 28918

# Running tests:


Finished tests in 0.001187s, 1684.9200 tests/s, 2527.3799 assertions/s.

2 tests, 3 assertions, 0 failures, 0 errors, 0 skips
Run options: –seed 26846

# Running tests:


Finished tests in 0.000936s, 3205.1282 tests/s, 3205.1282 assertions/s.

3 tests, 3 assertions, 0 failures, 0 errors, 1 skips

Awesome. You can also run all the tests, just a group of tests, or even just a single test file. This looks tasty enough to overtake test/unit as my go-to testing framework.

Leave a Comment

Filed under programming, ruby