I’ve been using docker a lot, and on occasion I need to transfer images between two machines that are on a local network. If a particular image is large, I might not want to download it twice from two machines, so I download it on one machine and transfer it to the other over the local network.

Now, I could stand up a local docker registry and use that, but it’s a bit of work. Instead, I’ve found that the quickest and easiest solution is to combine the docker ‘save’ and ‘load’ commands with a bit of netcat magic, and it’s pretty fast and easy. (Update: you can do it easily using SSH too, see the end of the post). Check it out.

First, on the destination machine (make sure your firewall allows traffic to the specified port, in this case 1234):

nc -v -l 1234 | docker load

Next, on the source machine, transfer the image (virtuald/etcd:0.4.6) to the destination IP (192.168.0.42):

docker save virtuald/etcd:0.4.6 | nc -v 192.168.0.42 1234

And that’s it!

The sad thing is that docker save/load doesn’t show a status message when saving/loading, so it might look like it’s not doing anything. However, using the -v flag for netcat shows when the connection is successfully opened/closed, so that’s something.

Security warning: Obviously, running netcat like this is a *huge* security hole while its up and listening, as anyone who can connect to the port can upload arbitrary images into your docker registry. This is mitigated a bit since netcat will immediately disconnect after the first client disconnects, but still risky on an untrusted network. Only use this on trusted networks!

Note: due to this bug, you’ll want to be using docker 1.2+, otherwise you may get unexpected results.

Update! As Joshua Barratt points out, since this method generalizes to any transport that allows piping via stdin/stdout, you can also do the transfer via SSH too, which is certainly more secure. Use the -C option to enable compression for faster transfers (thanks Andreas Steffan).

docker save virtuald/etcd:0.4.6 | ssh -C 192.168.0.42 ‘docker load’

Update II: As a number of people have pointed out, you can use PV to show a status message:

docker save virtuald/etcd:0.4.6 | pv | ssh -C 192.168.0.42 'docker load'

I’m just starting to play around with docker, and I’ve been investigating the use of CoreOS for deploying a cluster of docker containers. Though I’ve only been using it for a week, I really like what I’ve seen so far. CoreOS is makes it very easy to cluster together a group of machines using etcd, and in particular, I really like their fleet software, which allows you to manage systemd units (which you can use to run docker containers) across an entire CoreOS cluster. Fleet makes it easy to do things like high availability, failure recovery, and other useful things without too much extra effort right out of the box. The one piece missing is how to connect the containers together. There are some ways they’ve documented to do it, but honestly most of the ways I’ve seen on the internet consist of a bunch of shell script glue that feels really hacky to me.

In the docker community, something called the ‘ambassador’ pattern has emerged, which is this idea of proxying connections to container A from container B via container P, and container P has enough smarts in it to transparently redirect connections to many different containers depending on parameters. However, most of the stuff I’ve found on the web is very labor intensive and full of nasty shell scripting that is easy to mess up.

Jeff Lindsay has created the first stage of what I think is a really good general solution to this problem — namely, his projects called registrator and ambassadord. Registrator listens for docker containers to startup, and automatically adds them something like etcd or consul. You link your containers to ambassadord, and when your container tries to make an outgoing connection, it will do a lookup to figure out where the connection needs to go, and connect you there. It’s pretty easy, with very little configuration needed for the involved containers.

CoreOS already ships with etcd built-in, so CoreOS + registrator + ambassadord seems to be a great combination to me. I’ve modified CoreOS’s sample vagrant cluster to demonstrate how to use these to connect containers together.

(more…)