Speeding up docker builds with apt-cacher-ng

One thing I noticed when repeatedly running docker build while developing a new docker file is that re-downloading lots of packages every time I build gets old fast. Run the build, wait, something isn't right, tweak the docker file, run build again, oh look it's downloading all those packages again. I'm sure everyone playing with docker files know exactly what I'm talking about.

So I was thinking about working on my laptop possibly without an internet connection and I suddenly thought, man that would be basically impossible to work on the docker files I want to. That's when I remembered apt-cacher-ng. You see,  I had been wanting to set up an apt-cacher-ng server or equivalent for my home network for a while. I just never got around to doing it. But thinking about how to overcome this potential problem made me think to try using apt-cacher-ng with docker.

Long story short, I set up a quick apt-cacher-ng server (in a docker container of course) and set up another container to use it for apt. Installed a bunch of packages, deleted the container, then went to install the same packages again. It was amazing. The second install, the download phase was basically instantly done. No waiting whatsoever.

I'd publish an apt-cacher-ng docker image, but to do it right, you'll probably want to use volumes to keep the cached downloaded deb's outside of your image, and It's really nothing difficult anyway. There's also a decent example already on the docker website here http://docs.docker.com/examples/apt-cacher-ng/.

So if you've been complaining about re-downloading packages all the time, give this a try.

Java plugin is bad...but docker is good

So, I try to keep my systems java free whenever possible. Why? Same reason I try to avoid adobe reader and adobe flash player. They've become cross platform malware injection tools. Even us linux users aren't safe. So it kind of sucks when you actually NEED the java plugin to do your job. I've got a few servers I manage that have OOB (out of band for the non-initiated) management tools. Guess what, most of them are web based, and require java. Now simple stuff like changing the power state (powering on/off) remotely, no java is needed. However, most of them have the ability to access a virtual KVM if you will, over the network. Yep, you can use an ISO image to fake a cdrom, and get access to the monitor output and get keyboard and mouse input all remotely. So I don't even have to walk down the the server room and I can re-install the OS on my servers.

The only downside, it requires java! Boo. So I used to have java installed on my work machine just in case I needed to access those features. Well at some point I switched to 64bit OS and well, the 64bit java just didn't seem to work correctly anymore. It didn't matter what I installed. Openjdk, sun, etc, none of the 64bit versions worked. So I stopped installing it, and then it seemed vulnerabilities for java were popping up constantly. Even more reason to just not have it installed. Besides, the machines are working fine, no need to re-install right? Well, 4 years later, the OS is coming up on EOL (end of life) and I actually need to re-install some of them. Crap.

I had previously been playing with the idea of installing a 32bit firefox and 32bit java plugin inside a chroot. I got it mostly working with some serious caveats. I had been also following along watching docker evolve and I've been actively using it quite a bit. I also noticed lots of people talking about running gui programs inside docker. So when this new urgent need arose to actually get a java plugin working, I decided to create a docker image for it. That way, not only do I prevent tainting my desktop OS install with java (blah), I also get the benefits of resource isolation that containers give. Yay!

So I started out using the base ubuntu images published by the folks at docker. No go. I spent quite a few hours spread across many days banging my fists on keyboards (probably scaring people in adjacent offices) all to no avail. No matter what I tried, nothing would work. Now, I had a docker image up, with firefox and java working fine. That was not the problem. But whenever I tried running one of those test java applets, nothing would work right. I tried accessing my server's remote management, no dice. I tried different versions of openjdk, every available version of oracle java, and nothing worked. Everything failed for different reasons. I think the closest I got was oracle java 7, where I could see the remote monitor output, and the keyboard input worked, but virtual cdrom drive redirection, nope. That last part was actually crucial, because these machines have no physical cdrom! Oracle java 8 was a total failure. It requires all java applets to be signed or it refuses to run them. That's great for security reasons, but guess what, the manufacturer of my servers 5 years ago, weren't really thinking about signing their apps. And there's no way to update them, they're baked into firmware.

So I dropped the whole Idea. I mean, I still had a year or so before EOL, I'll think about it later. Then one day it just hit me. The 32bit version of openjdk actually worked pretty good. If only I could get a 32bit base OS in docker to work with. Wait, I remember seeing ubuntu published "core os" tarballs, which were meant for building vm images from, and docker can import a tarball to create an image from. So I did just that, downloaded the latest ubuntu 14.04 i386 core os tarball, imported it as a docker image, and used that for the basis of my docker recipe. Sure enough, firefox and java plugin work great.

I'll probably publish my 32bit base image based on the core os tarball. I just need to see if there are any tweaks missing that the main ubuntu image has. I may also publish my firefox docker file. I guess it depends on if anyone actually would find it useful.

SSH just makes you feel like a freakin' ninja

Ok, so this is not a new thing, I've done this many times before, but every time I use SSH to solve some odd problem, I feel like I'm way more awesome than I really am. I think it must be how the cool kids in high school felt all the time.

So, again, I needed to test something on a customer's network, and I still don't have a dedicated VPN. I keep thinking I'll set one up, but I just don't need to use it that often. And usually, the only thing I'd want a dedicated VPN for, is to connect a physical device to their network from home. In order to do that, I'd need like a dedicated VPN box and a dedicated vlan for traffic to each customer setup like that. But I wouldn't have them connected 24/7, so really, it just doesn't make sense. It's much easier to just create an adhoc VPN with ssh when needed.

So for the un-initiated, openssh for quite a while now, has had support for creating virtual network adapters (tun and tap) on both sides of an ssh connection, and all traffic between the two virtual adapters are tunneled over your ssh connection. It's pretty slick. You can either create a point-to-point connection (tun device) or virtual ethernet (tap device). The latter lets you create a bridged network, which is what I tend to do.

So, I had a customer with an asterisk phone system issue. Their handsets weren't configured properly, and DTMF signals were either not being sent, or being interpreted incorrectly. But I needed a handset like theirs, on their network, connected to their phone system to test out the various settings to find the right combination. I really didn't want to drive in. Luckily, I had one of their spare handsets at my house. Now all I needed was a way to make this wired phone think it was physically on their network. SSH to the rescue.

Now, this isn't a setup that you can pull off easily. There are many caveats. For example, to created an ethernet bridge, you need machines with SSH on both sides, that also have an ethernet adapter setup in bridge mode. Luckily, I have an embedded debian box on their premise just for SSH'ing into. I just had to change it's network config to create a bridge device, and add it's eth0 to the bridge on startup, then reboot and hope i didn't screw up. Luckily, this time, I didn't screw up, and I was able to get back in after reboot.

So here's my setup, I had a debian desktop at my house with a spare ethernet adapter (I've also used a laptop with a usb to ethernet adapter in the past). I set up the spare ethernet adapter (eth1) to be part of a bridge.

ifconfig eth1 0.0.0.0 promisc up
brctl addbr br0
brctl addif br0 eth1

 Next, I made sure the other end was set up similarly, I simply edited it's
/etc/network/interfaces

iface eth0 inet manual
auto br0
iface br0 inet static
    bridge_ports eth0
    address 192.168.1.42
    netmask 255.255.255.0
    gateway 192.168.1.254

Made it look something along those lines, and rebooted the machine, crossed my fingers and waited for it to come back up. Luckily it did, because if it didn't, I'd be driving out late at night, and that didn't sound fun to me.

Now, you just use SSH and tell it to use 'ethernet' tunnel devices (i.e. tap devices):

sudo ssh root@REMOTE -o Tunnel=ethernet -w any:any

A couple things to note above. First of all, notice I'm both doing sudo, and also ssh'ing to the remote hosts root account. This is because it seems that only root can create the necessary tun/tap devices on either end. The next thing to note, I had to specify the extra -o option before the -w option. If the -w option came first in the command line, it created tun devices on both ends instead of tap devices (tun can only do point to point networks, i.e. routed networks, I want bridged networks).

Once you do that, you should be logged into the remote machine, and it should have an extra tap0 device (or some other number if you already have tap devices defined). At this point, you simply bring up the tap0 device on both ends, and add the device to your bridge. So something like:

ifconfig tap0 0.0.0.0 promisc up
brctl addif br0 tap0

Run that on both ends, replacing tap0 with the tap device that was created, and br0 with the name of the bridge device you created. At this point, you now have a bridge between your local spare ethernet device, and your remote network. Essentially, anything you connect to your local spare ethernet device will act as if it was directly plugged into the remote network!

So what I did, was connect up a small 10/100 switch to the spare ethernet port, so now this 10/100 switch is essentially bridged to the switch at the remote network. Pretty cool. Now I just plug in my SIP phone, power it on, and what do you know, it gets a dhcp address from the remote network, and registers itself with the phone system at my client's location. At this point, I can test the phone system at my leisure just like I was physically on their premises.

But every time I use SSH to pull off something cool like that, it just feels awesome. I just wish all sysadmin tasks were as exciting. I think it's more that I over came the challenge of testing something without having to physically go there. Either way, it's cool, at least I think so.