Thursday, July 26, 2012

Jenkins forces reinstall of server images on Rackspace

This is much more DevOps than Dev, but it is important as well ! One difficulty we have doing remote development with a team spread over at least a million cities (I haven't actually counted) is that the local development LAN for Bob is frankly inaccessible for Alice and Charlie. Maintaining n-1 VPN links is just going to drive us all mad.
So the simple solution is to rent a couple of cloud servers and host the development builds on there. The cost is pretty minimal, and it seemed good - until I realised that Jenkins, our friendly Continuous Integration Butler, was unable to re-install a cloud server from an image. Or rather it *could* - but Jenkins did not know when the server was rebuilt, and the server itself, which could signal jenkins, would have no state, so could not signal jenkins with *why* it had been rebuilt. So, Pantry Bell (https://github.com/lifeisstillgood/pantrybell). We simply run a webserver, that when called (perhaps from one Jenkins job) will force a re-imaging of a Rackspace server. This has a simple callback in rc.local (wget http://jenkins.frozone.mikadosoftware.com/pantrybell//`hostname`) So - a server forces a rebuild, on successful rebuild, the server calls home, and then the "real" jenkins job can be triggered using the Jenkins API calls. Amazingly enough it works, has trivial security, no REST compliance but it works

2 comments:

  1. Thank you for writing this.

    ReplyDelete
  2. I think you should automatically donate your organs because that would turn the balance of organ donation in a huge way. I would donate whatever anybody would take, and I'd probably do the cremation bit.
    qwest

    ReplyDelete