Writings on various topics (mostly technical) from Oliver Hookins and Angela Collins. We currently reside in Sydney after almost a decade in Berlin, have three kids, and have far too little time to really justify having a blog.
A couple of years ago now at least, at my previous employer, I was just starting to feel reasonably comfortable with Ruby. I can attribute a large part of that to having had a great boss who gave me the time to sink a bunch of time into coding up our IaaS and Config Management systems while still being effectively a Systems Administrator, and another large part to reading Metaprogramming Ruby. A…
Well, despite my best intentions, here I am again writing Ruby. I decided to automate a small part of some data analysis I've had to do a few times, starting with the database queries themselves. Unfortunately the data is spread over several hosts and databases and the first implementation simply queried them serially. The next iteration used the mysql2 gem's asynchronous query functionality but…
Almost two years ago now, I wrote a post about how I was fed up with resorting to shell scripting as my knee-jerk reaction to computer problems. At the time, I had been attacking any problem that required more than a couple of commands at the prompt by writing a shell (usually BASH) script and hit major limitations that I really should have been solving with a legitimate programming language. I…
Unlike many, I'm actually a good boy and do backups of my personal data (for which I can mostly thank my obsessive-compulsive side). However, up until now I've been remiss in my duties to also take these backups off-site in case of fire, theft, acts of god or gods etc. Without a tape system or rotation of hard drives (not to mention an actual "off-site" site to store them), this ends up being a…
As I've previously mentioned, I've been doing a lot of work recently with various aspects of AWS on a daily basis (or close to it). My primary language these days is still Ruby, but I've been labouring through the excellent Seven Languages in Seven Weeks book in the hope I can broaden my horizons somewhat. I'm fairly comfortable with Python, somewhat familiar with Javascript now after playing with…
I've been using S3 a lot in the last couple of months, and with the Amazon SDK for Ruby it really is dead simple to work with (as well as all of the other AWS services the SDK supports currently). So simple in fact, that you could quite easily delete all of your objects with very little work indeed. I did some benchmarks and found that (with batch operations) it took around 3 minutes to delete…
Just a quick note since this tripped me up yesterday. If you take a look at the changelog of RSpec between 2.6.0 and 2.6.1 you'll see that it now "intelligently" senses when being used with RCov and requires the autorun file (which basically just sets the tests to start automatically after they have been set up).
At some point for one of our simple apps I upgraded all of the Gems in use to the…
So I was going to post a comment on the original blog post which I linked to from here but Facebook connect was broken and I don't feel like setting up yet another account </1stworldproblems>... but there was a slight development.
I attempted to use the same methodology we had already followed with the first app, to another app so that this one was also packaged using Bundler and RPM. Needing to confirm that all was well before I committed the changes I did some testing in a CentOS virtual machine in Vagrant. To be expected, with a deployment bundle of a decent few gems the package size…
Recently I was working with one of our valued Thoughtworkers on an application we were trying to not only develop in a sane way, but package and deploy to production with just as much sanity. The status quo seems to favour bundler on the development side, but RPMs on the production side (if you judge these decisions based on what developers and ops folk prefer, generally).
After a reasonable amount of WTFing, we actually managed to get it working reasonably well. If you have Ruby apps with gem dependencies and want to develop and push to production with equal ease I suggest you read the blog post on the subject by Philip Potter here: …
These days I'm noticing myself saying more and more frequently that Duck Typing is great, except when it's not.
An amusing issue that briefly cropped up this afternoon was when we failed to correctly negotiate a data structure inside of a Rake task. Consider the following basic task: ``
desc "a test task"
task :test, :glob do |t,args|
if args[:glob].nil?
args[:glob] = 'some default value…I've been spreading this excellent talk by Gary Bernhardt around my co-workers and friends who universally love it. I'm also proud to say the "WAT" meme has (hopefully permanently) entered my team's culture as we find it adequately sums up our feelings towards various bits of software that we either have to work with or maintain.
Half my team is away on business trips or sick leave at the moment…
I'm still building up to my article on how to properly mock calls to Rake's sh to facilitate testing of your Rakefile tasks but I haven't quite worked up the strength to trace the calls through the libraries and into Eigenclass wonderland. For the moment, I've got just enough fodder to write a bit about determinism in testing.
Since Friday, both a coworker and I have been hit by the same testing…
One of the things that is pushed here a lot is the necessity to make all new applications really production-ready before being deployed as a service. There is of course the adage "If it's not monitored, it doesn't exist" but beyond that there are a lot of other little fiddly details just to make something run. I'm talking about things like init scripts, logging etc.
It's very easy to get wrapped…
Last week I spent a bit of time scratching an itch I've had with doing maintenance. We use Nagios for our monitoring (without any addons, plugins or frontends) and scheduling downtime is a pain. Not only that, but deleting downtime is impossible short of screenscraping or (shock) actually clicking in the user interface. When I'm done doing maintenance I want to delete all the downtime (regardless…
If you saw my talk in Amsterdam or read my slides (or even work with me,
that if you pass a Ruby symbol through the Puppet external node classifier interface, through a parametrised class and into an ERB template, it will arrive (relatively) unscathed? OK not really, but the special undef Puppet symbol does.
ENC says: myvariable: :undef
Puppet interprets it as a standard undef (which is internally represented as Puppet::Parser::AST::Undef).
When passed through to an ERB template, it again arrives as the symbol form :undef…
I've recently been satisfying my need for more programming time and reading up on some of the aspects which really complete a good application, like adequate organisation, good design and reusability of library functions, error handling and of course testing. I've been using Cucumber for a while now in conjunction with Puppet which represents the BDD aspect, but haven't really delved into TDD…
I think I will call this week the last I use shell script as my primary go-to language. Yes, by trade I am a systems administrator but I do have a Bachelor of Computer Science degree and there is a dormant programmer inside me desperately trying to get out. I feel like shell script has become the familiar crutch that I go back to whenever faced with a problem, and that is becoming frustrating to…
I've been working recently on building up a more reliable, predictable Puppet infrastructure. I'm not a developer at heart, but I've been trying very hard to apply software development principles to what I do, since generally configuration management and software deployment is becoming more and more just another software problem. One of the end goals of the system I am building is to have a pretty…
I've been hacking on a small Ruby on Rails project to summarise and then prettily display pmacct traffic data. It's not an original idea by any stretch and in fact is directly inspired by a system we had at my last job. For obvious reasons I have to at least reimplement the idea myself, and while the original was written in Python using PostgreSQL, mine will be using Ruby and MySQL.
One small pain point I've already encountered is the lack of IP address functions in MySQL. PostgreSQL at least as far back as 7.4 has supported IP address functions and in the case when you need to be comparing addresses and summarising based on local subnets they are sorely…
I consider myself a failed programmer, having never really excelled at it during University and only really having come to terms with some of the concepts several years later. I've always liked programming but at some point years ago I decided I didn't want to be a programmer/developer so that was that. Since cementing myself in the realm of Systems Administration I've come to miss the programming…