adrift on a cosmic ocean

We have an internal service that takes requests of the form something like /foo/{ID}/bar which basically is expected to generate data of the form "bar" about the ID entity within the collection "foo". Real clear! Because this service has a bunch of similar routes that have a resource identifier as part of the request path, we use the Gorilla Mux package. There are a lot of different approaches to…

One of the notably lacking metrics from CloudWatch has been the current and previous AutoScaling group sizes - in other words, how many nodes are in the cluster. I've worked around this by using the regular EC2 APIs, querying the current cluster size and the desired size and logging this to Graphite. However, it only gives you the current values - not anything in the past, which regular CloudWatch…

As I mentioned in my last post, I did Getting Tough - The Race in Rudolstadt yesterday. Check out that post for a link to a first-person-style video shot at last year's event. Having watched that video, and obsessively scoured the internet for photos and any other tidbits of information, I expected that I knew what I was getting myself into. I have to say first up, whoever "Iron Mike" (poster of…

I've just created a new category on this blog - Health - and recategorised one of the older articles I wrote a while ago. Being a confirmed computer geek, fitness has never really been very high on my priority list until I got into kayaking when I was about 24 or so. I managed to keep that up for several years and ended up quite fit - completing a 111km kayaking marathon three times in consecutive…

Way back in 2010 I wrote about my acquisition of a Zotac HD-ID11 ZBox "home theatre PC". Since then it has served me fairly well, although I suspect I ran it too hot for the last year or two. After several Ubuntu and XBMC upgrades I noticed that it tended to idle pretty hot despite not really doing much most of the time, and was able to ignore this until it started restarting itself frequently…

I've recently been building a website for my wife using relatively modern tools - lots of client-side Javascript and relatively minimal CSS and HTML. A recent-ish email alerted me to the existence of a Berlin-based Famous framework meetup group, which initially made no sense to me, but after I checked out the actual framework website my interest was piqued. I've got next to no website building…

A friend drew my attention to the fact that it has been around three months since my last post on this blog. Sadly, life has gotten in the way of my more technical posts - there have been a lot of things going on preventing me from writing more frequently. Vacation, how busy I am at work with various new responsibilities and a lot of features to deliver, team-mates to train up, learning some new…

While I was implementing a small, naive log aggregation tool I had a moment to consider the type of compression I wanted to use on the log files at rest. The main implication this has outside of the efficiency of compression and how much space the files will take up is how they can be used once stored. If you are using tools like gzip/gunzip with awk or other simple command-line tools, or even…

A few days ago I wrote about logging with Upstart and the various implications that has if you want more frequent log rotation than the defaults provide, or do something other than rotation with the files that logrotate cannot do. There doesn't seem to be any good canonical source of information about how to exclude certain files from the logrotation so I thought I'd make a small note about it…

I have a couple of services I own that run in EC2 that, sadly, still lack any log aggregation. In-house we have our own system for this, that technically can also run in EC2 but for a variety of reasons I consider it overkill for my use case in EC2. Basically, the solution I am after is to do the minimum amount of work to see these logs aggregated to an S3 bucket, making them available for…