Archive for September, 2009

N+1

Posted on September 2, 2009, under coding, general.

I rarely write introspective or meta blog posts, in fact I rarely even use the word “I” on this blog (one of the habits you should develop as a manager or team-member is to use the word “we” almost all of the time) … so I hope you’ll forgive this brief, obnoxious, self-centered, round-up of the last year. I’ll try to smatter it with some useful observations to make up for it.

Today it’s one year since I started working for Amazon.com, and officially moved in to my new place and hence back to Dublin. Before that, I’d been spending most of my time in Amsterdam and traveling. By just September last year, I’d managed 78 flights in 2008. In the year since, it’s been a much more comfortable, and relatively conservative, 24. Some visits to Seattle, New York, New Orleans, Barcelona, Bristol and London keeping me occupied and worldly.

Me!

I’ve learned a lot in the last year, a lot more and in very different directions, than I expected. Amazon turns out to be even more interesting than I had anticipated, and getting thrown in at the deep end early on – with a new Amazon Web Service to help build and support – has been an amazing experience. Before I started, I naively thought that scale was mostly about understanding distributed systems problems and performant designs. It is about those things, but not mostly.

In reality (well, in my opinion) the hard problems of scale are the incredible attention to detail and testing it requires, because when your rapidly-changing code is handling billions of requests – even a tiny fault that is triggered on one in every million requests will get you paged in no time at all. I’m not sure that there is any other way to learn it than having to support it for real. I recommend it.

I’m getting a chance to work on stuff I enjoy, building some big things and making some important and critical code go really really fast. It’s always nice to feel that features exist – that the universe is different in some way – because of work you’ve been involved in. That should always be the measure of progress; “How has the universe been changed?”, everything else is meta-work.

I am learning technical things at Amazon; when I started I had never written a line of perl, now I’ve written entire perl frameworks, a very basic perl compiler, a bytecode analyzer, perform low-level code-reviews, and teach perl once a week. The experience has reinforced my opinion that the notion of “knowing programming language X” is itself a broken anti-pattern.

That said, Perl was comparatively difficult to learn. Coming to it with about 15 years programming experience in many languages, it still took about 5 weeks to become proficient in it to the point that I really understood all of the magic symbols, operators and patterns in front of me at a fundamental level. At a guess, it took just 2 weeks to reach the same level of proficiency in python.

But that learning is self-driven, the real learning experience at Amazon is how things are organised and managed. How a huge multinational can structure and orientate itself such that things can happen incredibly efficiently and quickly is fascinating to observe and participate in. It’s like getting a free MBA. I can recommend that too.

On the college front – it’s been a strange year, as it was also my final year doing my BSc. in Computer Science in Trinity College, Dublin. It was tough going, mostly just to stick through it, but I think worth it in the end. Somehow came out with a first class honours degree. I’ve decided not to progress with a part-time postgrad just yet – it seems like a good time to try not doing so much college for a change, but maybe next year.

In the last year, I’ve learned two new instruments, banjo – for the fun of it – to a level where I can now keep up in sessions – and piano – to a lesser level but I can now arrange and play relatively intricate pieces. My place has worked out very well, and a year on I’m still very happy to be living here, it’s ideal. Ups and downs in my personal life in the last year have been extreme, but I’ve learned a lot from those too and am mostly the better for it.

But now that I have some more free time, I have to admit I’m not entirely sure what to do with it. I’m finding things to do with my evenings, and catching up with friends properly, but still have itches to be a bit more productive. I haven’t done as much Apache stuff as I’d have liked in the last 2 years, but the urge to write a webserver from scratch using a more functional programming oriented approach (though not necessarily a lamba-calculus derived language) is strong … and also pointless.

This past year also saw the final, conclusive, victory of sense in the Irish E-voting debacle. In short; we were correct all along, and the system has been completely abandoned. I have to admit it was a bit fun to be able to gloat about it on the radio.

So now … what next for my free time … well maybe you can help. To further compound the impression of arrogance and self-obsession I have no doubt created, it’s like this; I’m pretty clever. I’ve got above average maths, analytical and linguistics skills. I’m an expert programmer/developer and builder of things. I’m a fairly decent musician and photographer with some basic sense of style, design and composition.

Oh god there’s more! I’m politically knowledgeable, and I know how to manipulate the political and press systems and strategise (with a proven track record through two lobby groups). I’ve worked for two start-ups, and I know how to make things happen. I’m a quick learner, and I know that when something is worthwhile, realistic and interesting I can throw a huge amount of effort and pragmatism right at it. I like doing cool things for free, and earn just enough to be able to.

Now all that said, to moderate things I’m also basically an introvert and can be pretty awkward and quiet around too many unfamiliar people, am about the last person you’d ever want to take to a bar (I don’t drink for a start), know close to nothing about popular culture and have read maybe only 4 or 5 fiction books in the last 10 years. So there’s a strong counter-argument that I’m also an illiterate anti-social bore to be kept in mind here too. But thankfully, not a terrible one.

But with all of that in mind, what does need doing? Especially in Ireland/Dublin – because it would be nice to involve meeting people and getting better at that whole social thing. Any ideas? the bigger the better. Any technical, political, or social gaps that really need filling? anybody need some help? What would you, or indeed, jesus do? There are a few ideas knocking around already, I’ll be sure to update when they are more concrete. How should we change the universe?