[Catalyst] Let's port Twitter to Perl

[Chris Carline]

On Mon, May 5, 2008 at 3:29 PM, John Napiorkowski <jjn1056 at yahoo.com> wrote:
>  I heard Twitter was abandoning RoR.  It would be the coup of the year if we could bring them to Perl, but from what I understand it's not even on the consideration list.  So why not give them a little help?  We should have a hackathon to port Twitter to Catalyst using all our most timesaving and advanced tech, like Moose, DBIC, etc.

Knock yourselves out, but I remain to be convinced that the
scalability of RoR is anything but a red herring. The bottom line is
that the design decisions you make at the very beginning of a project
tend to be the limiting factor in this area. Sure, your choice of
implementation language might not be the fastest, but is it the
bottleneck?

It strikes me that an awful lot of what makes Twitter 'Twitter' are
the heavy-lifting processes that caching and implementation-language
choice have very small amounts of influence over. Without fully
understanding those, there's the distinct possibility of falling into
the same traps.

One thing I do know though: As much as OR-mapping systems are lovely
and all for the most part, that's the first part I'd ditch if I was
all out for screaming performance. Yes, it's possible to code around
this by binning and balancing and other techniques, but it's a
potentially enormous tradeoff between ease of development vs. sheer
unadulterated speed (where all the clever binning, balancing and other
techniques still apply).

So to conclude: Twitter is a typical prototype that didn't scale. No
shame in that. Mitigation can only get you so far, and it's really,
REALLY tough to migrate and swap out something as complicated whilst
making sweeping changes and still maintaining uptime.

None of this is intended as a slight on any of the magnificent work
that's been done, it's insanely good, and I'm happy to stick my hand
out, wave it around and say 'Thanks!' to everyone's CPAN code I rely
on for my day-to-day job.

But hubris, however well-intentioned, is still hubris unless you pay
particular attention to the lessons learned.

This post is not intended to be pooh-poohing or flamebait, I'd
personally love to see some kind of scalable twitteresque application
to come out of this, it's just that I've yet to see anyone point out
the enormity of such a task. :-)

Chris