[Catalyst] Catalyst vs Rails vs Django Cook off

Christopher H. Laco claco at chrislaco.com
Thu Nov 16 14:34:24 GMT 2006


Carl Franks wrote:
> On 16/11/06, Cory Watson <jheephat at gmail.com> wrote:
>> On 11/16/06, Carl Franks <fireartist at gmail.com> wrote:
>> > On 16/11/06, catalyst.20.chsg at spamgourmet.com
>> > <catalyst.20.chsg at spamgourmet.com> wrote:
>> > >
>> > > Essentially, according to his test, which doesn't take into account
>> > > ORM performance, Rails & Django knock the socks of Catalyst.
>>
>> <snip>
>>
>> > The first thing I noticed was that the content length of the document
>> > served by catalyst was longer than that served by rails.
>> > He doesn't seem to have tried very hard to test "apples for apples"
>> (his words)
>> >
>> > Also see the very good comment by "JayK" as to why it's not a very
>> > good real-world test at all.
>> > http://letsgetdugg.com/view/Catalyst_vs_Rails_vs_Django_Cook_off
>> >
>> > I'm not saying Catalyst's performance couldn't be improved, or that
>> > it's not slower than Rails - just that a bad benchmark is worthless.
>>
>> I agree with all your points Carl. I have not been present in teh IRC
>> for a few days to see any discussions related to this thread.  I'm
> =

> (me either)
> =

>> sure some optimizations were discussed and some things will be
>> implemented because of it.  So with the precondition that I haven't
>> kept up with the state of affairs I'd like to thank victori for
>> spending his time and effort to create _something_.  It's more than
>> his naysayers have to done to show us how fast Catalyst is.  I
>> respectfully suggest that those who criticize his work should use
>> their energies to /improve/ his test rather than merely dismissing it
>> as worthless.  Using his code as a base, couldn't one create a test
>> that was more fair?  Then someone would have a test that shows results
>> that are more 'real' and give potential users more information with
>> which to make a decision.
> =

>> From the off-list discussion I've already had, I know my use of the
> word 'worthless' will haunt me ;)
> =

> If a benchmark reveals something in the framework core which could be
> optimised, then that's great.
> If it helps teach more effective idioms, or highlights something that
> shouldn't be used, then that's great.
> But other than that, I don't think any application benchmark will have
> much worth other than for that specific application.
> =

> If I wanted to serve static pages (as the benchmark did), I wouldn't
> use a framework and then pipe them through TT.
> The reason I use a framework, is because I want to write a big
> application with lots of pages, and have things like sessions, ORM,
> templates.
> =

> I don't see /how/ the benchmark can be improved. Once you start
> getting into something that complicated, all you're testing is the way
> 1 person writes the application in perl compared to how they write it
> in ruby.
> Someone else might use a different session storage-backend, which
> would have different results, and your 'fastest' framework now isn't.
> =

>> Catalyst doesn't have to be the fastest in such a test.  That's
>> probably never been the One True Goal of the core devs.  But providing
>> people with information as to why Catalyst is good (or bad) should be
>> high on the list.
> =

> Carl

/me puts on flame suit.

I agree overall. However...

I think the fact still remains that new end users will see three
frameworks [all of which were destined for serving more than static
pages] where 2 of them serve the static content fast, and one doesn't
[Catalyst].

Regardless of whether the test is 'real world', and regardless of
whether the frameworks 'were meant to serve more complicated things',
Catalyst is slower in this instance. All things being unequal, if I tell
my boss we have 3 frameworks to choose from, and one is flexible, and
the others are fast, he's going to choose fast every time...even knowing
the testing may be faulty. Yes, I know better. He probably does too. But
that's how the world works.

I always fall on the side of the non majority it seems, and this is
another example. [The list, not you specifically] Stop being defensive
that the test is bogus. It's not. It shows that in one circumstance,
Catalyst is sadly slow. Let's fix that. Explaining why the test may be
invalid, or why it's bunk still won't change that fact that in this
circumstance, it sucks.

/me removes suit and goes back to writing tests

-=3DChris

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: OpenPGP digital signature
Url : http://lists.scsys.co.uk/pipermail/catalyst/attachments/20061116/6637=
05f4/signature.pgp


More information about the Catalyst mailing list