[Catalyst] Catalyst vs Rails vs Django Cook off

Christopher H. Laco claco at chrislaco.com
Thu Nov 16 14:59:06 GMT 2006


Carl Franks wrote:
> On 16/11/06, Christopher H. Laco <claco at chrislaco.com> wrote:
>> Carl Franks wrote:
>> > On 16/11/06, Cory Watson <jheephat at gmail.com> wrote:
>> >> On 11/16/06, Carl Franks <fireartist at gmail.com> wrote:
>> >> > On 16/11/06, catalyst.20.chsg at spamgourmet.com
>> >> > <catalyst.20.chsg at spamgourmet.com> wrote:
>> >> > >
>> >> > > Essentially, according to his test, which doesn't take into
>> account
>> >> > > ORM performance, Rails & Django knock the socks of Catalyst.
>> >>
>> >> <snip>
>> >>
>> >> > The first thing I noticed was that the content length of the
>> document
>> >> > served by catalyst was longer than that served by rails.
>> >> > He doesn't seem to have tried very hard to test "apples for apples"
>> >> (his words)
>> >> >
>> >> > Also see the very good comment by "JayK" as to why it's not a very
>> >> > good real-world test at all.
>> >> > http://letsgetdugg.com/view/Catalyst_vs_Rails_vs_Django_Cook_off
>> >> >
>> >> > I'm not saying Catalyst's performance couldn't be improved, or that
>> >> > it's not slower than Rails - just that a bad benchmark is worthless.
>> >>
>> >> I agree with all your points Carl. I have not been present in teh IRC
>> >> for a few days to see any discussions related to this thread.  I'm
>> >
>> > (me either)
>> >
>> >> sure some optimizations were discussed and some things will be
>> >> implemented because of it.  So with the precondition that I haven't
>> >> kept up with the state of affairs I'd like to thank victori for
>> >> spending his time and effort to create _something_.  It's more than
>> >> his naysayers have to done to show us how fast Catalyst is.  I
>> >> respectfully suggest that those who criticize his work should use
>> >> their energies to /improve/ his test rather than merely dismissing it
>> >> as worthless.  Using his code as a base, couldn't one create a test
>> >> that was more fair?  Then someone would have a test that shows results
>> >> that are more 'real' and give potential users more information with
>> >> which to make a decision.
>> >
>> >> From the off-list discussion I've already had, I know my use of the
>> > word 'worthless' will haunt me ;)
>> >
>> > If a benchmark reveals something in the framework core which could be
>> > optimised, then that's great.
>> > If it helps teach more effective idioms, or highlights something that
>> > shouldn't be used, then that's great.
>> > But other than that, I don't think any application benchmark will have
>> > much worth other than for that specific application.
>> >
>> > If I wanted to serve static pages (as the benchmark did), I wouldn't
>> > use a framework and then pipe them through TT.
>> > The reason I use a framework, is because I want to write a big
>> > application with lots of pages, and have things like sessions, ORM,
>> > templates.
>> >
>> > I don't see /how/ the benchmark can be improved. Once you start
>> > getting into something that complicated, all you're testing is the way
>> > 1 person writes the application in perl compared to how they write it
>> > in ruby.
>> > Someone else might use a different session storage-backend, which
>> > would have different results, and your 'fastest' framework now isn't.
>> >
>> >> Catalyst doesn't have to be the fastest in such a test.  That's
>> >> probably never been the One True Goal of the core devs.  But providing
>> >> people with information as to why Catalyst is good (or bad) should be
>> >> high on the list.
>> >
>> > Carl
>>
>> /me puts on flame suit.
>>
>> I agree overall. However...
>>
>> I think the fact still remains that new end users will see three
>> frameworks [all of which were destined for serving more than static
>> pages] where 2 of them serve the static content fast, and one doesn't
>> [Catalyst].
>>
>> Regardless of whether the test is 'real world', and regardless of
>> whether the frameworks 'were meant to serve more complicated things',
>> Catalyst is slower in this instance. All things being unequal, if I tell
>> my boss we have 3 frameworks to choose from, and one is flexible, and
>> the others are fast, he's going to choose fast every time...even knowing
>> the testing may be faulty. Yes, I know better. He probably does too. But
>> that's how the world works.
>>
>> I always fall on the side of the non majority it seems, and this is
>> another example. [The list, not you specifically] Stop being defensive
>> that the test is bogus. It's not.
> =

> I agree with everything up to here.
> =

>> It shows that in one circumstance,
>> Catalyst is sadly slow. Let's fix that.
> =

> Matt has just pointed out that Cat's optimised for large applications
> with lots of paths, and for flexible programming.
> Only fix it if that doesn't compromise this, which is more important
> than looking good in one flawed benchmark.

Agreed.

> =

>> Explaining why the test may be
>> invalid, or why it's bunk still won't change that fact that in this
>> circumstance, it sucks.
> =

> Catalyst sucks because it doesn't come with a pony, so let's hope the
> next benchmark to hit the web doesn't benchmark against that ;)

Google has a pony, why can't I?
http://googleblog.blogspot.com/2006/10/yes-you-can-have-pony.html

> =

>> /me removes suit and goes back to writing tests
> =

> Um, shouldn't you leave the suit on until you've received the replies?

Nope. That's what filters to Trash are for. :-P

> I hope it wasn't needed anyway
> =

> Carl
> =

> _______________________________________________
> List: Catalyst at lists.rawmode.org
> Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@lists.rawmode.or=
g/
> Dev site: http://dev.catalyst.perl.org/
> =

> =



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: OpenPGP digital signature
Url : http://lists.scsys.co.uk/pipermail/catalyst/attachments/20061116/647e=
05c6/signature.pgp


More information about the Catalyst mailing list