[Catalyst] [OT] what would constitute a sensible set of benchmarks?

[Daniel McBrearty]

so a set of benchmarks would give you the chance to show that
TIMTOWTDI, and the trade offs that exist between them. That would be
pretty interesting to someone trying to compare frameworks.

That's where having simple tests that exercise one aspect of the
framework in isolation, as far as is possible, might have advantages.
You see the differences between techniques, you see the effect they
have, you have the opportunity to try to extrapolate what that
actually means in terms of your app.

At least, that's a technique that has served well in other branches of
engineering - you reduce complexity to it's simplest cases, then wotk
back .... maybe this is fundamentally different and that can't work,
but at the moment I don't see why. Or any documented attempts to do it
that have clearly failed ...


On 1/16/07, Robert 'phaylon' Sedlacek <rs at 474.at> wrote:
> Perrin Harkins wrote:
> > On Mon, 2007-01-15 at 13:24 +0100, Robert 'phaylon' Sedlacek wrote:
> >
> >> Well, templating benchmarks maybe, but for an ORM I just have the
> >> feeling the larger factor is how you use it, not which.
> >
> > This is true, but the SQL generated by an ORM can have a big effect on
> > performance.  I usually hand-code the parts where I need the most
> > performance, but many people will just rely on their ORM and hope for
> > the best, and that can vary quite a bit between implementations (e.g.
> > deleting multiple rows in one statement vs. thousands).
>
> And you can do both in DBIC. That's why benchmarking is so hard,
> TIMTOWTDI, which one will you benchmark? Same with Catalyst. For my
> apps, I use uri_for massively, some people don't, which way are we going
> to benchmark? Same with controller base classes vs action classes vs
> external modules.
>
> IMHO you can only really benchmark developers together with their
> framework of choice.
>
> --
> # Robert 'phaylon' Sedlacek
> # Perl 5/Catalyst Developer in Hamburg, Germany
> { EMail => ' rs at 474.at ', Web => ' http://474.at ' }
>
> _______________________________________________
> List: Catalyst at lists.rawmode.org
> Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@lists.rawmode.org/
> Dev site: http://dev.catalyst.perl.org/
>


-- 
Daniel McBrearty
email : danielmcbrearty at gmail.com
www.engoi.com : the multi - language vocab trainer
BTW : 0873928131