[Dbix-class] Performance Issue

[Peter Rabbitson]

On Tue, Feb 21, 2012 at 08:49:31AM -0600, Bill McCormick wrote:
> I'm not surprised that it takes more time. I am, however, surprised that  
> it takes more than 0.5 sec. 

Near-second loadtimes are *nothing* unusual in today's perl ecosystem. I
do not like this fact myself, and try to keep DBIC as lean as possible, so
I am *very* interested in your cooperation to enumerate the hotspots in
your case and possibly even fix oversights in DBIx::Class.

> That fact that it takes more than 1.0 sec  
> really has me scratching my head. I was hoping to run this with tt2 in a  
> cgi web application on an a small system (running Squeeze) that's closer  
> to an embedded device than a server class machine or even a desktop  
> system. I'm using thttpd and SQLite as well, since they are lightweight  
> and have small footprints.

DBIC does a lot of stuff to optimize for long-running, large-dataset apps.
While we probably can take care of the startup costs, keep in mind that
we deliberately trade memory for speed where possible, so if your embedded
environment has limited memory amounts - you will need to profile this
aspect as well.

> Speed *is* an issue, however, I need to keep things as lightweight as  
> possible. So the number of module dependencies is also an issue,  
> especially if I won't be needing them.  So, for example, when I install  
> DBIx::Class::Schema::Loader ... and it depends on some Lingua modules  
> ... which depend on some Text::German and Snowball::Swedish modules ...  
> I think I'm looking at a lot of bloat.

You are looking at English (the language) being bloated and hard to work
with. Schema::Loader *must* be able to singularize/pluralize table names
which it does with a ton of natural language processing tools. However
this is irrelevant to your case, as DBIx::Class::Schema::Loader is not
(and never has been) a dependency of DBIx::Class, so I am not sure what
is the exact complaint here...

> I only have a 2G flash drive to  
> work with, and I need to save as much of that as possible for data 
> storage.

An average distribution is about 1m unpacked, I doubt the amount of modules
used (even without you cleaning up your clearly bloated dep-tree) will not
even make a dent in the 2G available.


> If I cannot resolve these issues, then I don't think I should use  
> DBIx::Class.

If you are serious about getting these issues diagnosed - please dump your
MyApp::Schema (and all result classes) somewhere public, and also generate
and provide a full profile of the test script run. Something like:

NYTPROF=clock=2 perl -d:NYTProf -I<inclist> ./recipe-test-DBIx.pl
nytprofhtml -l<same inclist like above, important>

The resulting nytprof directory will contain both your source and a very
detailed profile of all slowdowns and tight places. Looking forward to
these datapoints, so we can help you further.

Cheers!