[Dbix-class] DBIx::Class::Fixtures speedup

[Drew Taylor]

On Wed, Apr 23, 2008 at 2:43 PM, Matt S Trout <dbix-class at trout.me.uk> wrote:
> On Wed, Apr 23, 2008 at 01:03:22PM +0100, Drew Taylor wrote:
>  > On Wed, Apr 23, 2008 at 12:43 PM, luke saunders
>  > <luke at shadowcatsystems.co.uk> wrote:
>  > > On Wed, Apr 23, 2008 at 12:14 PM, Drew Taylor <taylor.andrew.j at gmail.com> wrote:
>  > >
>  > > > I'm using Fixtures in my app, and am overall pleased with it. But I
>  > >  >  now have a couple thousand fixture items which are imported, and
>  > >  >  that's taking upwards of 1min+. Out of curiosity I commented out the
>  > >  >  creation of the tmp_fixtures_dir and just read from fixtures_dir. I
>  > >  >  shaved my populate() time in approximately half. What's the reasoning
>  > >  >  for the tmp dir? Safety in case anything goes wrong? For me, it's just
>  > >  >  a lot of wasted IOs and time. :-) Luke, any objections if I remove
>  > >  >  this?
>  > >  >
>  > >
>  > >  The tmp directory is used so that if someone dumps while you are
>  > >  populating the populate is unaffected. Since this is an issue for some
>  > >  people I'd suggest making the tmp directory thing configurable,
>  > >  probably in the call to ->new. Normally if the fixtures are on a local
>  > >  drive using a tmp directory doesn't take too long so I'd prefer to
>  > >  keep it the default.
>  >
>  > Ahhh, that makes sense. The copy doesn't take _that_ long (I'm
>  > guessing around 30-60 seconds for my current record set), but when I
>  > want to do quick, successive test runs it adds up rather quickly.
>
>  What if you used link() instead of a copy?
>
>  Or just make the dump code make the tmp dir, and populate open a file handle
>  to everything in advance so you don't have to worry about paths changing
>  under you?

As I understand your suggestion, dumping to the tmp dir wouldn't work
for me because I make a dump only every so often, but populate from it
many times per day as I run my tests. I just want to zip through the
existing files w/o any IO other than reading/slurping the .fix files.

Drew
-- 
----------------------------------------------------------------
 Drew Taylor * Web development & consulting
 Email: drew at drewtaylor.com * Site implementation & hosting
 Web : www.drewtaylor.com * perl/mod_perl/DBI/mysql/postgres
 ----------------------------------------------------------------