[Dbix-class] Caching DBIx-Class Fixtures

[Ovid]

--- Ash Berlin <ash at cpan.org> wrote:

> First thing I'd suggest is to look at DBIx::Class::Fixtures and see  
> how close it comes to your needs. (I'm guessing you haven't looked at
 
> it yet judging by your useperl post)

You're right, I didn't know about that.

Unfortunately, I'm having trouble with it.  I have something like this:

  use PIPTest::Setup;
  use TestPM::All;

  my $setup  = PIPTest::Setup->new;
  my $schema = $setup->init_db;
  TestPM::All->load($schema);

That particular snippet loads the 11 fixture classes we have so I
figured I would test with that.  I managed to successfully get
something dumped out with this:

  my ($config_dir, $fixture_dir) = qw( 
    t/var/configs t/var/fixtures 
  );

  my $fixtures = DBIx::Class::Fixtures->new({
      config_dir => $config_dir,
      debug      => 0,
  });
  $fixtures->dump({
      all       => 1,
      schema    => $schema,
      directory => $fixture_dir,
  });

And that scattered 927 (holy crap!) fixture files around in the
$config_dir directory.  Right off the bat, this looks problematic. 
With 14,000 tests and growing, I don't want each test program needing
fixtures to have to read in a separate file for every object in a
fixture, parse it and execute the code for it.  I/O issues are already
killing us.  For those (admittedly few) tests which to read in all
fixtures, I don't want them to have to read in 927 files, parse 'em,
and shove 'em in the database.  Heck, with only 11 fixture classes, it
looks like they'll each have to read an average of 84 fixture files to
build the fixture.  I want them to read *one*.

Then it gets worse.  In reading the populate method, it sort of looks
like we drop all tables and have to recreate and populate them each
time fixtures are used.  This model breaks one of our very aggressive
performance strategies (http://tinyurl.com/55zlsv).  We don't drop
databases or tables (see the link if you're curious) because it slows
our test suite down even more.

I need this to be FAST.  FAST, FAST, FAST.  So I would *much* prefer
that a single fixture class -- written out as proper DBIx::Class code
which we already know -- can be marked as "cacheable" and I can cache
all of its SQL into a file.  So "Test::Fixture::WakingTheDead" could
generate a test_fixture_waking_the_dead.sql file which gets read in
once and bypasses all all DBIx::Class code.

Cheers,
Ovid

--
Buy the book  - http://www.oreilly.com/catalog/perlhks/
Personal blog - http://publius-ovidius.livejournal.com/
Tech blog     - http://use.perl.org/~Ovid/journal/