[Dbix-class] [ANNOUNCE] A much faster DBIx::Class public trial - v0.08240

Peter Rabbitson rabbit+dbic at rabbit.us
Thu Feb 14 08:38:32 GMT 2013


Greetings

Almost 4 years[1] to the day of the first commits toying with the idea, 
the constructor rewrite / prefetch modernization project seems to be 
nearing completion. At least it passes enough tests to be deemed a CPAN
experimental release candidate, which *should not* eat your data.

So what is in this release? - First the notable parts of the changelog:

    * New Features / Changes
        - Rewrite from scratch the result constructor codepath - many bugfixes
          and performance improvements (the current codebase is now capable of
          outperforming both DBIx::DataModel and Rose::DB::Object on some
          workloads). Some notable benefits:
          - Multiple has_many prefetch
          - Partial prefetch - you now can select only columns you are
            interested in, while preserving the collapse functionality
            (collapse is now exposed as a first-class API attribute)
          - Prefetch of resultsets with arbitrary order
            (RT#54949, RT#74024, RT#74584)
          - Prefetch no longer inserts right-side table order_by clauses
            (massively helps the deficient MySQL optimizer)
        - Massively optimize codepath around ->cursor(), over 10x speedup
          on some iterating workloads.

Yes, you are reading this right - in addition to many of the structural 
improvements DBIC now is pretty damn fast. All while maintaining 100% 
backwards compatibility our users have grown to expect. Some numbers on 
non-threaded perl 5.16.2 (DBI: 1.623, DBD::SQLite: 1.37, RDBO: 0.804, 
DBIx::DM: 2.31 )

- Laurent Dami's benchmarks [2][3] from his talk at FPW2012 [4]
Numbers before (DBIC 0.08206): http://paste.debian.net/plain/234081
Numbers after (DBIC 0.08240-TRIAL): http://paste.debian.net/plain/234080

- A *part* of the RDBO benchmark suite [5](svn) (DBIC needs further
optimizations to win more of these benchmarks):
Numbers before (DBIC 0.08206): http://paste.debian.net/plain/234083
Numbers after (DBIC 0.08240-TRIAL): http://paste.debian.net/plain/234082

Pretty damn good I must say. Also keep in mind that none of this requires
any changes to your code. In fact if something works with 0.08206 but
*does not* function under 0.08240 - this is a bug and it needs to be fixed
before we go for GA.

Speed is good - but what about these new features? From here on you get
more fine-grained control over your selection. Prefetch[6] is now a thin
wrapper around join/+columns/collapse, and while you can use it like
you did before you no longer have to. Instead you can do things like
this [7], or this [8]. You should receive the result you would otherwise
assemble manually, but much much faster. And random order works too, just
as described in [9].

The changeset is massive, and does not easily yield to review. This is why
the dev team needs your help - PLEASE TEST THIS RELEASE. Really test it
stress it and bend it. Complain if a result doesn't match your
expectations. Complain if the performance seems to take a nosedive on a
specific workload. Whatever you are unhappy with - let us know.

If despite all warnings you want to subject your eyes to the workhorse
behind these improvements - check out [10] (but you have been warned :))
A thing to note is that the *really gnarly* code assemblers are all
nicely tucked behind an interface [11] with minimal dependency on DBIC.
In other words the collapser generator is ready to be extracted as a
standalone CPAN offering (the DBIC dev-team would like this *very*
much).

So that should be in for now - please give it a try and let us know
where/how we screwed things up. And if you are rather happy with
what you see - we can always use a ++ on metacpan ;P

Cheers

[1] Proof that benchmarking against walltime is a very bad idea ;)
[2] https://github.com/damil/compare-ORM
[3] http://lists.scsys.co.uk/pipermail/dbix-class/2012-July/010595.html
[4] https://www.youtube.com/watch?v=7NmJfuN3vUc
[5] https://rose.googlecode.com/svn/trunk/modules/Rose-DB-Object/t/benchmarks/
[6] https://metacpan.org/module/RIBASUSHI/DBIx-Class-0.08240-TRIAL/lib/DBIx/Class/ResultSet.pm#prefetch
[7] https://metacpan.org/source/RIBASUSHI/DBIx-Class-0.08240-TRIAL/t/prefetch/manual.t#L46
[8] https://metacpan.org/source/RIBASUSHI/DBIx-Class-0.08240-TRIAL/t/prefetch/manual.t#L226
[9] http://git.shadowcat.co.uk/gitweb/gitweb.cgi?p=dbsrgits/DBIx-Class.git;a=commit;h=fcf32d045
[10] https://metacpan.org/source/RIBASUSHI/DBIx-Class-0.08240-TRIAL/lib/DBIx/Class/ResultSource/RowParser/Util.pm
[11] https://metacpan.org/source/RIBASUSHI/DBIx-Class-0.08240-TRIAL/lib/DBIx/Class/ResultSource/RowParser.pm#L71



More information about the DBIx-Class mailing list