[Dbix-class] Soliciting Design Input for an Object Caching Module

[Matt S Trout]

On Sat, May 12, 2007 at 03:31:12AM +0100, Dave Cardwell wrote:
> Hello list.
> 
> I'd like to solicit your thoughts on the appropriate architecture of a
> DBIx::Class module for caching objects, similar to the functionality
> provided by Data::ObjectDriver [1].
> 
> mst has made me aware that there are several existing, private
> implementations so I would be particularly interested in those
> developers' input on authoring a solution for general release.

I think we actually want to look at more than one layer of caching :)

A resultset plugin that allows you to share the cached result of a particular
query across processes would be useful.

The D::OD cache functionality requires two things -

(1) An ability to cache fetches by PK
(2) An ability to effectively expire caches on change

I think we can probably achieve both by indirecting both result and resultset
operations via the resultsource object - it's the last thing that really
understands the PKs, uniques etc. (the storage object doesn't really and
I think shouldn't). That would then allow the source to fill caches "on the
way through" - and also to make simple fetches only fetch the primary key
and then fill results either from cache or via a 'pk IN (...)' second select
(which is admittedly a gamble that you mostly hit cache but the idea here is
that you -do- mostly hit cache). Plus when updates occur the resultsource can
clear the caches (and in some cases update them) appropriately.

The second thing we want to steal from D::OD is the ability to distribute
fetches across partitioned databases. I'm currently torn as to whether this
is better happening at the source or storage level - I -think- we probably
want to put this logic in the resultsource as well, since the choice of
partition is linked tightly into a level of data definition that again the
storage doesn't need to know about.

My thought would be to have a composite source that talks to multiple
underlying source objects, one per partition, and for those to refer back to
a partition schema object with an appropriate storage object.

What do people think?

-- 
      Matt S Trout       Need help with your Catalyst or DBIx::Class project?
   Technical Director    Want a managed development or deployment platform?
 Shadowcat Systems Ltd.  Contact mst (at) shadowcatsystems.co.uk for a quote
http://chainsawblues.vox.com/             http://www.shadowcatsystems.co.uk/