[Catalyst-dev] Paging Ash Berlin

[John Goulah]

On Fri, Oct 10, 2008 at 6:17 PM, J. Shirley <jshirley at gmail.com> wrote:
> On Fri, Oct 10, 2008 at 2:22 PM, Ash Berlin <ash_cpan at firemirror.com> wrote:
>>
>> On 10 Oct 2008, at 20:46, J. Shirley wrote:
>>
>>>
>>>
>>> I really really really don't want to start a (very much so offtopic)
>>> flamewar, but I would like to get a discussion going about this versus
>>> TheSchwartz.  It seems roughly similar (at least in function).
>>
>> TBH one of the reasons I avoided TheSchwartz was that I couldn't work out
>> what was going on. I did feel kinda iffy about wheel re-invention here, but
>> there was something about TheSchwartz when i looked at that didn't sit well
>> with me. Can't remember what it was anymore.
>>
>>>
>>>
>>> Here are the features that TheSchwartz has that I didn't see in
>>> MooseX::JobQueue (and yes, please name it something other than
>>> MooseX::JobQueue)
>>>
>>> The following are handled because of Data::ObjectDriver, but want to
>>> list them as features anyway:
>>> 1. Partitioning of jobs in the database
>>> 2. Built-in replication handling
>>
>> Not really sure what these two things are? Shouldn't replication be done at
>> a DB level? Partitioning - as having jobs live in two different tables/DBs?
>> If so then App::JobQueue (lets call it that for lack of a better
>> alternative) does that.
>>
>
> Well, I mean horizontal partitioning.  So, automatic partitioning
> based on some algorithm (like "if job->id % 2 => use this cluster").
>
> I didn't realize it did that... couldn't find that bit.
>
> As far as replication goes, DBIC handles some replication schemes but
> there isn't the same support that D::OD has.  I'm not championing
> D::OD at all here, I prefer DBIC for all things; however D::OD has a
> lot of code to support multiplexing and caching that DBIC hasn't
> culled yet.
>
> So, while replication happens at the database layer, the interactions
> there require client side behaviors.  Such as reading from slaves,
> write to masters, etc.  DBIC already has basic slave/master support
> but without support for slave read-delay (which is unfortunately
> application specific in most cases) App::JobQueue won't have that...
>
> Which means worse replication support than TheSchwartz.


This is correct.  When it was put in production on several servers
under a replicated DBIC things went a bit haywire with the job locking
I believe when slaves got delayed and we had to point all queries at
the master.  Otherwise it does scale beautifully to multiple machines.
 I wonder what the best solution is here.

John