[Catalyst-dev] Paging Ash Berlin

[J. Shirley]

On Fri, Oct 10, 2008 at 2:22 PM, Ash Berlin <ash_cpan at firemirror.com> wrote:
>
> On 10 Oct 2008, at 20:46, J. Shirley wrote:
>
>>
>>
>> I really really really don't want to start a (very much so offtopic)
>> flamewar, but I would like to get a discussion going about this versus
>> TheSchwartz.  It seems roughly similar (at least in function).
>
> TBH one of the reasons I avoided TheSchwartz was that I couldn't work out
> what was going on. I did feel kinda iffy about wheel re-invention here, but
> there was something about TheSchwartz when i looked at that didn't sit well
> with me. Can't remember what it was anymore.
>
>>
>>
>> Here are the features that TheSchwartz has that I didn't see in
>> MooseX::JobQueue (and yes, please name it something other than
>> MooseX::JobQueue)
>>
>> The following are handled because of Data::ObjectDriver, but want to
>> list them as features anyway:
>> 1. Partitioning of jobs in the database
>> 2. Built-in replication handling
>
> Not really sure what these two things are? Shouldn't replication be done at
> a DB level? Partitioning - as having jobs live in two different tables/DBs?
> If so then App::JobQueue (lets call it that for lack of a better
> alternative) does that.
>

Well, I mean horizontal partitioning.  So, automatic partitioning
based on some algorithm (like "if job->id % 2 => use this cluster").

I didn't realize it did that... couldn't find that bit.

As far as replication goes, DBIC handles some replication schemes but
there isn't the same support that D::OD has.  I'm not championing
D::OD at all here, I prefer DBIC for all things; however D::OD has a
lot of code to support multiplexing and caching that DBIC hasn't
culled yet.

So, while replication happens at the database layer, the interactions
there require client side behaviors.  Such as reading from slaves,
write to masters, etc.  DBIC already has basic slave/master support
but without support for slave read-delay (which is unfortunately
application specific in most cases) App::JobQueue won't have that...

Which means worse replication support than TheSchwartz.

>>
>>
>> General stuff:
>> 3.  Doesn't rely on POE; just has its own loop.  I can see the
>> benefits to using POE but it seems like unnecessary overhead for a
>> jobqueue that has workers that should simply do work and nothing else.
>> It seems the scripts to control workers would "have more" in them
>> (not sure if this is a bad thing, just want to start a discussion on
>> pros/cons)
>
> Code reuse? An excuse to learn POE? Doesn't need to use it at all, just
> does.
>

Ok, good enough :)

> It could be quite easily made to not work processes to run certain classes
> of small jobs due to POE, which is a nice benefit i think. /me did not just
> think of that on the spot >_>
>
>>
>> 4.  Why does the job get a ResultSet?  This seems like a very odd and
>> strict tie into DBIC that doesn't seem to make much sense, tbh.  As
>> John said, tying it to DBIC limits its applications.  Could the job
>> just not get a HashRef inflated struct and iterate over it without
>> objects?  Performance hits should be limited as much as possible, IMO.
>
>
> Yes, being tied into DBIC is def something that needs to be addressed at one
> point or another. For now its not a huge show stopper tho.
>
> But yes, overall I am aware that this is Yet Another Job Queue.
>


Well, don't get me wrong... I -want- to use it, but I have working
TheSchwartz code right now so just wondering if I want to switch over.
 If it weren't tied to DBIC but had a compat layer to use D::OD I
could, in theory, test them side by side.

Thanks for the response... other folks using it care to chime in?
Particularly on the POE bits... I'm starting to think that the POE
bits may be nice for an admin interface to push/pull status...

-J