[Catalyst] CatalystX::JobServer (anyone use it?)

Bill Moseley moseley at hank.org
Fri Jun 8 18:53:59 GMT 2012


On Tue, Jun 5, 2012 at 11:29 PM, Drew Taylor <drew at drewtaylor.com> wrote:

> On Tue, Jun 5, 2012 at 8:25 AM, Seth Daniel <catalyst at sethdaniel.org>
> wrote:
> > I found CatalystX::JobServer mentioned in a thread from about a year
> > ago.
>

This is a bit off topic, and maybe not a very useful question (as I asked
on RabbitMQ list w/o a response), but this thread was timely.   It's a
general question about messaging in web apps.   It's not really about any
specific messaging system.

How do you, or perhaps,  do you handle the possibility of message loss when
using messaging systems?

We try and "design for failure" assuming any part of our system can fail.
 So, that means an entire queue could be lost.  But, in some cases we don't
want[1] to lose any messages (which in this case are tasks to be
completed).   It's very possible we could have thousands of messages in a
queue any time.

As a simple example, in our web app we have a way for a user to generate a
report and have it emailed to them.  This happens in the background on a
separate server pool.  So, the web app queues a message to trigger this
task.

If the job is critical what I have done is track the state of the job with
a database table that has start time and a retry count.   Then I have cron
look for stale jobs and re-queue after some timeout and before retry count
exceeds some value.

The problem with this approach is 1) I don't really know if the message was
lost or if the queue is just backed up, and 2) it's quite a bit of overhead
and additional complexity having to also maintain state of the job.

Other than (or in addition to) using an HA setup like
RabbitMQ's<http://www.rabbitmq.com/ha.html> is
there a design pattern you use to handle the situation of a message or
entire queue being lost?




[1] perhaps more accurately, we *think* we can't lose any messages.


-- =

Bill Moseley
moseley at hank.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.scsys.co.uk/pipermail/catalyst/attachments/20120608/0d26e=
5f5/attachment.htm


More information about the Catalyst mailing list