[Catalyst] Handling long-running processes (was Re: Logging is not
immediate)
Toby Corkindale
toby.corkindale at strategicdata.com.au
Tue Oct 19 01:54:55 GMT 2010
On 19/10/10 11:05, Bill Moseley wrote:
>
>
> On Mon, Oct 18, 2010 at 4:46 PM, Ken Beal <KBeal at crosscountry-auto.com
> <mailto:KBeal at crosscountry-auto.com>> wrote:
>
> The issue is that when I call $c->log(), it doesn’t output anything
> until the “URL call” completes. This makes it difficult to watch a
> long-running process, because I don’t see anything until it’s done,
> and I don’t know if it’s hung up on something because I can’t see
> the log output.
>
>
> maybe try this: $c->log->_flush;
>
> Or try: warn "I'm stuck in a loop and the web user is wondering why I'm
> taking so long and will likely hit reload any second now!\n";
As a follow-up to Ken's message..
If you have long-running processes in a web server, then I suspect you
are Doing It Wrong.
(Or, naybe you're not, and have some kind of RPC system under Catalyst,
in which case ignore what I'm about to say.)
Can I suggest you look at a different model of serving these
long-running things to the web users? Consider having them hit a page
which says "Your request is being processed..", which then kicks off the
actual work in another process, and returns a token of some sort to the
user's browser. (eg. Cookie or URI parameter)
You then have the browser either reload the page (including the token
mentioned above), or better yet, use javascript to do it in the
background. You have two options here - first option is for this request
to get hung on the server, waiting for the process to complete, or you
can have it return quickly with the status or progress info to display
in the meantime.
Once this background request detects the long-running process has
completed, it then directs the browser to reload, and collects the final
information.
This has several advantages.
1) If the user keeps hitting reload, you can detect their work token,
and avoid starting more long-running processes.
2) The user doesn't get stuck with a blank screen and a loading....
status bar. Instead they can get progress info, or at least a message
saying "we're working on it, hang in there..."
3) You don't have your web server tied up with long-running processes,
holding open sockets and using memory.
4) Logging for your long-running processes is independent of your web
server messages.
So the URLs the user would see would be something like:
/report/megasize
# server kicks off background process, redirects user onto..
/report/megasize?token=d11a1658f614401782e8
# which says "please wait while we prepare your report".
# The user's browser waits for completion by polling:
/report/progress.json?token=d11a1658f614401782e8
# ..and eventually reloads:
/report/megasize?token=d11a1658f614401782e8
# which now displays the contents of their report
at least, this is the way I think it should be done.
I'm curious to know how other people approach this issue.
Also, what do you think about the polling approach vs a (background)
connection that stays connected waiting for the completion signal?
Cheers,
Toby
More information about the Catalyst
mailing list