[Catalyst] Scalable Catalyst

[Tomas Doran]

On 30 Jun 2009, at 11:58, Alejandro Imass wrote:

> Hi! Sorry for the lethargy, I've buried in a project and just recently
> saw the light of day :-)
>
> Yes, you  are correct [Tomas], BUT it all depends on the type of
> application. Web concurrency is often misinterpreted. The application
> I was referring to needs the ability to have many, many concurrent
> processes waiting for a response from another service which has a long
> response time. So in this case, having many, many threads sitting
> there waiting for a response is the way to go.

You're doing it wrong.

Don't block app server threads on a remote service if you have a slow  
remote service, the only thing that lies down that route is doom and  
fail.

Use a job queue or something so that your application servers don't  
sit waiting for slow remote services.

If you really actually need to block architecturally, then a heavy  
weight application server is just the wrong solution, full stop.

I'm sure that the worker mpm will give you more headroom if you have  
loads of mod_perl processes blocking than prefork, but I very much  
consider this to be an optimisation, not a solution.

That said, I'm not trying to be disparaging, and I'm happy this works  
for you, and is a viable option :)

The one thing that worries me about this is that it uses threads, and  
threads and perl don't get on.. For example, Moose's string style  
type constraints are a bad world (because the regex used makes  
various versions of at least perl 5.8 core dump). I don't think that  
this is an issue for anything in the Catalyst stack, as we're either  
type-constraint free, or use exclusively MooseX::Types stuff (which  
doesn't use those regular expressions, and therefore is safe, I  
think) - but may be a problem for other code..

I can certainly remember an un-fun world of apache puking it's guts  
on a coworker's machine as he was using the worker mpm..

So YMMV, depending on what portion of CPAN you use, and/or what your  
codebase looks like... :-/

> This is a very interesting diverse and complex subject, but the main
> idea of my post was to state that Catalyst works well under
> multi-threaded Apache with mod_perl, allowing, _in some cases_ better
> usage of the available resources. It does not apply, of course, to all
> cases, and your insight explains this very well.

Indeed - it all very much depends on the application & load profile  
etc - so all of this is just painting the bike shed, unless we're  
discussing a specific application on a real (i.e. something like  
production) set of hardware, and have actual performance metrics we  
want to ensure it fulfill. :)

> BTW, Ashley suggested I write a how-to on the WIki or something like
> that. Could some suggest exactly where, and I may have time to that
> this week.

http://dev.catalyst.perl.org/wiki/deployment

Making a section linked from 'Apache' in there, briefly outlining  
your config and what benefits you get from doing mod_perl this way  
would be great :)

Cheers
t0m