[Catalyst] Re: memory usage of mod_perl process

[Peter Edwards]

Jeffrey Ng wrote:
>so are you saying in general catalyst is pretty heavy, and its not for site
with many concurrent users?... 

No, not at all. It all depends on what your application does, how many web
screens, how many peak concurrent users and equivalent requests/second, how
many discrete users, how many data feeds in/out, data volumes, network
transfer volumes.

Your limiting factor might be CPU, database I/O or network bandwidth.

Different architectural models are best for different cases.

 

There is a cost equation between using a powerful framework that increases
robustness, maintainability and speed of development versus how many servers
you need.

Say a server costs you $100 per month to rent and each Catalyst server maxes
at 30 requests per second but you get a peak load of 90 reqs/sec. You would
need to load balance your app across 3 servers. There would be an initial
configuration cost and then you'd be paying $200 p.m. = $2400 p.a. more for
the two extra servers.

Let's say a developer, including overhead, costs you $1000 per day. If you
move away from Catalyst to a lighter framework and it costs you more than
2.5 developer days extra then you're losing money. Catalyst has a lot of
tried and tested pluggable modules and if you end up re-writing any of those
you're almost certainly spending money.

 

>what other light weight approach would you go for?

First thing, set up proper caching (either Squid or a lightweight Apache)
and use a tool like Fiddler to examine the HTTP request Cache headers to
make sure it's working properly. That's a huge saving.

Then do profiling and benchmarking to determine exactly what the bottlenecks
are before making any changes.

If you're doing a lot of dynamic Ajax stuff with complex SQL, there might be
a case for hand-tuning that part of the app. Look at whether you can
pre-calculate and cache expensive queries or change the app design to allow
this. If necessary, you could write a separate lightweight app to serve
those requests.

 

I didn't find a suitable lightweight mod_perl framework so wrote one using
CGI::Session, DBIx::Class, Template toolkit. It's not that difficult.

If you would like some details please feel free to email off-list or call
me. However, that was an atypical case and as soon as you start needing
internationalization, RSS feeds etc. I think you're better off with
Catalyst.

 

You said you need to serve JS from the same server as the heavyweight app.
Assuming the JS pages are static or can be precalculated and stored in a
static dir, and you have the JS below /js and your app below /app in the URI
namespace:

On the same server run two Apaches (or one Apache + one Fastcgi)

1) Lightweight proxy. Build it yourself from source, turn off all
unnecessary options and shared libraries except mod_proxy/mod_rewrite.

Use rewrite rules to proxy /app/. on to heavy server, e.g. based on an
Apache 1 setup I have that's live:

    RewriteEngine On

    RewriteRule    ^/app/(.*$)   http://localhost:2000/app/$1  [P]

    ProxyPassReverse   /   http://localhost:2000/

    RewriteRule    ^proxy:.*  -  [F]

The /js and /images get served by the proxy server (which has a <1MB memory
footprint per child)

2) Heavy mod_perl server listening at localhost:2000. Maps /app on to your
handler (40-100MB memory per child).

 

Say you had to handle the Ajax part separately. With this model you can add
another proxy entry that routes, say, /ajax/ on to another Apache server on
the same host (e.g. localhost:2001) or on another host altogether (e.g.
http://ajax1.zorpia.com <http://ajax1.zorpia.com/> ). If you need to scale
more, use DNS round robin to a set of front end proxies like this and route
requests to a farm of backend servers using a shared database server.

 

I'd be surprised if you couldn't get decent performance out of a
well-designed Catalyst app. There are businesses listed on the Catalyst
pages that offer consultancy help if you need further pointers. As a
business I used Shadowcat Systems last year and received good technical
design advice.

 

Regards,

Peter Edwards

www.dragonstaff.com <http://www.dragonstaff.com/>  ~ Business IT Consultancy

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.scsys.co.uk/pipermail/catalyst/attachments/20070210/caa45a6b/attachment-0001.htm