[Catalyst-dev] Catalyst Performance Testing

[Jason Wieland]

Hello Crew,

I've been using Catalyst for a while now, thank you great framework.  I 
have a new project that requires a much higher performance than my 
previous projects and I decided to do some load testing and would like 
to share my results for a sanity check.
I tested two flavors: nginx + fastcgi  and apache + modperl and I'm 
using version 5.71 of Catalyst

Baseline:  Just using the webserver (apache or nginx) to download a 
static file
Static Catalyst: Running the same query but going through Catalyst to 
retrieve the same static file (using Static::Simple)
Noop Catalyst: Processing a request on a empty controller

* all numbers in request per second
Nginx + Fastcgi + Catalyst
---------------------------
Baseline: 1553 req/s
Static Catalyst: 134
Noop Catalyst: 91

Apache + Modperl + Catalyst
-----------------------------
Baseline: 976
Static Catalyst: 120
Noop Catalyst: 84


So a couple questions:
* Looking at these results.  Do they seem consistent in what the 
overhead of Catalyst should be?
* Is there advice that can be done to disable feature in Catalyst I 
might not need in order to increase performance?
 - I've took out StackTrace and -Debug from the main pm file
 - I've removed the $log as well
* Does upgrading to 5.8 increase performance?

I wrote a very simple 10 line modperl app that just grabs a static file 
and dumps it to STDOUT just for a little more comparison:
Static Modperl: 472 reqs/sec

So there is ~71% efficiency hit in using Catalyst to serve a static file 
and simple modperl script.  Granted Catalyst is doing a lot more stuff 
but seems like a humongous hit to take.

How was the load testing performed:  apache ab from multiple client servers
ex: ab -t 10 -c 40 "http://x-x-x-x.compute-1.amazonaws.com/foo/"
* Each test was run three times and the results where averaged
* modperl and fastcgi process memory was preloaded prior to test execution

Hardware: AWS EC2 - small instance

Apache Settings:
<IfModule mpm_prefork_module>
    StartServers         45
    MinSpareServers      30
    MaxSpareServers      45
    MaxClients           45
    MaxRequestsPerChild   0
</IfModule>

FastCGI Setttings:
..../script/xxxx_fastcgi.pl -n 35 -d


Also completely unrelated. I could not get Catalyst working on the 
mpm_worker_module.  Is Catalyst not thread safe?


Thanks in advance for your time,

Jason Wieland