[Catalyst] Catalyst benchmark 5.7 vs 5.8 - new test

[Toby Corkindale]

Fayland Lam wrote:
> I'm wondering if someone here did a benchmark between Catalyst 5.7 and 5.8

I have a vested interest in knowing the difference between the two 
versions as well, so knocked up a "proper" test.
I have two identical virtual machines, only on one I installed 
Catalyst::Runtime 5.71001 and the other with 5.80013.

Running the exact same app, I hit them up with Siege for a while, 
results follow at the end of this email.

If you want to replicate the test or examine my extremely-simple test 
app, see: http://github.com/TJC/Catalyst-Performance-Test
(Patches gleefully accepted ;)

It's interesting to note the headline figures have 5.71 performing 316 
tps, vs 5.80 making only 283 tps.
Memory usage (for this small app) has increased by 4MB, but is 
presumably shared. I guess I should look into that more.

The same system can serve small static pages from the webserver at about 
1900 tps. A real-world application there on Cat 5.8 gets 90 tps.

I don't see that performance difference (5.71 vs 5.80) as significant, 
since most of your time ends up being spent in application code, rather 
than the Catalyst framework itself.
ie. If you want to make your code go faster, look at optimising your 
templating and database queries before you worry about downgrading Catalyst.

-Toby

------------------= results =----------------------
Running 10 second warmup on 5.7..
Running main test on 5.7..

Transactions:		       94796 hits
Availability:		      100.00 %
Elapsed time:		      300.00 secs
Data transferred:	       77.35 MB
Response time:		        0.03 secs
Transaction rate:	      315.99 trans/sec
Throughput:		        0.26 MB/sec
Concurrency:		       10.00
Successful transactions:       94796
Failed transactions:	           0
Longest transaction:	        0.98
Shortest transaction:	        0.00

Process size:
101m VIRT, 34m RES



Running 10 second warmup on 5.8..
Running main test on 5.8..

Transactions:		       84805 hits
Availability:		      100.00 %
Elapsed time:		      300.00 secs
Data transferred:	       69.20 MB
Response time:		        0.04 secs
Transaction rate:	      282.68 trans/sec
Throughput:		        0.23 MB/sec
Concurrency:		        9.99
Successful transactions:       84805
Failed transactions:	           0
Longest transaction:	        1.07
Shortest transaction:	        0.00

Process size:
103m VIRT, 38m RES