[Catalyst] Memory usage with FCGI? Server running out of RAM
lasse at unity3d.com
Thu Jun 11 13:53:15 GMT 2015
In my experience, it's usually a few naughty requests that gobble up a lot
of memory. We found it useful to record process vsize and rss at the
beginning and end of the request cycle to generate warnings like this:
Warning: PID 28710 grew rss by 20% from 42 MiB to 51 MiB serving GET
One of the common mistakes in our app was inflating all row objects in a
result set at the same time when it wasn't needed. We prefetch a lot so row
objects and their related objects get big. In those cases, this:
foreach my $row ($rs->all)
push @results, $row->to_json,
Will use a lot more memory than this:
while (my $row = $rs->next)
push @results, $row->to_json,
DBIx::Class's HashRefInflator is often touted for its speed but it's also a
lot lighter on memory...
Another thing you could consider is using Plack to run all your Perl apps
in the same server/process. Memory for all common dependencies would be
shared among the apps in that case which might get you a decent gain...
There's also a bunch of Plack::MiddleWare's on CPAN to help with
debugging/profiling of apps...
On Thu, Jun 11, 2015 at 3:03 PM, Denny <2015 at denny.me> wrote:
> I run a server with 18 separate installs of ShinyCMS running, with at
> least 3 and sometimes 20 processes per install. 122 perl-fcgi processes in
> total. They look a bit bigger than most of your processes. I've got 8GB of
> RAM, and I'm pushing up against the edges of it a bit, but I don't usually
> have any problems.
> Hope that's useful info!
> On 11 June 2015 13:31:16 BST, Jesse Sheidlower <jester at panix.com> wrote:
>> Sysadmin-fu needed: I have a personal server in the cloud that has 1 GB
>> of RAM. It's running Debian, and the usual server basics--MySQL, nginx,
>> etc. No X, of course.
>> There are three Catalyst apps running, all of them fairly small, using
>> nginx and FCGI (I run the MyApp_fastcgi.pl script to a socket, that
>> nginx then responds to). But they're taking up all the memory on the
>> system, and I can't run another Cat app (and sometimes even system
>> commands hang for lack of memory). I tried reducing --nproc to 1 for two
>> of the three apps that I'm the only one using (the other has --nproc 3).
>> The output of "top" with "M" (sorting by RES) looks like (I removed a
>> few columns for spacing):
>> PID USER VIRT RES SHR S %MEM TIME+ COMMAND
>> 23766 www-data 440m 230m 2584 S 23.0 0:13.40 perl-fcgi
>> 16958 mysql 428m 188m 0 S 18.8 118:14.98
>> 25992 www-data 375m 158m 2608 S 15.9 0:06.80 perl-fcgi
>> 26884 www-data 238m 80m 2616 S 8.0 0:01.61 perl-fcgi
>> 2513 www-data 282m 76m 0 S 7.7 0:06.01 perl-fcgi
>> 2512 www-data 182m 66m 0 S 6.6 0:00.00 perl-fcgi-pm [Library]
>> 1825 www-data 172m 61m 0 S 6.1 0:03.44 perl
>> 2487 www-data 163m 46m 60 S 4.6 0:00.18 perl-fcgi-pm [CatSF]
>> 1692 root 119m 5128 140 S 0.5 0:56.32 rsyslogd
>> 19998 www-data 79452 3296 1136 S 0.3 6:10.61 nginx
>> 2265 www-data 79768 3148 504 S 0.3 15:57.81 nginx
>> Is this an expected or appropriate amount of memory usage? I appreciate
>> that 1GB isn't all that much memory nowadays, but this is a very lightly
>> loaded machine, and I don't want to pay a lot more for a larger cloud
>> server to handle this if it's not necessary.
>> List: Catalyst at lists.scsys.co.uk
>> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
>> Searchable archive: http://email@example.com/
>> Dev site: http://dev.catalyst.perl.org/
> Sent from my phone. Please excuse typos, terseness, and top-posting.
> List: Catalyst at lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive:
> Dev site: http://dev.catalyst.perl.org/
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Catalyst