[Catalyst-dev] Re: still struggling with fastcgi (Stuart Watt)
Stuart Watt
stuart at morungos.com
Fri Oct 8 18:35:52 GMT 2010
As I understand it, MaxRequestPerChild and ThreadsPerChild are both =
Apache configurations which handle the Apache processes, and have no =
(direct) effect in FastCGI. They may well have an indirect effect, as =
they can cause requests to be queued in Apache before they are forwarded =
to Perl processes. FastCGI will have a second set of process management =
controls, probably somewhere else on configuration. What Apache FastCGI =
system are you using? I'll see if I can run our system under Windows =
Apache (it's about time I tried it) and test that out.
--S
On 10/8/2010 12:56 PM, Joseph He wrote:
> Thank you, Stuart.
>
> Process Explorer really helps, at least I can see which script is =
> alive, even though I still cannot figure out it is process or thread. =
> Here is what I see on Process Explorer:
> 1. There is httpd.exe. Its command line is C:\Apache2.2\bin\httpd.exe =
> -k runservice
> 2. Under httpd.exe, there is another httpd.exe, its command line is =
> C:\Apache2.2\bin\httpd.exe -d C:/Apache2.2 -f =
> C:\Apache2.2\conf\httpd.conf -d C:\Apache2.2\.
> 3. Under httpd.exe mentioned in 2, there are multiple perl.exe, their =
> command lines are typically C:\Perl\bin\Perl.exe =
> C:\...\fcgi-bin\index.fcgi, etc.
>
> ## All these perl.exe seems to be process instead of thread to me =
> 'cause its memory usage is typically from 15M - 23M, which is a bit =
> higher to me if it is a thread.
>
> I believe this is resource issue, whenever my cpu usage reaches to =
> 100%, problem happens. So I studied Apache configure directives and =
> find that MaxRequestPerChild and ThreadsPerChild are supported by =
> mdm_winnt, no other configuration directives are useful in windows. =
> Below is my testing and observation:
>
> MaxRequestsPerChild 6
> ThreadsPerChild 5
>
> Now after clicking and clicking, on Process Explorer, I can see Apache =
> launches another httpd.exe under httpd.exe mentioned in 1, then the =
> 'older' httpd.exe and all the perl.exe under its umbrella are killed. =
> The server just repeated this for three days without problem. But, =
> sorry, please bear with me that there is still but here :)
>
> With these two directives, perl.exe processes cannot just grow more =
> and more now, but I don't know how Apache counts this =
> MaxRequestPerChild, some pages I can refresh 30 times before new =
> httpd.exe is launched, some others might be 15, might be 8 or 9. Some =
> times I can see a new server is launched with 5 perl.exe, sometimes it =
> might happen with more that 20 perl.exe processes running. This really =
> puzzled me, because it is hard to optimize the server for production, =
> especially our clients might use different kinds of server/hardware.
>
> Who can explain what exactly MaxRequestPerChild and ThreadsPerChild =
> are in Windows environment? Apache document does not help me much here.
>
> Really appreciate all your helps!
>
> Joe
>
>
>
> Today's Topics:
>
> 1. Re: still struggling with fastcgi (Stuart Watt)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sat, 2 Oct 2010 13:24:23 -0400
> From: Stuart Watt <stuart at morungos.com <mailto:stuart at morungos.com>>
> Subject: Re: [Catalyst-dev] still struggling with fastcgi
> To: Development of the elegant MVC web framework
> <catalyst-dev at lists.scsys.co.uk
> <mailto:catalyst-dev at lists.scsys.co.uk>>
> Message-ID: <33E64D3E-4005-479A-A800-5619A90ADE59 at morungos.com
> <mailto:33E64D3E-4005-479A-A800-5619A90ADE59 at morungos.com>>
> Content-Type: text/plain; charset=3D"us-ascii"
>
> If you see multiple processes in the Windows task manager, these
> will be distinct Perl processes rather than threads. I haven't
> used Apache FastCGI, but under IIS, FastCGI is a single Perl
> thread per worker process, and this is likely to be the case. If
> you use the SysInternals process explorer (I'd strongly recommend
> using these tools for performance debugging in Windows) this will
> allow you to see how many threads each process is using inside. So
> it sounds like Apache may be creating Windows processes on you,
> but I could be wrong.
>
> I haven't seen this kind of an error with IIS/FastCGI, but we tend
> not to use that many worker processes. I've been using 20 worker
> processes at most.
>
> There are connection limits in MSSQL which could be a factor. That
> would be something I'd check.
>
> The 109 implies you are using named pipes to communicate between
> Perl and MSSQL. I tend to use TCP/IP. These are settings in the
> MSSQL Configuration Manager tool, and it might be worth looking to
> see whether changing the protocols might result in a different
> error. I'm not a low-level Windows expert, and there are many
> other resource limits you can hit in Windows apart from memory
> issues. Again, the SysInternals tools might help.
>
> Not having used Apache FastCGI that's about all I can say for now,
> but at the Perl/MSSQL boundary (which we hit even using
> IIS/FastCGI) these are issues I'd suggest looking into.
>
> All the best
> Stuart
> --
> Stuart Watt
> stuart at morungos.com <mailto:stuart at morungos.com>
>
>
>
> On 2010-10-01, at 3:59 PM, Joseph He wrote:
>
> > It seems to be a resource issue, so it is a matter of how to
> configure Apache/Windows to better handle/allocate resources.
> fastCGI helps to keep scripts stay in cache instead of killing and
> loading it every time.
> >
> > My understanding is on Windows, there is only one process with
> many threads, so my assumption is if I set limit on the threads,
> then there should not be a lot of spare threads staying in the
> cache, thus, the system will not be running to resource limit.
> >
> > I add these to conf file.
> > ThreadLimit 5
> > ThreadsPerChild 5
> >
> > Unfortunately, on windows process monitor, I still see the
> number of Perl processes just goes up and up with my clicking and
> then give me error when it reaches some level. This is not a
> striking test yet, just clicking 40 different pages by myself can
> exhaust all the resources ....
> >
> > On 29 September 2010 22:15, Joseph He <joseph.he.2008 at gmail.com
> <mailto:joseph.he.2008 at gmail.com>> wrote:
> >
> > > Thank you, T0m. Actually it is 5.8.7.
> > >
> > > I have done more testing while having windows Tast Manager
> open. I find o=3D
> > ut
> > > that these issues only happen when I already have about 35-38
> Perl proces=3D
> > ses
> > > running. Mainly two types of errors:
> > >
> > > 1. [Wed Sep 29 15:06:11 2010] [error] [client 127.0.0.1] (OS
> 109)The pipe
> > > has been ended. : FastCGI: comm with server "C:.....fcgi"
> aborted:
> > > GetOverlappedResult() failed, referer ......
> > >
> > > ## I searched and know that "(OS 109)The pipe has been ended"
> is SQL serv=3D
> > er
> > > error message.
> > >
> > > 2. [Wed Sep 29 15:10:52 2010] [error] [client 127.0.0.1]
> FastCGI: server
> > > "C:......fcgi" stderr: install_driver(ODBC) failed: Can't load
> > > 'C:/Perl/site/lib/auto/DBD/ODBC/ODBC.dll' for module
> DBD::ODBC: load_file=3D
> > :A
> > > dynamic link library (DLL) initialization routine failed at
> > > C:/Perl/lib/DynaLoader.pm line 230., referer: ....
> > > #
> >
> >
> > Sounds like an out of memory condition triggering a problem.
> What does the
> > Windows monitor show with respect to memory usage?
> > I have seen random errors from FastCGI on Linux, too, which
> aren't all that
> > easy to track down.
> > How about trying it through an Apache server on Windows and see
> if that
> > works or gives more information.
> > Or running your app on Linux if that's an option.
> >
> > Regards, Peter
> > http://perl.dragonstaff.co.uk
> >
>
>
>
> _______________________________________________
> Catalyst-dev mailing list
> Catalyst-dev at lists.scsys.co.uk
> http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.scsys.co.uk/pipermail/catalyst-dev/attachments/20101008/3=
a70ae22/attachment-0001.htm
More information about the Catalyst-dev
mailing list