[Catalyst-dev] Re: still struggling with fastcgi (Stuart Watt)

Stuart Watt stuart at morungos.com
Fri Oct 8 18:35:52 GMT 2010


  As I understand it, MaxRequestPerChild and ThreadsPerChild are both =

Apache configurations which handle the Apache processes, and have no =

(direct) effect in FastCGI. They may well have an indirect effect, as =

they can cause requests to be queued in Apache before they are forwarded =

to Perl processes. FastCGI will have a second set of process management =

controls, probably somewhere else on configuration. What Apache FastCGI =

system are you using? I'll see if I can run our system under Windows =

Apache (it's about time I tried it) and test that out.

--S


On 10/8/2010 12:56 PM, Joseph He wrote:
> Thank you, Stuart.
>
> Process Explorer really helps, at least I can see which script is =

> alive, even though I still cannot figure out it is process or thread. =

> Here is what I see on Process Explorer:
> 1. There is httpd.exe. Its command line is C:\Apache2.2\bin\httpd.exe =

> -k runservice
> 2. Under httpd.exe, there is another httpd.exe, its command line is =

> C:\Apache2.2\bin\httpd.exe -d C:/Apache2.2 -f =

> C:\Apache2.2\conf\httpd.conf -d C:\Apache2.2\.
> 3. Under httpd.exe mentioned in 2, there are multiple perl.exe, their =

> command lines are typically C:\Perl\bin\Perl.exe =

> C:\...\fcgi-bin\index.fcgi, etc.
>
> ## All these perl.exe seems to be process instead of thread to me =

> 'cause its memory usage is typically from 15M - 23M, which is a bit =

> higher to me if it is a thread.
>
> I believe this is resource issue, whenever my cpu usage reaches to =

> 100%, problem happens. So I studied Apache configure directives and =

> find that MaxRequestPerChild and ThreadsPerChild are supported by =

> mdm_winnt, no other configuration directives are useful in windows. =

> Below is my testing and observation:
>
> MaxRequestsPerChild 6
> ThreadsPerChild         5
>
> Now after clicking and clicking, on Process Explorer, I can see Apache =

> launches another httpd.exe under httpd.exe mentioned in 1, then the =

> 'older' httpd.exe and all the perl.exe under its umbrella are killed. =

> The server just repeated this for three days without problem. But, =

> sorry, please bear with me that there is still but here :)
>
> With these two directives, perl.exe processes cannot just grow more =

> and more now, but I don't know how Apache counts this =

> MaxRequestPerChild, some pages I can refresh 30 times before new =

> httpd.exe is launched, some others might be 15, might be 8 or 9. Some =

> times I can see a new server is launched with 5 perl.exe, sometimes it =

> might happen with more that 20 perl.exe processes running. This really =

> puzzled me, because it is hard to optimize the server for production, =

> especially our clients might use different kinds of server/hardware.
>
> Who can explain what exactly MaxRequestPerChild and ThreadsPerChild =

> are in Windows environment? Apache document does not help me much here.
>
> Really appreciate all your helps!
>
> Joe
>
>
>
>     Today's Topics:
>
>       1. Re: still struggling with fastcgi (Stuart Watt)
>
>
>     ----------------------------------------------------------------------
>
>     Message: 1
>     Date: Sat, 2 Oct 2010 13:24:23 -0400
>     From: Stuart Watt <stuart at morungos.com <mailto:stuart at morungos.com>>
>     Subject: Re: [Catalyst-dev] still struggling with fastcgi
>     To: Development of the elegant MVC web framework
>     <catalyst-dev at lists.scsys.co.uk
>     <mailto:catalyst-dev at lists.scsys.co.uk>>
>     Message-ID: <33E64D3E-4005-479A-A800-5619A90ADE59 at morungos.com
>     <mailto:33E64D3E-4005-479A-A800-5619A90ADE59 at morungos.com>>
>     Content-Type: text/plain; charset=3D"us-ascii"
>
>     If you see multiple processes in the Windows task manager, these
>     will be distinct Perl processes rather than threads. I haven't
>     used Apache FastCGI, but under IIS, FastCGI is a single Perl
>     thread per worker process, and this is likely to be the case. If
>     you use the SysInternals process explorer (I'd strongly recommend
>     using these tools for performance debugging in Windows) this will
>     allow you to see how many threads each process is using inside. So
>     it sounds like Apache may be creating Windows processes on you,
>     but I could be wrong.
>
>     I haven't seen this kind of an error with IIS/FastCGI, but we tend
>     not to use that many worker processes. I've been using 20 worker
>     processes at most.
>
>     There are connection limits in MSSQL which could be a factor. That
>     would be something I'd check.
>
>     The 109 implies you are using named pipes to communicate between
>     Perl and MSSQL. I tend to use TCP/IP. These are settings in the
>     MSSQL Configuration Manager tool, and it might be worth looking to
>     see whether changing the protocols might result in a different
>     error. I'm not a low-level Windows expert, and there are many
>     other resource limits you can hit in Windows apart from memory
>     issues. Again, the SysInternals tools might help.
>
>     Not having used Apache FastCGI that's about all I can say for now,
>     but at the Perl/MSSQL boundary (which we hit even using
>     IIS/FastCGI) these are issues I'd suggest looking into.
>
>     All the best
>     Stuart
>     --
>     Stuart Watt
>     stuart at morungos.com <mailto:stuart at morungos.com>
>
>
>
>     On 2010-10-01, at 3:59 PM, Joseph He wrote:
>
>     > It seems to be a resource issue, so it is a matter of how to
>     configure Apache/Windows to better handle/allocate resources.
>     fastCGI helps to keep scripts stay in cache instead of killing and
>     loading it every time.
>     >
>     > My understanding is on Windows, there is only one process with
>     many threads, so my assumption is if I set limit on the threads,
>     then there should not be a lot of spare threads staying in the
>     cache, thus, the system will not be running to resource limit.
>     >
>     > I add these to conf file.
>     > ThreadLimit 5
>     > ThreadsPerChild 5
>     >
>     > Unfortunately, on windows process monitor, I still see the
>     number of Perl processes just goes up and up with my clicking and
>     then give me error when it reaches some level. This is not a
>     striking test yet, just clicking 40 different pages by myself can
>     exhaust all the resources ....
>     >
>     > On 29 September 2010 22:15, Joseph He <joseph.he.2008 at gmail.com
>     <mailto:joseph.he.2008 at gmail.com>> wrote:
>     >
>     > > Thank you, T0m. Actually it is 5.8.7.
>     > >
>     > > I have done more testing while having windows Tast Manager
>     open. I find o=3D
>     > ut
>     > > that these issues only happen when I already have about 35-38
>     Perl proces=3D
>     > ses
>     > > running. Mainly two types of errors:
>     > >
>     > > 1. [Wed Sep 29 15:06:11 2010] [error] [client 127.0.0.1] (OS
>     109)The pipe
>     > > has been ended.  : FastCGI: comm with server "C:.....fcgi"
>     aborted:
>     > > GetOverlappedResult() failed, referer ......
>     > >
>     > > ## I searched and know that "(OS 109)The pipe has been ended"
>     is SQL serv=3D
>     > er
>     > > error message.
>     > >
>     > > 2. [Wed Sep 29 15:10:52 2010] [error] [client 127.0.0.1]
>     FastCGI: server
>     > > "C:......fcgi" stderr: install_driver(ODBC) failed: Can't load
>     > > 'C:/Perl/site/lib/auto/DBD/ODBC/ODBC.dll' for module
>     DBD::ODBC: load_file=3D
>     > :A
>     > > dynamic link library (DLL) initialization routine failed at
>     > > C:/Perl/lib/DynaLoader.pm line 230., referer: ....
>     > > #
>     >
>     >
>     > Sounds like an out of memory condition triggering a problem.
>     What does the
>     > Windows monitor show with respect to memory usage?
>     > I have seen random errors from FastCGI on Linux, too, which
>     aren't all that
>     > easy to track down.
>     > How about trying it through an Apache server on Windows and see
>     if that
>     > works or gives more information.
>     > Or running your app on Linux if that's an option.
>     >
>     > Regards, Peter
>     > http://perl.dragonstaff.co.uk
>     >
>
>
>
> _______________________________________________
> Catalyst-dev mailing list
> Catalyst-dev at lists.scsys.co.uk
> http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.scsys.co.uk/pipermail/catalyst-dev/attachments/20101008/3=
a70ae22/attachment-0001.htm


More information about the Catalyst-dev mailing list