<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-GB link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Hi,<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>It’s beginning to look like this might be a problem with
the way the concurrency of the fastcgi processes is configured.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Most of our other fastcgi scripts handle their own concurrency
and start with something like this:<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>for (my $i = 0; $i < 10; $i++) {<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'> if (fork() == 0) {<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'> last;<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'> }<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>}<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>For this reason the concurrency setting in Zeus was set to 0.
However, I don’t think the Catalyst process forks, at least not by
default. I’ve never seen more than one Catalyst process running. The
lock-ups may just have been overload.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Yesterday at about 11am we set the Zeus concurrency to 5. We
haven’t had a lock up since, so that’s obviously helped.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>However, the whole mode of operation of the system has changed
and there has been a performance hit.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>When a request arrives, we get multiple catalyst processes
spawned. These hang around for a while, handle the request and die within a
couple of minutes. When the next request arrives, the processes have to be
started up again.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>That’s analogous to cgi operation and not what we had
before. I thought that fastcgi processes were supposed to handle requests in a
loop and I was expecting at least one of the processes to persistent for a
significant period of time (eg 30 minutes).<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Is this correct operation?<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Regards<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Duncan<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'>
<p class=MsoNormal><b><span lang=EN-US style='font-size:10.0pt;font-family:
"Tahoma","sans-serif"'>From:</span></b><span lang=EN-US style='font-size:10.0pt;
font-family:"Tahoma","sans-serif"'> Peter Edwards
[mailto:peter@dragonstaff.co.uk] <br>
<b>Sent:</b> 12 January 2011 10:55<br>
<b>To:</b> The elegant MVC web framework<br>
<b>Subject:</b> Re: [Catalyst] Zeus and Catalyst<o:p></o:p></span></p>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<div>
<p class=MsoNormal>On 12 January 2011 09:02, Duncan Garland <<a
href="mailto:Duncan.Garland@motortrak.com">Duncan.Garland@motortrak.com</a>>
wrote:<o:p></o:p></p>
<div>
<p class=MsoNormal> <o:p></o:p></p>
</div>
<blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;
margin-left:4.8pt;margin-right:0cm'>
<p class=MsoNormal>0x0000003bbe20d8e0 in __read_nocancel () from /lib64/libpthread.so.0<br>
(gdb) (gdb) #0 0x0000003bbe20d8e0 in __read_nocancel () from
/lib64/libpthread.so.0<br>
No symbol table info available.<br>
#1 0x00002ac7089512a2 in snttread () from
/oracle/product/10.2.0/db_1/lib/libclntsh.so.10.1<br>
#14 0x00002ac70861bb27 in OCIStmtExecute () from
/oracle/product/10.2.0/db_1/lib/libclntsh.so.10.1<br>
#15 0x00002ac70803adc9 in ora_describe (h=0x108af110, imp_sth=0x15fdfa10) at
oci8.c:1997<br>
my_perl = <value optimized out><br>
imp_dbh = 0x15594520<br>
long_readlen = 6553500<br>
num_fields = <value optimized out><br>
num_errors = <value optimized out><br>
has_longs = <value optimized out><br>
est_width = <value optimized out><br>
nested_cursors = <value optimized out><br>
i = <value optimized out><br>
status = <value optimized out><o:p></o:p></p>
</blockquote>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>When you execute an Oracle statement OCIStmtExecute ()
it gets stuck in __read_nocancel ().<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>Looks like it's either blocked waiting on an uncommitted
session elsewhere or you've hit an Oracle bug.<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>Use the Oracle tools to see what other queries are hanging
around when it happens. <o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>See if there are reports of deadlocks or anything in
the db error log.<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>You can also check out<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal><a
href="http://www.google.co.uk/search?sourceid=chrome&ie=UTF-8&q=libclntsh.so+__read_nocancel">http://www.google.co.uk/search?ie=UTF-8&q=libclntsh.so+__read_nocancel</a><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal><a href="https://savannah.cern.ch/support/?106453">https://savannah.cern.ch/support/?106453</a><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><a href="https://savannah.cern.ch/bugs/?47435">https://savannah.cern.ch/bugs/?47435</a><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>etc.<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>This kind of weirdness is one of the reasons I'm not an
Oracle lover :-~<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>Regards, Peter<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><a href="http://perl.dragonstaff.co.uk">http://perl.dragonstaff.co.uk</a><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal> <o:p></o:p></p>
</div>
</div>
</div>
</body>
</html>