[Catalyst] fastcgi, production catalyst and sigpipe

[Jason Kohles]

On Dec 27, 2007, at 5:24 AM, Ruben Fonseca wrote:

>
> Hi! I have a Catalyst website in production, but started to hit some  
> problems lately. But first a little about my configuration:
>
> Catalyst website using the standard and the latest stable modules.
> Linux + Apache 2 + fastcgi running in external mode.
> fastcgi server (FCGIProcessManager) with 2 Catalyst children.
>
> The problem is that, once a day, at random time, my application  
> simply dies, and the only error I see on apache logs is:
>
> FastCGI: server "/tmp/etrading-secure.fcgi" stderr : FastCGI: server  
> (pid 8122): safe exit after SIGTERM, referer: https://__mysite__"
>
> Digging on the situation, I put strace saving the system calls of  
> both the fcgiprocessmanager (8120), and the 2 Catalyst children  
> (8121, 8122) and found this:
>
> - The first child (8121) receives a SIGPIPE and imediatly dies! Here  
> are the last lines of strace before 8121 dies:
>
> 8121  18:17:20 write(4, "\1\6\0\1\26J\6\0\"POST\">\n\t\t\t\t\t<input  
> type"..., 5744) = -1 EPIPE (Broken pipe)
> 8121  18:17:20 --- SIGPIPE (Broken pipe) @ 0 (0) ---
>
> I thing SIGPIPE is sent when the client closes the connection before  
> all data is sent by Catalyst. But why does it terminate?
>
SIGPIPE means that the process was writing to a pipe and there was  
nothing listening on the other end of the pipe.  Look back through the  
trace and see what file descriptor 4 was assigned to and that might  
give you a clue what died.

> - The process manager (8120) sees SIGPIPE and sends SIGTERM to the  
> other child (8122). After the child dies, it turns itself down:
>
> 8120  15:42:58 waitpid(-1, [{WIFSIGNALED(s) && WTERMSIG(s) ==  
> SIGPIPE}], 0) = 8121
> 8120  18:17:20 --- SIGCHLD (Child exited) @ 0 (0) ---
> 8120  18:17:20 write(2, "FastCGI: manager (pid 8120): ser"..., 69) =  
> 69
> 8120  18:17:20 getppid()
> (...)
> 8120  18:17:20 write(2, "FastCGI: manager (pid 8120): sen"..., 56) =  
> 56
> 8120  18:17:20 kill(8122, SIGTERM)      = 0
> 8120  18:17:20 waitpid(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}],  
> 0) = 8122
> 8120  18:17:21 --- SIGCHLD (Child exited) @ 0 (0) ---
> 8120  18:17:21 write(2, "FastCGI: manager (pid 8120): ser"..., 68) =  
> 68
> 8120  18:17:21 write(2, "FastCGI: manager (pid 8120): dyi"..., 61) =  
> 61
> (...)
> 8120  18:17:21 exit_group(0)            = ?
>
> What do you think is wrong here? I think that the catalyst  
> application should not die when it receives a SIGPIPE. But it could  
> be another problem. Please advice me. If you need more information  
> about my environment, please ask!
>
What would you suggest it do if not die?  SIGPIPE means it is trying  
to talk to a process that is not listening, there isn't a great deal  
the application is going to be able to do about that in a portable  
manner...

-- 
Jason Kohles, RHCA RHCDS RHCE
email at jasonkohles.com - http://www.jasonkohles.com/
"A witty saying proves nothing."  -- Voltaire