[Catalyst] Re: Have exceeded the maximum number of attempts (1000) to open temp file/dir

[Bill Moseley]

On Fri, Oct 25, 2013 at 6:51 AM, Bill Moseley <moseley at hank.org> wrote:

>
> [ERROR] "Caught exception in engine "Error in tempfile() using
> /tmp/XXXXXXXXXX: Have exceeded the maximum number of attempts (1000) to
> open temp file/dir
>


I don't really see how this can be a Catalyst issue, but I can't reproduce
it outside of Catalyst -- and outside of our production environment.

Can anyone think of anything else that might be going on here?


The template has 10 "X" that are replaced by I think 63 random ascii
characters.   63^10 is a huge number of random strings.   File::Temp only
loops when sysopen returns EEXISTS -- that is, when sysopen fails AND the
error is that the file already exists.

Sure, there's 50 web processes but the odds of them all being in exact
lock-step with calling rand() is unlikely.  And even if they started out
that way if two processes opened the exact same name at the same time one
process would just try the next random name and be done.

I have something like 26K files in /tmp, so nothing compared to 63^10.
And each web server is only seeing about 10 request/sec.

It's just not making sense.


Again, I'm unable to replicate the problem with a simple test script that
is designed to clash.

I fork 50 (or more) child processes to replicate the web server processes
and then in each one I do this:


        # Wait until top of the second so each child procss starts about
the same time.

        my $t =3D time();  # Time::HiRes
        sleep( int( $t ) + 1 - $t );


        for ( 1 .. 500 ) {

            my $fh =3D File::Temp->new(
                TEMPLATE =3D> 'bill_XXXXX',
                DIR =3D> '/tmp',
            );

        }


And never see any contention.



>
> The File::Temp docs say:
>
> If you are forking many processes in parallel that are all creating
>> temporary files, you may need to reset the random number seed using
>> srand(EXPR) in each child else all the children will attempt to walk
>> through the same set of random file names and may well cause
>> themselves to give up if they exceed the number of retry attempts.
>
>
> We are running under mod_perl.   Could it be as simple as the procs all
> were in sync?   I'm just surprised this has not happened before.   Is the=
re
> another explanation?
>
> Where would you suggest to call srand()?
>
>
> Another problem, and one I've commented<https://rt.cpan.org/Public/Bug/Di=
splay.html?id=3D84004>on before, is that HTTP::Body doesn't use File::Temp'=
s unlink feature and
> depends on Catalyst cleaning up.  This results in orphaned files left on
> temp disk.
>
>
>
>
>
> --
> Bill Moseley
> moseley at hank.org
>



-- =

Bill Moseley
moseley at hank.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.scsys.co.uk/pipermail/catalyst/attachments/20131104/6f3ef=
f79/attachment.htm