[DBD-SQLite] Re: [sqlite] SQLite version 3.7.0

[Stefan Evert]

On 22 Jul 2010, at 20:58, Darren Duncan wrote:

> Or specifically, they said this:
> 
>     I don't have time to investigate right now, but both failing tests seem to 
> be connected with concurrent access to a table by two forked processes (the test 
> script forks a child, which does concurrent access).
> 
>     At least in the second case, the DROP TABLE and CREATE TABLE commands are 
> issued by the main process (after the child has dropped table2) and are supposed 
> to succeed, so I believe there's something else going on than changed error 
> codes (unless they trigger a bug within SQLite itself).

Some more information on this issue, which will hopefully help us close in on the problem.  I don't want do cross-post, so I'm sending this to the DBD::SQLite list, which seems the appropriate forum at this stage.

I've taken a closer look at t/28_schemachange.t. It turns out that the two commands in question actually succeed, i.e. when I arrange for the temporary database file not to be cleaned up and inspect it with the command-line tool, it's exactly in the expected state.  So to some extent the error messages must be spurious.

Playing around with DBI's tracing functionality I found that as soon as the main process does waitpid(), all hell breaks loose -- even if it doesn't open the second database connection while the child is still alive.  I get either the I/O errors, or under some conditions the entire database seems to be empty, i.e. the CREATE TABLE commands from the first SCOPE: in the main program don't seem to have an effect.

The key problem seems to be when the child process exits.  If I add a few sleep() statements so that the child can execute its commands while the main process sleeps, but then the child keeps sleeping (after closing its DB connection!) while the main process executes the remaining commands (after a short sleep() but no waitpid()), the test passes all steps without errors.

On the other hand, even if I remove all DB calls from the child and don't waitpid() for it, but let the main process sleep() long enough for the child to exit, then I get the spurious disk I/O errors.

No idea where to look next, but I suspect there's some bad interaction between the fork() and something inside SQLite or DBD::SQLite -- these are all separate database handles that aren't shared between the main process and the child.

The other failing test (t/08_busy.t, which is somewhat more complicated) also does a fork(), so I believe it will also passed when we've fixed the problem described above.

Best,
Stefan