[DBD-SQLite] added support for FTS3

[Stefan Evert]

On 14 Jul 2010, at 03:05, Kenichi Ishigaki wrote:

> Hi, I just svn-updated and tried to compile the latest trunk
> but failed with a series of syntax errors. It looks like
> there's something more to add for older perl (I'll look into
> it further later; I must be leaving now).
> 
> Stefan, would you send me (or to the list) the result of
> "prove -bvw t/rt_44891_string_looks_like_numbers.t", with
> your sqlite3 executable version?
> 

I'm using the standard sqlite3 command-line tool that comes with Mac OS X 10.6:

$ sqlite3 
SQLite version 3.6.12

However, the test script says

> # requires sqlite3 3.6.23.1 executable for extra tests

and schedules "only" 829 tests.

Here's what GDB tells me about the crash, which is perfectly reproducible:

> Program received signal SIGABRT, Aborted.
> 0x00007fff873b03d6 in __kill ()
> (gdb) backtrace
> #0  0x00007fff873b03d6 in __kill ()
> #1  0x00007fff87450913 in __abort ()
> #2  0x00007fff87445157 in __chk_fail ()
> #3  0x00007fff873a82cf in __strncpy_chk ()
> #4  0x000000010060fea6 in sqlite_st_execute (sth=0x100942b00, imp_sth=0x1005c6270) at dbdimp.c:162
> #5  0x0000000100603704 in XS_DBD__SQLite__st_execute (my_perl=0x100800000, cv=<value temporarily unavailable, due to optimizations>) at SQLite.xsi:589
> #6  0x000000010019b4ac in XS_DBI_dispatch (my_perl=0x100800000, cv=0x1008f5528) at DBI.xs:3426
> #7  0x0000000100075708 in Perl_pp_entersub ()
> #8  0x000000010006e6cc in Perl_runops_standard ()
> #9  0x000000010006945c in Perl_call_sv ()
> #10 0x000000010019b5e2 in XS_DBI_dispatch (my_perl=0x100800000, cv=0x1008ec0e8) at DBI.xs:3444
> #11 0x0000000100075708 in Perl_pp_entersub ()
> #12 0x000000010006e6cc in Perl_runops_standard ()
> #13 0x000000010006e43a in perl_run ()
> #14 0x0000000100000da4 in ?? ()
> #15 0x0000000100000cb8 in ?? ()

Turns out the crash happens when your try to copy 22 bytes into a 19-byte array in line 162 of dbimp.c:

> 	char tmp[19];
>         strncpy(tmp, v, z - v + 1);

Duh. :-)

Two more comments on this:

a) 22 bytes? Yes, because it's a 20-digit number + sign + NUL terminator.  Your counting ignores the first digit, so digit==19 is actually the 20-digit case.  Is this intentional?

b) You have the same tmp[19] array both in the 64-bit and 32-bit case (where numbers are limited to 10 digits); in the latter, that seems to provide a lot of unnecessary padding; in the former, not enough.

If I extend the temporary array to tmp[22], the test runs through and only produces three failures (instead of all the spurious errors I used to get previously):

> t/rt_44891_strings_look_like_numbers.t ... # requires sqlite3 3.6.23.1 executable for extra tests
> t/rt_44891_strings_look_like_numbers.t ... 586/829 
> #   Failed test 'type: real got: -9.22337203685478e+18 expected: -9223372036854775807 old_behavior:  sqlite3_behavior: '
> #   at t/rt_44891_strings_look_like_numbers.t line 90.
> 
> #   Failed test 'type: real got: 9.22337203685478e+18 expected: +9223372036854775806 old_behavior:  sqlite3_behavior: '
> #   at t/rt_44891_strings_look_like_numbers.t line 90.
> 
> #   Failed test 'type: real got: 9.22337203685478e+18 expected: +9223372036854775807 old_behavior:  sqlite3_behavior: '
> #   at t/rt_44891_strings_look_like_numbers.t line 90.
> # Looks like you failed 3 tests of 829.
> t/rt_44891_strings_look_like_numbers.t ... Dubious, test returned 3 (wstat 768, 0x300)
> Failed 3/829 subtests 


These also seem to be marked as "TODO" (when running with "prove", as suggested), so it's perhaps normal to fail them.


Hope this helps!
Stefan