[DBD-SQLite] policy for FTS5 integration ?

[Dami Laurent (PJ)]

Hi there,

I noticed that the 3.9.0 amalgamation now contains FTS5. So it's time to think about the evolution of FTS support in DBD::Sqlite.

At the moment, FTS5 is still experimental, but at some point it will stabilize and become mainstream.
FTS5 contains incompatible changes from FTS3/4 (see http://www.sqlite.org/fts5.html#appendix_a).
Some of those incompatibilities can be handled in the DBD::Sqlite driver and therefore will be invisible to users ; but some other points change the SQL API to FTS and therefore *will* be visible.

So what should be the future evolution of DBD::Sqlite with respect to FTS ? Here are some options :


a)      Stick with FTS4 forever to preserve backwards compatibility, and fork a new distrib DBD::SQLite::FTS5 (but then how to keep both distribs in sync ?)

b)      Target for a backwards-incompatible move to FTS5 at some point in the future, and advertise it long in advance

c)       Support 2 variants, with an option in Makefile.PL to choose the FTS version at compilation time (but this could be a nightmare for automatic toolchains).

d)      Try to simultaneously activate FTS3 and FTS5 in the same compilation (I don't know if this is possible), and let users choose through runtime options.

e)      Any other idea ?

DBD::Sqlite never encountered this issue before, because the first FTS support was for FTS3, and then when sqlite.org introduced FTS4 it was fully compatible with FTS3. The only somewhat similar situation was in2010 when the FTS syntax changed from < Standard Query Syntax > to < Extended Query Syntax > ;  at that time I published the DBD::Sqlite::FTS3Transitional module to help with the migration, but this time it will not be that simple.

So what do you think ? My personal preference would be for option b). It's not very nice to break backwards compatibility, but sometimes it must happen. Besides, I never saw any public discussions, comments or bug reports about FTS support in DBD ::Sqlite, so the user community is probably quite small, and therefore the impact of such a change would be limited. Of course this should only be activated when FTS5 has becomes stable, which should still take a couple of months.

Once the policy issue is settled, I'm interested to volunteer for working on the technical aspects - but I have very little free time to do so, so I will not progress very fast. If anybody else has more time, they are welcome to take the job.

Cheers, Laurent Dami





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.scsys.co.uk/pipermail/dbd-sqlite/attachments/20151026/3d7dce4b/attachment.htm>