[DBD-SQLite] policy for FTS5 integration ?

[Darren Duncan]

I believe the best option is d) to support both FTS4 and FTS5 at the same time, 
as if they were different data types.  Failing that, I like b) the best.  I also 
think a) is the worst option; this is not something worth forking over. -- 
Darren Duncan

On 2015-10-25 11:46 PM, Dami Laurent (PJ) wrote:
> Hi there,
>
> I noticed that the 3.9.0 amalgamation now contains FTS5. So it’s time to think
> about the evolution of FTS support in DBD::Sqlite.
>
> At the moment, FTS5 is still experimental, but at some point it will stabilize
> and become mainstream.
>
> FTS5 contains incompatible changes from FTS3/4 (see
> http://www.sqlite.org/fts5.html#appendix_a).
>
> Some of those incompatibilities can be handled in the DBD::Sqlite driver and
> therefore will be invisible to users ; but some other points change the SQL API
> to FTS and therefore **will** be visible.
>
> So what should be the future evolution of DBD::Sqlite with respect to FTS ? Here
> are some options :
>
> a)Stick with FTS4 forever to preserve backwards compatibility, and fork a new
> distrib DBD::SQLite::FTS5 (but then how to keep both distribs in sync ?)
>
> b)Target for a backwards-incompatible move to FTS5 at some point in the future,
> and advertise it long in advance
>
> c)Support 2 variants, with an option in Makefile.PL to choose the FTS version at
> compilation time (but this could be a nightmare for automatic toolchains).
>
> d)Try to simultaneously activate FTS3 and FTS5 in the same compilation (I don’t
> know if this is possible), and let users choose through runtime options.
>
> e)Any other idea ?
>
> DBD::Sqlite never encountered this issue before, because the first FTS support
> was for FTS3, and then when sqlite.org introduced FTS4 it was fully compatible
> with FTS3. The only somewhat similar situation was in2010 when the FTS syntax
> changed from « Standard Query Syntax » to « Extended Query Syntax » ;  at that
> time I published the DBD::Sqlite::FTS3Transitional module to help with the
> migration, but this time it will not be that simple.
>
> So what do you think ? My personal preference would be for option b). It’s not
> very nice to break backwards compatibility, but sometimes it must happen.
> Besides, I never saw any public discussions, comments or bug reports about FTS
> support in DBD ::Sqlite, so the user community is probably quite small, and
> therefore the impact of such a change would be limited. Of course this should
> only be activated when FTS5 has becomes stable, which should still take a couple
> of months.
>
> Once the policy issue is settled, I’m interested to volunteer for working on the
> technical aspects – but I have very little free time to do so, so I will not
> progress very fast. If anybody else has more time, they are welcome to take the job.
>
> Cheers, Laurent Dami