[Dbix-class] Feature planning - DBP (was Re: Orders of Business)

[Darren Duncan]

So I'm now opening up a generic "feature planning" thread ostensibly in the 
manner that Matt requested.

On 2017-01-24 4:06 PM, Matt S Trout wrote:
> Time to figure out what we need to do to get moving again.
>
> Note that I'm going to list a bunch of things below, with brief notes on each
> one - as you reply, please adjust the subject line to indicate which one you're
> responding to, and separate replies for each response. Hopefully that'll give
> us a reasonable balance between keeping things clear and not ending up in
> [1/27] territory.
>
> * Feature planning
>
> I have the odd idea for things that might potentially be of interest to people
> as features once we've got current master shipped, but that's definitely a
> 'might' and I regard everybody to have a stake in the direction at this point.
>
> I suspect it's probably worth having a 'request for feature requests' thread,
> and maybe also mirroring that somewhere else (a github issue? a blog post?
> all of the above plus a bat signal?).

As a matter of disclosure:  I am an off-and-on user of DBIC.  I have been aware 
of this project nearly since its inception, when Matt invited me to join this 
mailing list during OSCON in the summer of 2005, and have been primarily 
following it due to my general interest in database technology.  Years later I 
also became an actual DBIC user when I worked with several different companies 
that were heavily dependant on DBIC in their codebases, though I am not 
presently with them.  My own stake in the general case is to help improve the 
experience of using databases in Perl applications.  I consider helping to 
improve DBIC to be a means to that end rather than and end of its own.  AFAIK, 
DBIC is the most popular interface for databases in Perl today besides its 
dependency DBI, and so any serious effort I would make towards my goal would 
also substantially involve helping DBIC specifically.  But I don't have a stake 
in DBIC maintaining a popularity lead over alternatives, only that the Perl 
database technologies as a group continue to improve for users.

My main proposed DBIC feature is compatability with my alternative to the Perl 5 
DBI which has the working name DBP for "DataBase Protocol".  Initially this 
means adding support for DBP alongside DBI but later on DBIC should only need to 
support DBP directly.

DBP is intended to be a "Plack for databases".  It occupies a similar level of 
abstraction as DBI but it is designed more for 2017 best practices than 1995 
best practices.  Using DBP should bring as much advancement to the Perl 
ecosystem as using PSGI/Plack did, if not more.

The single most important thing is that the DBP ecosystem is "shared-nothing". 
That is, DBP would actually just be an API specification document for a 
duck-typing/etc API that conforming libraries and applications would implement 
for themselves.  A conforming library would either provide the API or consume 
the API.  Example providers are updated versions of or wrappers of DBD::SQLite 
or DBD::Pg and so on.  Example consumers are either end-user applications or 
some kinds of middleware such as updated versions of DBIx::Class or Rose::DB and 
so on.  Sometimes you can have libraries that are both providers
and consumers, such as a proxy layer or an emulation layer.  There would
absolutely NOT be any code/module/class/role/etc named "DBC" which consumers
would need to "use" to mediate access to providers.  There would absolutely not
be any mandatory shared dependency such as "DBC" to use the shared API.  This
doesn't rule out the existence of libraries that provide implementation code
which API providers can choose to depend on in common, such as DBI is often used 
for today, but this would strictly be an implementation detail of the providers, 
and not mandatory for using "DBC".

Therefore, the main net user-visible result of migrating DBIC from DBI to DBP is 
simply to remove DBI and any DBD::* modules from its hard dependency list, so 
DBIC can be installed and run without having to also install that dependency chain.

Instead, any application or other middleware that has the hard dependency on 
DBIC, it would also have explicit direct dependencies on 1 or more DBD::* 
modules that it is actually using, either in its code with "use" or in its 
config files for a runtime-defined DBD::* dependency.

Similarly, the DBIC test suite would be like such an application, the test suite 
itself would name a DBD::* etc dependency or dependencies either in code or in a 
config file so that the test suite can operate and test everything DBIC can do 
which requires a DBMS external to itself.

Given the depth of internals changes to some parts of DBIC that this involves, 
along with at least a small user interface change (anything that takes or emits 
a user-supplied DBI object), I fully expect this work to be treated as a major 
feature branch and not something that would be released like it were a mere bug 
fix release.  It is possible for DBIC to provide 100% backwards compatibility to 
its users in the new version so they don't need to change anything, but the most 
effective users would involve small application changes in some corners.

I am still in the process of building reference provider implementations of the 
DBP spec, as well as writing the DBP spec itself, but they will hopefully be 
ready to use soon.  In the meantime, you can see part of the spec on GitHub here:

    https://github.com/muldis/Muldis-DBP-Perl5

And its corresponding Perl 6 version:

    https://github.com/muldis/Muldis-DBP-Perl6

I had previously discussed this effort here:

   http://www.nntp.perl.org/group/perl.dbi.dev/2016/11/msg8002.html

I will post on this again when I have something ready to execute.

-- Darren Duncan