[Dbix-class] 1/5 What is stability

[Peter Rabbitson]

On 10/07/2016 08:40 PM, David Golden wrote:
>
>> * As a final point on "going forward": I am concerned that the "software
>> stability" argument has been grossly micharacterized[sic]: it was presented
>> as a binary "does it lose data" argument, when for me the main question has
>> always been "is it opinionated / does it insist on usurping the end-users
>> time".
>
> I think this would be a wonderful topic for you to elaborate on.
>

Before answering "what is stable" it is worth noting what is the purpose 
of the software we provide (in this case on CPAN). Using my own words 
from a private conversation:

> I would think that a software platform should optimize for folks to consistently being able to leave the office at a reasonable time instead of manufacturing calamities every N units of time "in order to stay relevant"

It is a commonly held belief in open source that us providing our labor 
for free exempts us from the responsibility to respect the time and 
labor of the people we build for. This belief is embarrassingly 
misguided. Accepting this would be akin to accepting a volunteer 
firefighter packing up their gear and going home when things quite 
literally get too hot.

To drive this even closer home I encourage all participants of these 
threads to see the first 8 minutes of another presentation by Aral 
Balkan[1]. These first 8 minutes are a brilliant unscheduled impromptu 
commentary on the frantic attempts of the conference organizers to get a 
failed projector working. If 8 minutes is too much time to spare here is 
a small transcribed excerpt from this improv:

> ...So experiences matter. And the reason that we have to focus on building
> things that don't just "work", especially don't just "work for us", but are
> great experiences for other people: is not because we want to show off, not
> because we want to be seen as rockstars. But because we respect peoples
> experiences, we respect, fundamentally, their lives.
> And I think this (n.b. presentation hardware failure) is a beautiful way of
> making that point, so I am really glad that the projector is not working, I
> don't know about you.
> (n.b. in response to a staff member) Heh, it's "working" but it's "not
> connecting". Ok: great! See: that's great to know, but for someone sitting in
> the audience it's just not working. It could be not working for many amazing
> reasons. For things that are very interesting, that maybe we could write a
> paper on. But for you sitting in those seats: it's just not working. And it's
> the same with software and with the things that we build: if something doesn't
> work we'd love to take it apart and find out why. But for someone who needs
> that to work for them to do what they need to do: it just doesn't work. And we
> need to start building things that "just work".

So back to "what is stable" when applied to a library: quite simply it 
is reduction of churn experienced by the end user. This in no way means 
"no new features ever", but instead means planning things ahead with the 
full understanding what are the consequences of ones actions if they 
were to fail.

To put it differently here is me again from about a year ago[2]:

> I believe that if an end-user (corporate and individual alike) needs to change
> their application code upon library upgrade, while the changes do not offer
> compelling direct benefits to their codebase - this is a sign I have failed
> as a maintainer, and it is on me to find a better way, and make things right
> for the user.

CPAN has a very long history of well-, and not-so-well- maintained 
libraries. It is very rare for almost anything to just "cause data 
loss". Additionally these are the really simple cases: they are 
discovered and fixed almost instantly.

It is the subtle changes deep down in an dependency stack that have been 
made without regard to "what sits on top", that are the real problem. 
These changes take a long time to propagate to their victims, and take 
even longer time (if at all) to traverse the feedback loop back to the 
authors. For a much better and in-depth (albeit somewhat tangential) 
writeup about this, please consider reading [3].


Words are well and good, but what does this actually mean in practice? 
In retrospect "freeze" was an unfortunate choice of words, which got 
hijacked in the subsequent threads to mean "stagnation" or even 
inexplicably "death of the project". What I really meant was "API 
freeze", which has already been the case for quite some time as far as 
this project is concerned.

Instead of trying to explain this in prose let me illustrate this in 
numbers since the time I got back to DBIC at the end of 2012:

~/devel/dbic$ git log --reverse -p -C -w -U0 v0.082840 ^v0.08203 lib \
| grep -Po '^[+-]sub (?!_)\S+' \
| sort -s -t ' ' -k2 \
| grep '^-' -A1 \
| grep -v -- '^--' \
| wc -l

Places the count of ~4 years of events for non-private methods being 
moved around at 48.

Even if we add in the not-yet-released *fully backwards compatible* 
overhaul of callchains 1cf609901 [4]:

~/devel/dbic$ git log --reverse -p -C -w -U0 dc7d89911 ^v0.08203 lib \
| grep -Po '^[+-]sub (?!_)\S+' \
| sort -s -t ' ' -k2 \
| grep '^-' -A1 \
| grep -v -- '^--' \
| wc -l

we end up with 196.

As a comparison let's examine another popular perl framework during the 
same time period:

~/devel/moose$ git log --reverse -p -C -w -U0 2.1806 ^2.0600 lib \
| grep -Po '^[+-]sub (?!_)\S+' \
| sort -s -t ' ' -k2 \
| grep '^-' -A1 \
| grep -v -- '^--' \
| wc -l

This comes back at 76.

And another:

~/devel/mojolicious$ git log --reverse -p -C -w -U0 v7.08 ^v3.48 lib \
| grep -Po '^[+-]sub (?!_)\S+' \
| sort -s -t ' ' -k2 \
| grep '^-' -A1 \
| grep -v -- '^--' \
| wc -l

This comes back with 1135.


The above is by no means a robust measurement - it is obviously a 
heuristic. However it demonstrates in practical terms what "stability" 
actually means. To take a (rare on CPAN) example of an end-user App look 
at the end of commit message 12e7015a [5]: in order to be able to run 
the test suite of Expense::Tracker[6] I have to downgrade Mojolicious 
back more than 250(!) releases. And once I do that the application 
works, and passes its tests. So as far as Expense::Tracker - what is the 
benefit here? In the end it simply "doesn't work".


This is the kind of churn I have been trying to avoid with DBIC, and 
which was my #1 job (and is arguably the job of any responsible 
maintainer). Not churning out new features, but steadfastly saying "no" 
to things with questionable cost/benefit ratios.

I do not feel great about the fact that I had to turn people's code 
away, but I remain unapologetic: the code that has not yet merged is 
simply not yet good enough.


Excellence should not be negotiable.


[1] https://youtu.be/A23Xoa5CE7g?t=35
[2] 
https://gist.github.com/ribasushi/d3769fa75822b7b080e9#file-tilt_description-L197-L201
[3] https://eev.ee/blog/2016/02/10/we-have-always-been-at-war-with-ui/
[4] 
https://blog.afoolishmanifesto.com/posts/open-source-infrastructure-and-dbix-class-diagnostics-improvements/
[5] https://github.com/dbsrgits/dbix-class/commit/12e7015a
[6] https://metacpan.org/release/Expense-Tracker