[Dbix-class] extended paging

[Matija Grabnar]

Rodrigo wrote:
> Ideally, I think the paging algorithm somewhere should handle a 
> callback filter sub and be able to fetch a few more rows (until a page 
> is completed), in case the filter dropped. Well, something in the lines:
> ....
> At worst case, all million CD table rows are read and thrown out, 
> which could result in a performance loss, but a working paging algorithm.
I've thought of that, but it wouldn't work (without somehow saving state 
in external table or something).
Why? Imagine that you have a page with 20 lines, and a huge table. You 
fetch the first page, throw away
2000 lines, and finally get the 20 lines you need.

Then the script fetches the second page. But where do you start 
scanning? You would have to somehow remember where the end of the first 
page was. In a script that was breaking the table into one page after 
another, that's not a problem. But in a Catalyst context, it definitely 
IS a problem: you can't rely on the same process fetching page 2 as was 
the process fetching page 1. So you would have to SAVE the position of 
the end of page 1 somewhere outside the process. Which is why I talked 
about setting up another table that would hold the indexes of the pages 
in the first table.