[Catalyst] Downloading BLOBs

[Toby Corkindale]

Michael Alan Dorman wrote:
> Toby Corkindale <toby at ymogen.net> writes:
>> It's rather hacky, but I thought I'd submit it for your comments.
> 
> Unless you've got something historical to support, don't use BLOBs in
> Pg.  Seriously, with real prepared statements, supported by DBD::Pg,
> the one remaining argument for using the LO interface (that all that
> data didn't have to be parsed, making things much faster) has gone the
> way of the dodo.

So, are you suggesting that one just makes extensive use of the 
substring() Pg function to retrieve chunks-at-a-time from the field?
And include some hacks to prevent Class::DBI or DBIx::Class from 
automatically attempting to retrieve the entire BYTEA column 
(potentially a gig or two) from the DB every time you hit that record?

What is the performance of substring() like over very large BYTEA 
columns? I'm concerned that it wouldn't match the lo_read interface.
Does it support in-place writing, a-la perl's substr()? Or would one 
need to read out the entire BYTEA column in order to be able to make a 
change and write it back?


I like the lo_ interface, as it means that I can send/receive/access 
large files, yet keep a very low memory footprint. When you're talking 
about having many files being sent over the wire simultaneously, and 
slowly, it's advantageous to keep memory use low and thus let the 
process-count go high.

Cheers,
Toby