[Catalyst] Re: Large requests with JSON?

Aristotle Pagaltzis pagaltzis at gmx.de
Tue Feb 9 19:27:15 GMT 2010

* Bill Moseley <moseley at hank.org> [2010-02-09 16:10]:
> On Tue, Feb 9, 2010 at 2:36 AM, Aristotle Pagaltzis <pagaltzis at gmx.de>wrote:
> > That sounds like the case I was thinking about: just do a PUT
> > request with X-MyApp-Filename, X-MyApp-Timestamp etc headers.
> Of course, I left out the ability to upload multiple flies at
> once. Doing that with headers could get ugly.
> (X-MyApp-Filename-01, X-MyApp-Filename-02, ...) Of course,
> could just not provide that multiple-file upload ability to API
> users and limit it to web users. That would work ok.

I would seriously just not provide multiple uploads via the API.
For the browser UI they’re a necessity because it’s so awkward to
upload files one at a time, but the API is a completely different
category. This falls under “batching”, and all the HTTP sages
will tell you “don’t do that”. It makes both the server and the
client more complicated without any discernible upsides. (In
fact, if you do pipelining, then separate PUT requests are
actually more efficient in terms of roundtrips and overhead.)

> From past experience I can assume some customers will have
> trouble adding request headers for the libraries they are
> using.

That would be a problem, yes. (Damn people treating HTTP as
a transport protocol… *mutter*)

> form-data is possible serialization, but it's a flat
> serialization so also need to have fields like filename_01,
> title_01, filename_02, title_02  to handle multiple uploads at
> once. (Plus, the app already handles that form-data).

Just don’t do batch uploads in the API.



> JSON provides the nice nested structures but, IIUC, has to be
> in-memory to parse.

Not in principle, although it may well be that there isn’t any
library that implements a streaming parser yet.

> Not pretty at all, but maybe using form-data with
> a JSON-encoded "meta" field that has a list of uploads with
> associated meta-data including a field_name with each upload
> that associates it with a field that contained the uploaded
> file. Most client libraries have a way to send form-data, so
> that would be easy for customers to implement.
> None of those are great options.

Actually, that sounds like a decent option if you really need
a nested data structure and can’t use headers. (I’d still not
do batch uploads, though.)

Aristotle Pagaltzis // <http://plasmasturm.org/>

More information about the Catalyst mailing list