[Catalyst] More detailed proposal for changes related to content
negotiation and REST
John Napiorkowski
jjn1056 at yahoo.com
Mon Aug 12 21:05:05 GMT 2013
> On Monday, August 12, 2013 4:56 PM, Marius Olsthoorn <olsthrn at gmail.com> wrote:
> > I'm not sure if it fits this discussion, but I think automatic JSON
> serialization is problematic since the serializer often has to guess
> if a value needs to be serialized as a string or a number. This is not
> a problem if you consume it with Perl or Javascript but it might be
> problematic for some other languages (I have not seen it though).
>
Any more votes in favor of dropping the automatic global serialization stuff? If so I will edit the spec and resubmit.
Lets not be afraid to kill bad ideas!
John
> Regards,
> Marius
>
> On 12 August 2013 22:26, John Napiorkowski <jjn1056 at yahoo.com> wrote:
>>
>>
>>
>>
>>
>>> On Monday, August 12, 2013 2:33 PM, Alexander Hartmaier
> <alexander.hartmaier at t-systems.at> wrote:
>>> > On 2013-08-12 16:58, John Napiorkowski wrote:
>>>> Hey Bill (and the rest of you lurkers!)
>>>>
>>>> I just updated the spec over at
>>>
> https://github.com/perl-catalyst/CatalystX-Proposals-REStandContentNegotiation/blob/master/README.pod
>>>>
>>>> I decided to remove regular expression matching from the body
> parser stuff,
>>> which I think resolves the merging questions (since now there will not
> be a
>>> possibility that more that one can match the request). I think we can
> add this
>>> back eventually using some standard request content negotiation, using
> mime type
>>> patterns and quality settings, so that we can have some rules that
> dictate what
>>> is the best match, rather than try to invent our own. For example:
>>>>
>>>>
> https://metacpan.org/module/HTTP::Headers::ActionPack::ContentNegotiation
>>>>
>>>>
>>>> The idea would be to not reinvent. I think we could instead of
> doing an
>>> equality match here we just use something like this to figure out what
> the best
>>> match is works pretty well. Thoughts?
>>>>
>>>> jnap
>>>
>>> Hi John,
>>> I thought about it for the last few days and wonder why the, lets call
>>> it rendering, of the data isn't left to the view as defined by the
> MVC
>>> pattern?
>>> I'd expect that a different view is used depending on the
> negotiated
>>> content-type.
>>> How do other MVC frameworks choose the view to use?
>>> Should a single action be able to limit the output format or is
>>> controller level granular enough?
>>>
>>> Best regards, Alex
>>>
>>>>
>>
>> Alex,
>>
>> I think you put your finger on one of the major uneasiness I (and others)
> have around the idea of having in the global application model all these
> registered formatters. Yes, in theory it feels like we are cheating on the View
> side of MVC. I have personally always thought that Catalyst doesn't exactly
> get it right the way it is (I think C and V are actually a little too detached
> for one thing) and that leads to odd code sometimes. The commonly used
> Catalyst::Action::Renderview is a bit too detached for my taste. And what we
> call a View tends to mostly just be a View handler (or factory I guess). On the
> other hand the basic idea of separation of concerns is sound.
>>
>> I think the main thing is that this is not intended to replace view, but
> for the simple cases where people just want to quickly serialize data (like for
> all those ajax endpoints you need to do nowadays, not full on RESTful APIs but
> quick and dirty communication between the back and front end. Maybe that's
> not a great thing for Catalyst (and honestly I put this out there in the hopes
> of provocation some strong reactions.
>>
>> Personally I prefer to use templates even for creating JSON output, I think
> you get cleaner separation that is easier to maintain over time (I really
> don't like when I see something like ->body (json_encode
> $sql->run->get_all_rows). That feels fragile to me. On the other hand I
> see the attraction of some of the more lightweight web frameworks where they
> make it easy to just spew out JSON.
>>
>> This is partly why I sketched out an action/controller level alternative,
> with the proposed response->body_format thing and the proposed new action
> subroutine attributes (just to recap)
>>
>> sub myaction :Local {
>> My ($self, $c) = @_;
>>
>> # ...
>> # ...
>>
>> $c->response->format(
>> 'application/json' => sub { json_encode $stuff },
>> # ...
>> # ...
>> );
>> }
>>
>> I think this approach feels similar to how some other frameworks operate.
> Some offer more sugary syntax for the common stuff, perhaps
>>
>> $c->response
>> ->json( sub { ... } )
>> ->html ( sub { ... } ).
>> -> ...
>> -> ... ;
>>
>> and I guess we could say there's a shortcut to forward to a View
> instead
>>
>> $c->response
>> ->json("JSON")
>> ->html ("TTHTML").
>> -> ...
>> -> ... ;
>>
>> But that can all be worked out after the basic thought is in place.
>>
>> and again, some other frameworks (some java system) they use annotations
> similar to our action level subroutine attributes. I think we also try to hit
> that with the proposed "Provides/Consumes" attributes. The main thing
> is I can't see a way to properly do content negotiatin with ssubroutine
> attributes given the exiting catalyst dispatcher (basically the system is mostly
> a first match win)
>>
>> Perhaps that is all we need, and we can skip idea of needing default global
> body formatters? Or maybe we'd prefer to think about leveraging more of
> Web::Dispatch, and mst has this great notion of setting response filters, which
> we could get for free if we use web-dispatch. Instead of setting a global point
> for the encoding, we could control is more granularly that way.
>>
>> I guess ultimately it comes down to a question over do we need a full on
> view for handling REST and straight up data encoding. Personally I do think
> there is a use case here that Catalyst isn't hitting right, and I am pretty
> sure some of the ideas in the stand alone Catalyst-Action-REST do apply but
> I'd like to see that more native and probably scoped more tightly (I
> don't think we need or should have the full CAR in core, but I do think we
> should ask ourselves.
>>
>> I guess its a bit tough to look at other frameworks since one thing about
> Catalyst is that our idea of a Controller isn't so central as in other
> frameworks, since with action chaining all the fun happens in the action
> really. ALthough chaining is powerful it does lead to some confusions in terms
> of how to lay out the applications and so forth.
>> I have some thoughts about that, but its really aimed at the future.
>>
>> I guess we could just drop the global format stuff, given the questions and
> controversies. I'd love to find a way that doesn't suck for people to
> be able to do JSON response in Catalyst without a lot of boilerplate, but maybe
> Catalyst isn't aimed to cater to that... The Catalyst::View::JSON is not too
> bad, just has some docs that need updating I think.
>>
>> More thoughts and comments?
>>
>>>>
>>>>
>>>>> On Friday, August 9, 2013 5:38 PM, John Napiorkowski
>>> <jjn1056 at yahoo.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>> On Friday, August 9, 2013 4:52 PM, Bill Moseley
>>> <moseley at hank.org> wrote:
>>>>>
>>>>>
>>>>>>
>>>>>> On Fri, Aug 9, 2013 at 12:11 PM, John Napiorkowski
>>> <jjn1056 at yahoo.com>
>>>>> wrote:
>>>>>>
>>>>> What's the use case you have in mind? Something like
> first check
>>> for
>>>>> something like 'application/vnd.mycompany.user+json'
> and then
>>> fall back
>>>>> to 'application/(?:vmd.*+)?json' if you don't find
> it? Is
>>> that an
>>>>> actual case you've come across?
>>>>>>
>>>>>> Ya, that's kind of what I was thinking. Or also
> having a
>>> final
>>>>> fallback parser that tries to figure out the type by other
> means than
>>> just
>>>>> looking at the Content type provided in the request. Or even
> a
>>> '.'
>>>>> final match-anything that does some special logging.
>>>>>>
>>>>>> It would be easy enough to find out if application/json
> was in the
>>> array
>>>>> more than once by mistake.
>>>>>>
>>>>> Seems like a reasonable use case then, although I would
> encourage
>>> future
>>>>> development to aim at putting more specificity in the
> controller,
>>> rather than
>>>>> rely on the global application. The primary reason to have
> anything
>>> here at all
>>>>> is to have a base people can build on. I do fear the
> globalness of it,
>>> but it
>>>>> seems not an unreasonable compromise based on how Catalyst
> actually
>>> works today.
>>>>>
>>>>>>
>>>>>>
>>>>>>> We've spoken before about the parsing larger
> incoming and
>>> chunked
>>>>> data thing before. I would love to address this, but right
> now it
>>> seems like
>>>>> something we need better agreement on in the psgi level. For
> example,
>>> since
>>>>> star man already buffers incoming input, it feels silly to me
> to have
>>> catalyst
>>>>> then try to re-stream that. You've already paid the full
> price of
>>> buffering
>>>>> in terms of memory, and performance right? Or am I not
> understanding?
>>>>>>
>>>>>> I added a Plack middleware to handle chunked encoded
> requests -- I
>>> needed it
>>>>> for the Catalyst dev server and for Apache/mod_perl. Yes,
> Starman
>>> already
>>>>> de-chunks and buffers and works perfectly.
>>>>>>
>>>>>> Apache actually de-chunks the request, but doesn't
> update the
>>>>> Content-Length header and leaves on the Transfer-Encoding:
> chunked
>>> header. So,
>>>>> sadly, I do flush this to a temporary file only to get the
>>> content-length to
>>>>> make Catalyst happy.
>>>>>>
>>>>> Right, so I think in the end we all agreed it was psgi that
> should be
>>>>> responsible for dealing with chunks or whatever (based on the
> http
>>> level support
>>>>> of the server). The only think would be could there be some
> sane
>>> approach that
>>>>> exposed the input stream as a non blockable file handle that
> has not
>>> already
>>>>> been read into a buffer (memory or otherwise). I do see the
> possible
>>> advantage
>>>>> there for processing efficiently large POST or PUT. However
> again this
>>> needs to
>>>>> be done at the PSGI level, something like input.stream or
> similar.
>>> That would
>>>>> smooth over chucked versus non chunked and expose a readable
> stream of
>>> the input
>>>>> that has not yet been buffered.
>>>>>
>>>>>>
>>>>>> I'd really like to have something at the Catalyst
> level that
>>> sanely
>>>>> acheives this end, but I think part of the price we paid when
> going to
>>> PSGi at
>>>>> the core, is that most of the popular plack handlers are pre
> loading
>>> and
>>>>> buffering input, even large request input. This seems to be
> an area
>>> where it
>>>>> might behoove us to work with the psgi group to find something
> stable.
>>> Even the
>>>>> optional psgix.io isn't always going to work out, since
> some people
>>>>> don't want to support that in the handler (its a somewhat
> vague
>>> definition I
>>>>> guess and makes people uncomfortable).
>>>>>>> Until them, or until someone helps me understand that
> my
>>> thinking is
>>>>> totally wrong on this score, it seems the best thing to do is
> to put
>>> this out of
>>>>> scope for now. That way we can move on supporting a goodly
> number of
>>> real use
>>>>> cases.
>>>>>>
>>>>>> Agreed.
>>>>>>
>>>>>>
>>>>>>> I intended to say that $_ equals a string that is the
> buffered
>>> request
>>>>> body. This way we can reserve other args for handling the
> future
>>> streaming
>>>>> case. I was actually pondering something were the sub ref
> returns a
>>> sub ref
>>>>> that gets called over and over to do the parse.
>>>>>>
>>>>>> I just don't want file uploads in memory. (Oh, I
> have another
>>> post
>>>>> coming on that -- thanks for the reminder.)
>>>>> Well, Catalyst doesn't but I think Starman might depending
> on the
>>> size of
>>>>> the incoming. However I think you can override that with a
> monkey
>>> patch.
>>>>>> >
>>>>>>> I not quite sure about $c->res->body( \%data
> ); I
>>> think body
>>>>> should be the raw body. What does $c->res->body
> return? The
>>> serialized
>>>>> json? The original hashref?
>>>>>>>>
>>>>>>> I'm not sure I like it either. I would say body
> returns
>>> whatever
>>>>> you set it to, until the point were encoding happens. It does
> feel a
>>> bit flaky,
>>>>> but I can't actually put my finger on a real code smell
> here.
>>>>>>> Any other suggestions? This is certainly a part of
> the
>>> proposal that is
>>>>> going to raise doubt, but I can't think of something
> better, or
>>> assemble
>>>>> problematic use cases in my head over it either.
>>>>>>
>>>>>> I don't really mind adding to
> $c->stash->{rest}.
>>> It's
>>>>> kind of a staging area to put data until it's ready to be
> encoded
>>> into the
>>>>> body. I might get it partially loaded with data and then
> never use it
>>> and
>>>>> return some other body. Noting precludes that, of course.
> Ya, tough
>>> one.
>>>>> Well, I definitely don't want to stick this in the stash,
> you all
>>> will have
>>>>> to tie me down to get that past! Given that body already
> allows a file
>>> handle,
>>>>> I thought adding into that would be the most simple thing, but
> lets
>>> give it more
>>>>> thought and maybe some other minds will come up with better
> ideas.
>>> I'll
>>>>> bounce it off t0m and mst as well.
>>>>>
>>>>>
>>>>>>>> If a parser dies what kind of exception is
> thrown? You
>>> say they
>>>>> would not set any response status, but wouldn't we want to
> catch
>>> the error
>>>>> and then set a 400? (I use exception objects that carry http
> status, a
>>> message
>>>>> to return in the body and a message used for logging at a
> given level.)
>>>>>>> How people do exceptions in Perl tends to be nearly
> religious,
>>> and I
>>>>> didn't want to hold this up based on figuring that stuff
> out :) I
>>> was
>>>>> thinking to just raise an exception and let the existing
> Catalyst stuff
>>> do its
>>>>> thing. I'm just thinking not to add anything special for
> this type
>>> of
>>>>> error, but just do the existing behavior, for better or worse.
>>>>>>
>>>>>> Agreed. If I were to write everything from scratch again
> I'd
>>> be doing
>>>>> $c->throw_not_found or $c->throw_forbidden with
> exception objects
>>> as the
>>>>> code ends up much cleaner and sane. But, everyone has their
> own
>>> approaches.
>>>>>>
>>>>> One thing is to have the response->from_psgi thing which
> would make
>>> ti easy
>>>>> to graft in something like
> https://metacpan.org/module/HTTP::Throwable
>>>>>
>>>>> sub myaction :Local {
>>>>> my ($self, $c) = @_;
>>>>> $c->res->from_psgi( http_throw({
>>>>> status_code => 500,
>>>>> reason => 'Internal Server Error',
>>>>> message => 'Something has gone very wrong!'
>>>>> }))
>>>>> }
>>>>>
>>>>> somthing along those lines I think.
>>>>>
>>>>>>
>>>>>> since request->body_data is intended to be lazy, we
> won't
>>> run that
>>>>> parse code until you ask for the data. We don't need to
> parse the
>>> data to
>>>>> do the basic match here, this is just based on the HTTP meta
> data, no
>>> the actual
>>>>> content. I think for common cases this is fine (I realize
> that yet
>>> again this
>>>>> might not be the best approach for multipart uploads...)
>>>>>>
>>>>>> Another tough one. Just seems like PUT /user should
> accept the
>>> same data
>>>>> regardless of how it is serialized. And GET /user would get
> the user
>>> data and
>>>>> then serialize that to JSON or whatever but it's the same
> data.
>>>>>>
>>>>>> But, maybe you have a point. I would worry that someone
> assumes
>>> JSON and
>>>>> adds that content type match and then wonder why later
> it's not
>>> working for
>>>>> other request serializations.
>>>>>>
>>>>> well strikely speaking restful content negotiation should tell
> the
>>> client what
>>>>> can and can't be accepted, for other purposes we have
> docs. I
>>> think its
>>>>> safe for the first go to just support json since for one of
> the main
>>> use cases,
>>>>> making it easy for people building websites with some ajax
> forms, that
>>> is all
>>>>> you need. For more hard core REST I could easily see
> returning data
>>> very
>>>>> differently based on what is asked. Like an endoint could
> serve an
>>> image if you
>>>>> ask for png, but metadata on the image if you ask for json.
>>>>>
>>>>>
>>>>>>
>>>>>> --
>>>>>> Bill Moseley
>>>>>> moseley at hank.org
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> List: Catalyst at lists.scsys.co.uk
>>>>> Listinfo:
> http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
>>>>> Searchable archive:
>>> http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
>>>>> Dev site: http://dev.catalyst.perl.org/
>>>>>
>>>> _______________________________________________
>>>> List: Catalyst at lists.scsys.co.uk
>>>> Listinfo:
> http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
>>>> Searchable archive:
> http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
>>>> Dev site: http://dev.catalyst.perl.org/
>>>
>>>
>>>
>>>
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
>>> T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
>>> Handelsgericht Wien, FN 79340b
>>>
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
>>> Notice: This e-mail contains information that is confidential and may
> be
>>> privileged.
>>> If you are not the intended recipient, please notify the sender and
> then
>>> delete this e-mail immediately.
>
>>>
>>>
> *"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
>>>
>>> _______________________________________________
>>> List: Catalyst at lists.scsys.co.uk
>>> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
>>> Searchable archive:
> http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
>>> Dev site: http://dev.catalyst.perl.org/
>>>
>>
>> _______________________________________________
>> List: Catalyst at lists.scsys.co.uk
>> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
>> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
>> Dev site: http://dev.catalyst.perl.org/
>
More information about the Catalyst
mailing list