[Catalyst] Re: Patch for Catalyst::Plugin::Unicode::Encoding

[Bill Moseley]

On Sat, Mar 22, 2008 at 12:48:19AM -0500, Jonathan Rockway wrote:
> The need to check if something is already decoded hints at the existence
> of other bugs.

Or design flaw.  The rule is you decode and encoded at your
application's borders.  Catalyst makes it less easy to define where
those are considering the plugin system.

Maybe a more appropriate place to decoded body parameters is in
HTTP::Body as that's what is decoding the parameters from the input
stream.  Should use the request headers to determine the encoding,
I suspect, not a hard-coded encoding.

For XMLRPC requests I use an HTTP::Body:: subclass[1] to feed chunks to
XML::Parser, so in effect that's decoding with HTTP::Body -- so seems
like a natural place.

And query parameters would be decoded when first parsed from the
request.


You didn't respond to my last post with more specifics, but again
encoding should be pushed to the application border -- so I would
think Catalyst would look at the charset in the response headers (and
negotiate with the request headers) and use that to encode.

That would allow setting a configuration charset (like the current
plugin does) and also provide a way for a specific response to
override the encoding.

(What to do with a file handle in the body is something that would
need more discussion.)

Still, the design of Catalyst means that someone can always break this
by overriding, but I do think encoding and decoding are low-level
enough that they should be core and not a plugin.

> Encoding/decoding really boils down to knowing *for
> sure* what data you already have.  If we don't know what data we have,
> we can't safely decode it.

Well, one thing is for sure: if the utf8 flag is set we definitely
can't decode because if we do Perl with throw an exception.

But, moving the encoding and decoding to the engine pushes it out
toward the boarders more and makes it more clear to developers that
encoding and decoding should happen in a well defined place.

> Is anyone having a problem with decoding things that are already
> decoded?  Or is this only a theoretical problem?

You think I just wonder CPAN looking for broken code? ;)  I've noticed
the problem in the past with body_parameters not being decoded where
parameters were.  And what prompted this thread was testing utf8
content in XMLRPC requests.  It was hard to ignore the exceptions Perl
was throwing.



[1] and as a result I end up with HTTP::Body params and uploads
automatically so it makes the XMLRPC requests reasonably transparent
to the controllers.

-- 
Bill Moseley
moseley at hank.org