[Catalyst-dev] A try to patch Catalyst::Engine.pm

[Jonathan Rockway]

* On Tue, Jul 29 2008, Chae Lee wrote:
> On Tue, Jul 29, 2008 at 10:29 AM, Jonathan Rockway <jon at jrock.us> wrote:
>     Why are wide characters in the output buffer?  The network cannot
>     transfer Perl wide characters; you need to convert them to bytes with
>     Encode::encode first.  ("use bytes" is not acceptable.)
>
>
> Yeah, I almost always use Encode.pm, but Engine.pm is special. it shouldn't
> bother 'output'; it should just copy the output to the web browser
> byte-by-byte.

Exactly.  This is why I asked "why are wide characters in the output
buffer".  If your application is using characters outside of 7-bit ASCII
and it isn't encoding the characters itself, it is completely broken and
could stop working correctly at any time.

> To make sure "byte-by-byte" (perl implicitly tries encoding), I
> injected "use bytes;"

So you are saying you want to output a memory dump of Perl's internals
to the web browser?  I don't know of any web browsers that will
understand that.  (perl doesn't "implicitly try encoding", you are
assuming that an internal implementation detail is an actual perl
feature.  It's not.)

> I didn't consider of the plugin simply because of the name "Unicode." Today I
> looked at the code.

How do you have wide characters in the output buffer if you're not using
Unicode?

> Good! but drawbacks:
>
> 1. the plugin deals with ONLY utf8.

And "use bytes" deals with Perl's internal representation, which is
subject to change in any minor release.  Yes, sometimes it's UTF-8 now,
which is why "use bytes" sometimes works.  But it's not guaranteed at
all, and will break when the string can be represented as
latin-1. Surely explicitly using utf-8 is better than outputting
whatever random data perl has in memory.

> 2. effective ONLY on the test server in Catalyst dist. If I apply the plugin
> to, say Lighttpd, it happens to be purely an overhead.

I don't think you understand how Unicode in Perl works.  Please google
around and educate yourself.  I (and others) have written volumes about
this.

Regards,
Jonathan Rockway

--
print just => another => perl => hacker => if $,=$"