[Catalyst] Re: Unicode trouble with Catalyst::Engine::FastCGI

[Aristotle Pagaltzis]

Hi Bernhard,

* Bernhard Graf <catalyst4 at augensalat.de> [2009-11-23 20:00]:
> Aristotle Pagaltzis schrieb:
>
>>> While this fixes the problem, it is still unclear, why the
>>> utf8 flag is set for the whole buffer.
>>
>> It shouldn’t matter.
>
> But it does.

yes, because ::FastCGI is broken. :-)  Is what I’m saying.

>>> So Ladies and Gentleman, may I present you the culprit? It
>>> is YAML::XS! Everything read by YAML::XS
>>>
>>> perl -MYAML::XS -E '
>>> my $config = YAML::XS::LoadFile("myapp.yml");
>>> say((utf8::is_utf8($config->{name}) ? "is" : "is not"), " utf8");
>>> '
>>> is utf8
>>
>> No, that’s not the culprit.
>>
>> The culprit is Catalyst::Engine::FastCGI, which does not pay
>> attention to the UTF8 flag.
>
> Obviously YAML::XS doesn't do that either.

It does. It sets the flag to correctly reflect the state of the
internal byte buffer of the scalar so that its string value will
mean the right thing.

> But aside from that I agree with you, that something ist broken
> in F(ast)CGI. It seems more that F(ast)CGI pays attention to
> the utf8 flag where it shouldn't, because it seems to
> automatically utf8::encode the buffer due to the set utf8 flag.

Ah, that may be, yeah, it would produce the result you are
seeing, and would clearly be broken. C::E::FastCGI should just
output the string as it is and let perl worry about its meaning,
rather than (if that’s what it does) checking the UTF8 flag
itself and doing something broken in response.

> > No. YAML::XS is completely correct.
>
> No, it is not. Because it claims:
>
>   This module exports the functions Dump and Load. These
>   functions are intended to work exactly like YAML.pm's
>   corresponding functions.
>
> And that's not the case. And therefore you can't use it as
> a drop-in-replacement for other YAML modules.

Hmm. I guess it depends on whether think of “drop-in” as
something that replicates behaviour down to internal
implementation details, or just functions identically on the
semantic level. When I see a module advertise itself like that
I assume only the latter.

Regards,
-- 
Aristotle Pagaltzis // <http://plasmasturm.org/>