[Catalyst] Re: Catalyst Unicode woes ...

[Carl Franks]

Aristotle, thanks for your input - as soon as I saw your name in this
thread, I knew to sit up and take notice :)

It's taken a few hours, but I've figured out what's causing the
specific problems Tobias was having with some parts of the page being
double-encoded.

In a nut-shell, it's YAML::Syck's fault!

>From what I can make of the XS in YAML::Syck, it's messing with the utf8 flags.
So even if I pass YAML::Syck::LoadFile() a filehandle opened using the
correct encoding, the data returned is either encoded again or
incorrectly flagged (I'm not sure which).

If I then concatenate a string from YAML::Syck, with a string from a
correctly decoded filehandle, then the portion from YAML::Syck gets
double-encoded by Catalyst::Plugin::Unicode::finalize() doing
utf8::encode().

I've tried using YAML::XS instead of YAML::Syck, but that also
produces the same broken result.

For the moment, I can fix Tobias' problem by making HTML-FormFu
decode() all strings coming from a YAML file - but this is definitely
a temporary hack - I'll open an RT ticket on YAML::Syck and see what
the authors think.

Carl