[html-formfu] FormFu and Unicode

[Tobias Kremer]

Hi Carl,

following up on my previous post about problems with the DateTime::Locale
generated month names of the new date element I'd like to elaborate and discuss
the topic a little bit more.

IMHO, it all comes down to wrong Unicode handling. I see two potential problems
concerning HTML::FormFu:

1) The YAML parser that reads the form definition must assure that the file is
properly decoded (e.g. utf-8) into Perl's internal format.

2) The same goes for the TT snippets. TT has an undocumented option called
"ENCODING" which should be set to whatever format the template files are (e.g.
"utf-8") - See the Template::Provider source for details.

I believe that one or both of these conditions are causing the problems I'm
having with FormFu. Whenever Perl encounters a string it doesn't know the
encoding of (e.g. a field comment from the YAML file) and has to concatenate it
(via TT) with a string that is properly utf8-flagged (e.g. the output of
DateTime::Locale->month_names) it silently upgrades the unknown string to utf8
which results in a double-encoding. At least that's how I understand things. I
must admit that I just started taking a look at Perl's Unicode handling and
could be completely wrong :)

What do you think?

--Tobias