[Catalyst] Re: decoding in core

[Octavian Râşniţă]

From: "Peter Karman" <peter at peknet.com>
 Neo [GC] wrote on 02/23/2009 09:41 AM:
>
>> Does anyone know a _safe_ method to convert _any_ string-scalar to utf8?
>> Something like
>> anything_to_utf8($s)
>> , regardless if $s contains ascii, latin1, utf8, tasty hodgepodge or hot
>> fn0rd, utf8-flag is set or not and is neither affected by full moon nor
>> my horrorscope, _without_ doing double-encoding (there MUST be some way
>> to determine if it already is utf8... my silly java editor can do it and
>> perl makes difficult things at least possible).
>>
>>
>> I would greatly appreciate this philosophers stone and will send my hero
>> a bottle of finest bavarian (munich!) beer called Edelstoff ("precious
>> stuff" - tasty).
>>
>
> Search::Tools::UTF8::to_utf8() comes close. It won't handle mixed
> encoding in a single string (which would be garbage anyway) but it does
> try to prevent double-encoding and uses the Encode goodness under the 
> hood.
>
> -- 
> Peter Karman  .  peter at peknet.com  .  http://peknet.com/

I understand that there are reasons for not transforming all the encodings 
to UTF-8 in core, even though it seems to be not very complicated, because 
maybe there are some tables that contain ISO-8859-2 chars and other tables 
that contain ISO-8859-1 chars, and when the data need to be saved, it should 
keep its original encoding.

But if somebody wants to create a new Catalyst app, with a new database, new 
templates, controllers, etc, I think it could be very helpful if the 
programmer would only need to specify only once that he wants to use UTF-8 
everywhere - in the database, in the templates, in the configuration files 
of HTML::FormFu, in the controllers, and not in more places in the 
configuration file, or specify UTF8Columns in DBIC classes...
It could be a kind of default.

Octavian