[Catalyst] UTF-8 with Catalyst::Controller::REST and JSON::Syck

[Francesc Romà i Frigolé]

On Mon, Dec 15, 2008 at 6:37 PM, J. Shirley <jshirley at gmail.com> wrote:
> On Mon, Dec 15, 2008 at 9:09 AM, Francesc Romà i Frigolé
> <francesc.roma+catalyst at gmail.com> wrote:
>> Hello,
>>
>> I have a controller which derives from Catalyst::Controller::REST, and
>> my data is in UTF-8 format. When I deserialize  with JSON the data is
>> not displayed properly. Everything else works fine with utf-8:
>> deserialization in xml works, and non-restful controllers work as
>> well.
>>
>> According to the manual Catalyst::Controller::REST uses JSON::Syck for
>> json serialization, but when I try to access variables like
>> $JSON::Syck::ImplicitUnicode or $JSON::Syck::SingleQuote from within
>> my controller they are undefined.
>>
>> How can I make a restful controller send utf-8 encoded data?
>>
>> Thanks,
>> Francesc
>
>
> Catalyst::Controller::REST actually just uses JSON.pm now... that Pod
> just missed an update (unfortunately).  Sorry for the confusion there.
>
> JSON.pm will use JSON::XS if available, and failing that use a
> pure-Perl implementation.  Can you check your JSON and JSON::XS
> versions, and if possible write a failing test case for the utf-8 data
> if it still isn't working?  I'll get a patch in to fix the pod, and
> would like to fix any unicode/utf-8 issues that happen with the code
> (though to my knowledge I don't think there should be any, and I am
> using some JSON+UTF-8 combinations in my applications)
>
> Thanks,
> -Jay
>


Jay,

Thanks for your informative reply.

After more testing I realized that the problem is not caused by the
JSON serializer: if I serialize a perl literal in a utf-8 encoded file
the data I get in the browser is correct. Sorry, I should have tested
this before posting.

It seems then that the data gets wrongly encoded somewhere between the
database and the serializer.

Now I still need to figure out why the data is sent properly in my
other (not restful) controllers, but fails in this one controller.

Thanks again,
Francesc