[Catalyst] UTF8 and content length

[Kroshka Yenot]

 >>> Looks like to a bug to me

tl;dr   I'm not sure its a Catalyst bug or problem. It's may be MY 
configuration problem or standard violation


Here are my investigation results


I created a test to reproduce this situation

# catalyst.pl test

# test/script/test_create.pl view HTML TT

# [editor]      test/lib/test/Controller/Root.pm

sub index :Path :Args(0)
{
     my ( $self, $c ) = @_;

     my $json_text = '{"id":1, "msg":"В Питере пить"}';
     $c->response->content_type('application/json');
     $c->response->body($json_text);
}


and found following:


  wget -S -O - http://domain.tld:3000
--2016-07-20 13:56:18--  http://domain.tld:3000/
Resolving cary.lv (cary.lv)... aaa.bbb.ccc.ddd
Connecting to domain.tld (domain.tld)|aaa.bbb.ccc.ddd|:3000... connected.
HTTP request sent, awaiting response...
   HTTP/1.0 200 OK
   Date: Wed, 20 Jul 2016 10:56:18 GMT
   Server: HTTP::Server::PSGI
   Content-Type: application/json
   X-Catalyst: 5.90106
   Content-Length: 42
Length: 42 [application/json]
Saving to: 'STDOUT'


content-Length is properly set. I see same using Firefox Dev tools

but in the log (build-in test server log)

[debug] Response Code: 200; Content-Type: application/json; 
Content-Length: unknown


Exactly same code, but app works as fastcgi daemon and Apache/2.4.23 
(FreeBSD) serves http requests

# wget -S -O - http://domain.tld/
--2016-07-20 15:02:28--  http://domain.tld/
Resolving domain.tld (domain.tld)... aaa.bbb.ccc.ddd
Connecting to domain.tld (domain.tld)|aaa.bbb.ccc.ddd|:80... connected.
HTTP request sent, awaiting response...
   HTTP/1.1 200 OK
   Date: Wed, 20 Jul 2016 12:02:28 GMT
   Server: Apache
   Set-Cookie: lang=ru; path=/; expires=Thu, 20-Jul-2017 12:02:28 GMT
   Set-Cookie: sid=3b2b88c4106b5e06c0c24a5c3a513ccbcb939299; 
domain=domain.tld; path=/; expires=Wed, 20-Jul-2016 12:52:28 GMT; HttpOnly
   X-Catalyst: 5.90106
   Content-Length: 31
   Keep-Alive: timeout=5, max=100
   Connection: Keep-Alive
   Content-Type: application/json
Length: 31 [application/json]


Content length here is in chars not in bytes

A solution by Aristotle Pagaltzis

$c->response->body(Encode::encode_utf8 $json_text);

gives proper content length in this situation

I'm getting same proper content length if I change content type to 
'text/html'


Finally, I've discovered Catalyst::View::JSON

and it not only solved this problem for me, but also gave me a much more 
comfortable solution to work with json

$c->stash->{msg} = "В Питере пить";
$c->stash->{id} = 1;
$c->forward('View::JSON');

Works like a charm


Taking this opportunity, thank you for this lovely framework!

I'll be happy to provide any additional information if you still 
consider there is something should be fixed







19.07.2016 19:10, John Napiorkowski пишет:
> Looks like to a bug to me, although I'm not personally keen on the 
> auto length setting in Catalyst it should be corrected.  I'm happy to 
> get a patch, or at the very least give me a broken test case (checkout 
> https://github.com/perl-catalyst/catalyst-runtime/blob/master/t/utf_incoming.t 
>
>
> and see if you can help me figure it out -jnap
>
> (created an issues for this, 
> _https://github.com/perl-catalyst/catalyst-runtime/issues/143
>
>
>
>
>
> On Friday, July 15, 2016 6:07 AM, Kroshka Yenot <trashbox at cary.lv> wrote:
>
>
> Hi!
> if content type is 'application/json' or 'application/json; 
> charset=utf-8' Catalyst sets content length in chars, NOT IN BYTES and 
> I'm getting
> {"id":1, "msg":"В Питере
> if content type is 'text/html' Catalyst sets content length in bytes 
> (properly) and everything works fine
> Is there any workaround to configure this behaviour, except setting 
> content length manually everytime ?
>
> my $json_text = '{"id":1, "msg":"В Питере пить"}';
> $c->response->content_type('application/json');
> $c->response->content_length(bytes::length $json_text);
> $c->response->body($json_text);
> Thanks in advance
>
>
> _______________________________________________
> List: Catalyst at lists.scsys.co.uk <mailto:Catalyst at lists.scsys.co.uk>
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: 
> http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/
>
>
>
>
> _______________________________________________
> List: Catalyst at lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.scsys.co.uk/pipermail/catalyst/attachments/20160720/5d33a5b6/attachment.htm>