[Catalyst] Catalyst crashing hard with UTF-8 string

[Paul Makepeace]

On Mon, Jul 13, 2009 at 9:33 AM, Paul Makepeace<paulm at paulm.com> wrote:
> Summary: this turns out to be an issue with Catalyst::Engine::HTTP's
> Remote handle not being set to UTF-8.
>
> On Wed, Jul 8, 2009 at 4:29 PM, Tomas Doran<bobtfish at bobtfish.net> wrote:
>>
>> On 8 Jul 2009, at 18:01, Paul Makepeace wrote:
>>
>>>> You know it's dieing inside TT, right? So you can Data::Dumper the input
>>>> which causes it to die to a file, write a program that instantiates
>>>> View::TT, calls ->render with the same input (and template), and that
>>>> should
>>>> crap out in the same way?
>>>
>>> Ya, I was kinda hoping you wouldn't say this, and that there was a way
>>> to catch whatever was happening in Catalyst or trace the execution
>>> path to get to the point where the things actually dieing.
>>
>> Well, start by throwing Devel::SimpleTrace, or Carp::Always at it to get a
>> strack trace.
>>
>> MyApp::View::TT->render is going to get called, so add something like this
>> to that class:
>>
>> use Data::Dumper;
>> sub render {
>>    my ($self, $c, $template, $args) = @_;
>>    local $Data::Dumper::Maxdepth = 4;
>>    warn Dumper([$template, $args]);
>>    $self->next::method($c, $template, $args);
>> }
>>
>> increase Maxdepth if needed until it pukes..
>>
>> You should literally be able to dump the Dumper glob into a .t file, and
>> build a test around it....
>>
>> use MyApp;
>> use MyApp::View::TT;
>>
>> my $view = MyApp::View::TT->new(%config_your_app_gives_TT_view);
>> my $VAR! = # Dumper crap here
>>
>> my ($template, $args) = @$VAR1;
>>
>> $view->render('MyApp', $template, $args); # Should blow up, in the 'correct'
>> way..
>
> Thanks for this. Since what I ended up is a little different from
> what's here I'll copy it for anyone else going down this path,
>
> use MyApp;
> use MyApp::View::TT;
>
> my $c = MyApp->new;
> my $view = MyApp::View::TT->new($c, {});
>
> my $error = "It\x{2019}s crazy";
> my $template = \'[% foo %]';  # or 'path/to/template';
>
> my $output = $view->render($c, $template, {foo => $error});
> binmode(*STDOUT, ':utf8');
> print "Output: $output\n";
> __END__
>
> It turns out that it's not the rendering in TT at all(!) as evidenced
> by this test script working.
>
> **
>
> So I can now produce this hard crash by replacing
> MyApp::View::TT::render with sub render { "\x{2639}" }. Digging even
> further back I found this has it work,
>
> --- /usr/local/share/perl/5.10.0/Catalyst/Engine/HTTP.pm.orig
> 2009-07-13 15:27:27.000000000 +0100
> +++ /usr/local/share/perl/5.10.0/Catalyst/Engine/HTTP.pm
> 2009-07-13 15:25:58.000000000 +0100
> @@ -263,6 +263,7 @@
>             select Remote;
>
>             Remote->blocking(1);
> +            Remote->binmode(':utf8');
>
>             # Read until we see all headers
>             $self->{inputbuf} = '';

Ah, awesome, this breaks multipart upload.

If the utf8 encoding is set just before the handler it seems to work,

--- /usr/local/share/perl/5.10.0/Catalyst/Engine/HTTP.pm.orig
2009-07-13 15:27:27.000000000 +0100
+++ /usr/local/share/perl/5.10.0/Catalyst/Engine/HTTP.pm
2009-07-13 16:14:47.000000000 +0100
@@ -288,6 +288,7 @@
                     }
                 }

+                Remote->binmode(':utf8');
                 $self->_handler( $class, $port, $method, $uri, $protocol );

                 if ( $self->_has_write_error ) {

P

> I don't know enough about how to make this a general solution
> (presumably not all Catalyst::Engine::HTTP output will be UTF-8).
>
> I'm still not sure why it's die'ing or why Apache is serving this
> Proxy Error / thinking the output is unexpected. Maybe a 'wide print'
> error is ending up in stderr or something. Any tips on this
> appreciated.
>
>> Cutting template / data down to smallest replicable size and throwing away
>> Catalyst app and using raw TT should be easy from there forwards :)
>>
>> Another thought - have you tried disabling the XS stash for TT? (IIRC there
>> is an env var to do this) and seeing if that affects the crash?
>
> As it turns out from above I didn't really need to pursue this but
> again for the record, the way AIUI to do this is,
>
> use Template::Config;
> $Template::Config::STASH = 'Template::Stash';
> __END__
>
> If there's an env var I didn't see it.
>
> (Exploring this one of my templates threw up a bug in Template::Stash
> though; reported to the TT list.)
>
> Paul
>