[Catalyst] Problems with Encoding support in Catalyst.

[John Napiorkowski]

Hi Bill,
is_utf8 probably doesn't do what you think it does:
http://stackoverflow.com/questions/14579560/am-i-using-utf8is-utf8-correctly

If we are throwing errors here that we'd prefer handled differently, see if you can workup a test case and I will bounce it around.
The Binmode problem is interesting.  I was never happy with that. lets work up some test cases and I will bounce it around some of the UTF8 experts.
At the least we might want some sort of configuration option.
John 

    On Saturday, January 7, 2017 11:56 PM, Bill Moseley <moseley at hank.org> wrote:
 

 I just finally removed my custom encoding code to use the built-in code Catalyst now provides to decode input data and encode output.
But, I came across two issues that I wanted to get some feedback on.
First one is simple.  This code will die if $value is already decoded.  Any reason not to test it?

 sub _handle_param_unicode_decoding {     my ( $self, $value, $check ) = @_;     return unless defined $value; # not in love with just ignoring undefs - jnap     return $value if blessed($value); #don't decode when the value is an object.+    return $value if Encode::is_utf8($value);  # already decoded?
This next one is what I have questions about.   This is also in Catalyst.pm in the encoding() method:
        # Let it be set to undef        if (my $wanted = shift)  {            $encoding = Encode::find_encoding($wanted)              or Carp::croak( qq/Unknown encoding '$wanted'/ );            binmode(STDERR, ':encoding(' . $encoding->name . ')');

The problem with that is it seems to turn off autoflush on STDERR.   In my case that is causing intermixing of lines in the log files.
If I turn on STDERR->autoflush(1) then the logging works again and log lines are not mangled.
But, I'm not sure we should apply a layer to STDERR like that.   For example, I log to stderr with JSON and that is already encoded.
Is there a compelling reason to binmode STDERR there?



-- 
Bill Moseley
moseley at hank.org
_______________________________________________
List: Catalyst at lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/


   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.scsys.co.uk/pipermail/catalyst/attachments/20170110/da865ea8/attachment.htm>