<html><head></head><body><div style="color:#000; background-color:#fff; font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px"><div id="yui_3_16_0_ym19_1_1484083120986_4218"><span>Hi Bill,</span></div><div id="yui_3_16_0_ym19_1_1484083120986_4218"><span><br></span></div><div id="yui_3_16_0_ym19_1_1484083120986_4218" dir="ltr"><span id="yui_3_16_0_ym19_1_1484083120986_7129">is_utf8 probably doesn't do what you think it does:</span></div><div id="yui_3_16_0_ym19_1_1484083120986_4218" dir="ltr"><span><br></span></div><div id="yui_3_16_0_ym19_1_1484083120986_4218" dir="ltr"><a href="http://stackoverflow.com/questions/14579560/am-i-using-utf8is-utf8-correctly" class="" id="yui_3_16_0_ym19_1_1484083120986_4345">http://stackoverflow.com/questions/14579560/am-i-using-utf8is-utf8-correctly</a><span><br></span></div><div id="yui_3_16_0_ym19_1_1484083120986_4218" dir="ltr"><br></div><div id="yui_3_16_0_ym19_1_1484083120986_4218" dir="ltr">If we are throwing errors here that we'd prefer handled differently, see if you can workup a test case and I will bounce it around.</div><div id="yui_3_16_0_ym19_1_1484083120986_4417"><br></div><div id="yui_3_16_0_ym19_1_1484083120986_4416" dir="ltr">The Binmode problem is interesting. I was never happy with that. lets work up some test cases and I will bounce it around some of the UTF8 experts.</div><div id="yui_3_16_0_ym19_1_1484083120986_4416" dir="ltr"><br></div><div id="yui_3_16_0_ym19_1_1484083120986_4416" dir="ltr">At the least we might want some sort of configuration option.</div><div id="yui_3_16_0_ym19_1_1484083120986_4416" dir="ltr"><br></div><div id="yui_3_16_0_ym19_1_1484083120986_4416" dir="ltr">John</div> <div class="qtdSeparateBR" id="yui_3_16_0_ym19_1_1484083120986_5188"><br><br></div><div class="yahoo_quoted" id="yui_3_16_0_ym19_1_1484083120986_5193" style="display: block;"> <div style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16px;" id="yui_3_16_0_ym19_1_1484083120986_5192"> <div style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16px;" id="yui_3_16_0_ym19_1_1484083120986_5191"> <div dir="ltr" id="yui_3_16_0_ym19_1_1484083120986_5190"><font size="2" face="Arial" id="yui_3_16_0_ym19_1_1484083120986_5189"> On Saturday, January 7, 2017 11:56 PM, Bill Moseley <moseley@hank.org> wrote:<br></font></div> <br><br> <div class="y_msg_container" id="yui_3_16_0_ym19_1_1484083120986_5196"><div id="yiv6948327696"><div dir="ltr" id="yui_3_16_0_ym19_1_1484083120986_5195"><div id="yui_3_16_0_ym19_1_1484083120986_5194">I just finally removed my custom encoding code to use the built-in code Catalyst now provides to decode input data and encode output.</div><div id="yui_3_16_0_ym19_1_1484083120986_5197"><br></div><div id="yui_3_16_0_ym19_1_1484083120986_5198">But, I came across two issues that I wanted to get some feedback on.</div><div id="yui_3_16_0_ym19_1_1484083120986_5199"><br></div><div id="yui_3_16_0_ym19_1_1484083120986_5200">First one is simple. This code will die if $value is already decoded. Any reason not to test it?</div><div id="yui_3_16_0_ym19_1_1484083120986_5201"><br></div><div id="yui_3_16_0_ym19_1_1484083120986_5203"><div id="yui_3_16_0_ym19_1_1484083120986_5202"><br></div><div id="yui_3_16_0_ym19_1_1484083120986_5224"><font face="monospace, monospace" id="yui_3_16_0_ym19_1_1484083120986_5223"> sub _handle_param_unicode_decoding {</font></div><div id="yui_3_16_0_ym19_1_1484083120986_5226"><font face="monospace, monospace" id="yui_3_16_0_ym19_1_1484083120986_5225"> my ( $self, $value, $check ) = @_;</font></div><div id="yui_3_16_0_ym19_1_1484083120986_5228"><font face="monospace, monospace" id="yui_3_16_0_ym19_1_1484083120986_5227"> return unless defined $value; # not in love with just ignoring undefs - jnap</font></div><div id="yui_3_16_0_ym19_1_1484083120986_5230"><font face="monospace, monospace" id="yui_3_16_0_ym19_1_1484083120986_5229"> return $value if blessed($value); #don't decode when the value is an object.</font></div></div><div id="yui_3_16_0_ym19_1_1484083120986_5232"><font face="monospace, monospace" id="yui_3_16_0_ym19_1_1484083120986_5231">+ return $value if Encode::is_utf8($value); # already decoded?</font></div><div><font face="monospace, monospace"><br></font></div><div id="yui_3_16_0_ym19_1_1484083120986_7029">This next one is what I have questions about. This is also in Catalyst.pm in the encoding() method:</div><br clear="all"><div id="yui_3_16_0_ym19_1_1484083120986_7011"><div id="yui_3_16_0_ym19_1_1484083120986_7028"><font face="monospace, monospace"> # Let it be set to undef</font></div><div><font face="monospace, monospace"> if (my $wanted = shift) {</font></div><div id="yui_3_16_0_ym19_1_1484083120986_7010"><font face="monospace, monospace" id="yui_3_16_0_ym19_1_1484083120986_7027"> $encoding = Encode::find_encoding($wanted)</font></div><div id="yui_3_16_0_ym19_1_1484083120986_7012"><font face="monospace, monospace" id="yui_3_16_0_ym19_1_1484083120986_7025"> or Carp::croak( qq/Unknown encoding '$wanted'/ );</font></div><div id="yui_3_16_0_ym19_1_1484083120986_7015"><font face="monospace, monospace" id="yui_3_16_0_ym19_1_1484083120986_7014"> <b id="yui_3_16_0_ym19_1_1484083120986_7013">binmode(STDERR, ':encoding(' . $encoding->name . ')');</b></font></div></div><div id="yui_3_16_0_ym19_1_1484083120986_7016"><b><br></b></div><div id="yui_3_16_0_ym19_1_1484083120986_7017"><br></div><div id="yui_3_16_0_ym19_1_1484083120986_7018">The problem with that is it seems to turn off autoflush on STDERR. In my case that is causing intermixing of lines in the log files.</div><div id="yui_3_16_0_ym19_1_1484083120986_7019"><br></div><div id="yui_3_16_0_ym19_1_1484083120986_7020">If I turn on STDERR->autoflush(1) then the logging works again and log lines are not mangled.</div><div id="yui_3_16_0_ym19_1_1484083120986_7021"><br></div><div id="yui_3_16_0_ym19_1_1484083120986_7024">But, I'm not sure we should apply a layer to STDERR like that. For example, I log to stderr with JSON and that is already encoded.</div><div id="yui_3_16_0_ym19_1_1484083120986_7022"><br></div><div id="yui_3_16_0_ym19_1_1484083120986_7023">Is there a compelling reason to binmode STDERR there?</div><div><br></div><div><br></div><div><br></div><div><br></div>-- <br><div class="yiv6948327696gmail_signature">Bill Moseley<br><a rel="nofollow" ymailto="mailto:moseley@hank.org" target="_blank" href="mailto:moseley@hank.org">moseley@hank.org</a></div>
</div></div><br>_______________________________________________<br>List: <a ymailto="mailto:Catalyst@lists.scsys.co.uk" href="mailto:Catalyst@lists.scsys.co.uk">Catalyst@lists.scsys.co.uk</a><br>Listinfo: <a href="http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst" target="_blank">http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst</a><br>Searchable archive: <a href="http://www.mail-archive.com/catalyst@lists.scsys.co.uk/" target="_blank" id="yui_3_16_0_ym19_1_1484083120986_7633">http://www.mail-archive.com/catalyst@lists.scsys.co.uk/</a><br>Dev site: <a href="http://dev.catalyst.perl.org/" target="_blank" id="yui_3_16_0_ym19_1_1484083120986_7634">http://dev.catalyst.perl.org/</a><br><br><br></div> </div> </div> </div></div></body></html>