<html><body><div style="color:#000; background-color:#fff; font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px"><div dir="ltr" id="yui_3_16_0_1_1419008847963_8542"><span id="yui_3_16_0_1_1419008847963_8541">Any chance you can test this on the current dev release on CPAN? There's a ton of utf8 fixes there. </span></div><div dir="ltr" id="yui_3_16_0_1_1419008847963_8542"><span><br></span></div><div dir="ltr" id="yui_3_16_0_1_1419008847963_8542"><a href="https://metacpan.org/release/JJNAPIORK/Catalyst-Runtime-5.90079_003" id="yui_3_16_0_1_1419008847963_8546">Catalyst-Runtime-5.90079_003 - The Catalyst Framework Runtime - metacpan.org</a><span><br></span></div><div style="width:450px; font-family: 'Georgia', 'Times', 'Times New Roman', 'serif';margin-top:5px; margin-bottom: 5px; background-color: #ffffff;" contenteditable="false" id="enhancrCard_0" class="link-enhancr-attachment link-enhancr-element"><table class="link-enhancr-element" cellspacing="0" cellpadding="0" border="0" style="width:450px; height:auto; position: relative; display: block;" id="yui_3_16_0_1_1419008847963_8905"><tbody id="yui_3_16_0_1_1419008847963_8904"><tr class="link-enhancr-element"><td class="link-enhancr-element" colspan="7" style="height: 1px; background-color: #e5e5e5; font-size: 1px; border-collapse: collapse;"><div class="link-enhancr-element" style="height: 1px; background-color: #e5e5e5; font-size: 1px; line-height:0px;"> </div></td></tr><tr class="link-enhancr-element"><td rowspan="5" class="link-enhancr-element" style="width: 1px; background-color: #e5e5e5; font-size: 1pt; border-collapse: collapse;"><div class="link-enhancr-element" style="width: 1px; background-color: #e5e5e5; font-size: 1pt;"> </div></td><td rowspan="5" class="link-enhancr-element" style="width: 14px; background-color: #ffffff; font-size: 0pt; border-collapse: collapse;"><div class="link-enhancr-element" style="width: 14px; background-color: #ffffff; font-size: 14pt;"> </div></td><td colspan="2" class="link-enhancr-element" style="height: 6px; background-color: #ffffff; font-size: 0pt; border-collapse: collapse;"><div class="link-enhancr-element" style="height: 6px; background-color: #ffffff; font-size: 6pt;"> </div></td><td rowspan="5" class="link-enhancr-element" style="width: 20px; background-color: #ffffff; font-size: 0pt; border-collapse: collapse;"><div class="link-enhancr-element" style="width: 20px; background-color: #ffffff; font-size: 20pt;"> </div></td><td class="link-enhancr-element" rowspan="5" width="1" style="width: 1px; background-color: #e5e5e5; font-size: 1pt; border-collapse: collapse;"><div class="link-enhancr-element" style="width: 1px; background-color: #e5e5e5; font-size: 1pt;"> </div></td></tr><tr id="yui_3_16_0_1_1419008847963_8903"><td class="link-enhancr-element" colspan="2" style="width: 100%; vertical-align: middle; font-family: 'Georgia', 'Times', 'Times New Roman', 'serif';" id="yui_3_16_0_1_1419008847963_8902"><div class="link-enhancr-text-part link-enhancr-element" style="line-height:16.5px; background-color: #ffffff; width: 414px;" id="yui_3_16_0_1_1419008847963_8901"><div class="link-enhancr-element" style="word-wrap: break-word; word-break: break-all;" id="yui_3_16_0_1_1419008847963_8900"><span class="link-enhancr-element icon icon-shrink link-enhancr-toggle"></span><span class="link-enhancr-element icon icon-close link-enhancr-delete"></span><a href="https://metacpan.org/release/JJNAPIORK/Catalyst-Runtime-5.90079_003" class="link-enhancr-card-urlWrapper link-enhancr-element" style="text-decoration: none !important; color: #000000 !important; line-height: 100%; font-size: 18px; display: block;" id="yui_3_16_0_1_1419008847963_8899"><span class="link-enhancr-element link-enhancr-card-title" style="margin: 0; font-weight: normal;margin-bottom: 3px; font-size: 18px; line-height: 21px; max-height: 43px; color: #000000; overflow: hidden !important; display: inline-block;" id="yui_3_16_0_1_1419008847963_8898">Catalyst-Runtime-5.90079_003 - The Catalyst Framework Runtime - metacpan.org</span></a><div style="font-size: 13px; line-height: 20px; color: #999999; max-height: 81px; font-family: 'Georgia', 'Times', 'Times New Roman', 'serif';overflow: hidden;" class="link-enhancr-card-description link-enhancr-element" id="yui_3_16_0_1_1419008847963_8906">The Catalyst Framework Runtime</div></div></div></td></tr><tr id="yui_3_16_0_1_1419008847963_8909"><td colspan="2" class="link-enhancr-element" style="height: 6px; background-color: #ffffff; font-size: 0pt; border-collapse: collapse;" id="yui_3_16_0_1_1419008847963_8908"><div class="link-enhancr-element" style="height: 6px; background-color: #ffffff; font-size: 6pt;" id="yui_3_16_0_1_1419008847963_8907"></div></td></tr><tr id="yui_3_16_0_1_1419008847963_8912"><td class="link-enhancr-element" style="vertical-align: middle; font-family: 'Arial', 'Helvetica Neue', 'Helvetica', 'sans-serif';" id="yui_3_16_0_1_1419008847963_8911"><div class="link-enhancr-element" style="font-size: 0pt;" id="yui_3_16_0_1_1419008847963_8910"><a href="https://metacpan.org/release/JJNAPIORK/Catalyst-Runtime-5.90079_003" class="link-enhancr-card-url link-enhancr-element" style="color: black; text-decoration: none !important;cursor:pointer !important;" target="_blank"><span class="link-enhancr-element link-enhancr-view-on" style="display: inline-block; line-height: 11px; max-width: 314px; min-width: 254px; overflow: hidden; max-height: 13px; word-break: break-all;" id="yui_3_16_0_1_1419008847963_8561"><span class="link-enhancr-element link-enhancr-mobile-no-resize" style="vertical-align:middle; font-size: 9px; line-height: 11px; color: #999999; -moz-text-size-adjust: none; -ms-text-size-adjust: none; -webkit-text-size-adjust:none; text-size-adjust:none;">View on <span style="font-weight: bold" class="link-enhancr-view-on-domain">metacpan.org</span></span></span></a></div></td><td class="link-enhancr-element" style="vertical-align: middle; width: 100px; font-family: 'Arial', 'Helvetica Neue', 'Helvetica', 'sans-serif';"><div class="link-enhancr-element link-enhancr-preview-wrapper" style="max-width: 100px; min-width: 80px; overflow: hidden; text-align: right; line-height: 11px; max-height: 13px; font-size: 0pt;"><span class="link-enhancr-element link-enhancr-preview-by link-enhancr-mobile-no-resize" style="vertical-align:middle; font-size: 9px; line-height: 11px; color: #999999; -moz-text-size-adjust: none; -ms-text-size-adjust: none; -webkit-text-size-adjust:none; text-size-adjust:none;">Preview by Yahoo</span></div></td></tr><tr><td colspan="2" class="link-enhancr-element" style="height: 9px; background-color: #ffffff; font-size: 0pt; border-collapse: collapse;"><div class="link-enhancr-element" style="height: 9px; background-color: #ffffff; font-size: 9pt;"></div></td></tr><tr class="link-enhancr-element"><td class="link-enhancr-element" colspan="7" style="height: 1px; background-color: #e5e5e5; font-size: 1px; border-collapse: collapse;"><div class="link-enhancr-element" style="height: 1px; background-color: #e5e5e5; font-size: 1px; line-height:0px"> </div></td></tr></tbody></table></div><div dir="ltr" id="yui_3_16_0_1_1419008847963_8542"><br></div><div dir="ltr" id="yui_3_16_0_1_1419008847963_8542">If trouble remains, I'd love an issue or ideally a test case. There's a big UTF8 test case over here</div><div dir="ltr" id="yui_3_16_0_1_1419008847963_8542"><br></div><div dir="ltr" id="yui_3_16_0_1_1419008847963_8542"><br></div><div dir="ltr" id="yui_3_16_0_1_1419008847963_8542"><a href="https://github.com/perl-catalyst/catalyst-runtime/blob/holland/t/utf_incoming.t" id="yui_3_16_0_1_1419008847963_8880">perl-catalyst/catalyst-runtime</a><br></div><div style="width:450px; font-family: 'Georgia', 'Times', 'Times New Roman', 'serif';margin-top:5px; margin-bottom: 5px; background-color: #ffffff;" contenteditable="false" id="enhancrCard_1" class="link-enhancr-attachment link-enhancr-element"><table class="link-enhancr-element" cellspacing="0" cellpadding="0" border="0" style="width:450px; height:170px; position: relative; display: block;" id="yui_3_16_0_1_1419008847963_8919"><tbody id="yui_3_16_0_1_1419008847963_8918"><tr class="link-enhancr-element"><td class="link-enhancr-element" colspan="8" style="height: 1px; background-color: #e5e5e5; font-size: 1px; border-collapse: collapse;"><div class="link-enhancr-element" style="height: 1px; background-color: #e5e5e5; font-size: 1px; line-height:0px;"> </div></td></tr><tr class="link-enhancr-element" id="yui_3_16_0_1_1419008847963_8924"><td rowspan="5" class="link-enhancr-element" style="width: 1px; background-color: #e5e5e5; font-size: 1pt; border-collapse: collapse;"><div class="link-enhancr-element" style="width: 1px; background-color: #e5e5e5; font-size: 1pt;"> </div></td><td class="link-enhancr-element" rowspan="5" style="vertical-align: middle; width: 168px; height: 168px; background-color: #000000;border-collapse: collapse;" id="yui_3_16_0_1_1419008847963_8923"><div class="link-enhancr-element" style="width: 168px;" align="center" id="yui_3_16_0_1_1419008847963_8922"><a href="https://github.com/perl-catalyst/catalyst-runtime/blob/holland/t/utf_incoming.t" class="link-enhancr-card-urlWrapper link-enhancr-element" style="text-decoration: none !important; color: #000000 !important;" id="yui_3_16_0_1_1419008847963_8921"><img alt="image" src="https://avatars0.githubusercontent.com/u/2287815?v=3&s=400" class="link-enhancr-thumbnail-image link-enhancr-element" width="168" height="168" style="display: block; margin: auto;" id="yui_3_16_0_1_1419008847963_8920"></a></div></td><td rowspan="5" class="link-enhancr-element" style="width: 1px; background-color: #e5e5e5; font-size: 0pt; border-collapse: collapse;"><div class="link-enhancr-element" style="width: 1px; background-color: #e5e5e5; font-size: 1pt;"> </div></td><td rowspan="5" class="link-enhancr-element" style="width: 14px; background-color: #ffffff; font-size: 0pt; border-collapse: collapse;"><div class="link-enhancr-element" style="width: 14px; background-color: #ffffff; font-size: 14pt;"> </div></td><td colspan="2" class="link-enhancr-element" style="height: 6px; background-color: #ffffff; font-size: 0pt; border-collapse: collapse;"><div class="link-enhancr-element" style="height: 6px; background-color: #ffffff; font-size: 6pt;"> </div></td><td rowspan="5" class="link-enhancr-element" style="width: 20px; background-color: #ffffff; font-size: 0pt; border-collapse: collapse;"><div class="link-enhancr-element" style="width: 20px; background-color: #ffffff; font-size: 20pt;"> </div></td><td class="link-enhancr-element" rowspan="5" width="1" style="width: 1px; background-color: #e5e5e5; font-size: 1pt; border-collapse: collapse;"><div class="link-enhancr-element" style="width: 1px; background-color: #e5e5e5; font-size: 1pt;"> </div></td></tr><tr id="yui_3_16_0_1_1419008847963_8917"><td class="link-enhancr-element" colspan="2" style="width: 100%; vertical-align: middle; font-family: 'Georgia', 'Times', 'Times New Roman', 'serif';" id="yui_3_16_0_1_1419008847963_8916"><div class="link-enhancr-text-part link-enhancr-element" style="line-height:16.5px; background-color: #ffffff; height: 135px; width: 245px;" id="yui_3_16_0_1_1419008847963_8915"><div class="link-enhancr-element" style="word-wrap: break-word; word-break: break-all;" id="yui_3_16_0_1_1419008847963_8914"><span class="link-enhancr-element icon icon-shrink link-enhancr-toggle"></span><span class="link-enhancr-element icon icon-close link-enhancr-delete"></span><a href="https://github.com/perl-catalyst/catalyst-runtime/blob/holland/t/utf_incoming.t" class="link-enhancr-card-urlWrapper link-enhancr-element" style="text-decoration: none !important; color: #000000 !important; line-height: 100%; font-size: 18px; display: block;"><span class="link-enhancr-element link-enhancr-card-title" style="margin: 0; font-weight: normal;margin-bottom: 3px; font-size: 18px; line-height: 21px; max-height: 43px; color: #000000; overflow: hidden !important; display: inline-block;">perl-catalyst/catalyst-runtime</span></a><div style="font-size: 13px; line-height: 20px; color: #999999; max-height: 81px; font-family: 'Georgia', 'Times', 'Times New Roman', 'serif';overflow: hidden;" class="link-enhancr-card-description link-enhancr-element" id="yui_3_16_0_1_1419008847963_8913">catalyst-runtime - The Elegant MVC Web Application Framework</div></div></div></td></tr><tr><td colspan="2" class="link-enhancr-element" style="height: 4px; background-color: #ffffff; font-size: 0pt; border-collapse: collapse;"><div class="link-enhancr-element" style="height: 4px; background-color: #ffffff; font-size: 4pt;"></div></td></tr><tr><td class="link-enhancr-element" style="vertical-align: middle; font-family: 'Arial', 'Helvetica Neue', 'Helvetica', 'sans-serif';"><div class="link-enhancr-element" style="font-size: 0pt;"><a href="https://github.com/perl-catalyst/catalyst-runtime/blob/holland/t/utf_incoming.t" class="link-enhancr-card-url link-enhancr-element" style="color: black; text-decoration: none !important;cursor:pointer !important;" target="_blank"><span class="link-enhancr-element link-enhancr-view-on" style="display: inline-block; line-height: 11px; max-width: 145px; min-width: 85px; overflow: hidden; max-height: 13px; word-break: break-all;" id="yui_3_16_0_1_1419008847963_8896"><span class="link-enhancr-element link-enhancr-mobile-no-resize" style="vertical-align:middle; font-size: 9px; line-height: 11px; color: #999999; -moz-text-size-adjust: none; -ms-text-size-adjust: none; -webkit-text-size-adjust:none; text-size-adjust:none;">View on <span style="font-weight: bold" class="link-enhancr-view-on-domain">github.com</span></span></span></a></div></td><td class="link-enhancr-element" style="vertical-align: middle; width: 100px; font-family: 'Arial', 'Helvetica Neue', 'Helvetica', 'sans-serif';"><div class="link-enhancr-element link-enhancr-preview-wrapper" style="max-width: 100px; min-width: 80px; overflow: hidden; text-align: right; line-height: 11px; max-height: 13px; font-size: 0pt;"><span class="link-enhancr-element link-enhancr-preview-by link-enhancr-mobile-no-resize" style="vertical-align:middle; font-size: 9px; line-height: 11px; color: #999999; -moz-text-size-adjust: none; -ms-text-size-adjust: none; -webkit-text-size-adjust:none; text-size-adjust:none;">Preview by Yahoo</span></div></td></tr><tr><td colspan="2" class="link-enhancr-element" style="height: 9px; background-color: #ffffff; font-size: 0pt; border-collapse: collapse;"><div class="link-enhancr-element" style="height: 9px; background-color: #ffffff; font-size: 9pt;"></div></td></tr><tr class="link-enhancr-element"><td class="link-enhancr-element" colspan="8" style="height: 1px; background-color: #e5e5e5; font-size: 1px; border-collapse: collapse;"><div class="link-enhancr-element" style="height: 1px; background-color: #e5e5e5; font-size: 1px; line-height:0px"> </div></td></tr></tbody></table></div><div id="yui_3_16_0_1_1419008847963_8542"> </div><div id="yui_3_16_0_1_1419008847963_8542" dir="ltr">Take a look and let me know if we need more here. The file upload stuff is something that is a bit confusing to me that I got it all correct</div> <div class="qtdSeparateBR"><br><br></div><div class="yahoo_quoted" style="display: block;"> <div style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16px;"> <div style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16px;"> <div dir="ltr"> <font size="2" face="Arial"> On Wednesday, December 17, 2014 7:22 PM, Bill Moseley <moseley@hank.org> wrote:<br> </font> </div> <br><br> <div class="y_msg_container"><div id="yiv6339117796"><div dir="ltr"><div>All my upload forms have accept-charset="utf-8". We expect that uploaded filenames could have wide-characters.</div><div><br></div>The problem I hit was ->basename does this:<div><br></div><div><div>$ perl -le 'use Catalyst::Request::Upload; my $upload = Catalyst::Request::Upload->new( { filename => q[документ обучения.pdf] } ); print $upload->basename;'</div><div>_.pdf</div><div><br></div><div>That's pretty mangled.</div><div><br></div><div><br></div><div>The problem is that $upload->filename is not decoded so the substitution is working on octets not characters. </div><div><div><br></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px;"><div><div><font face="monospace">sub _build_basename {</font></div></div><div><div><font face="monospace"> my $self = shift;</font></div></div><div><div><font face="monospace"> my $basename = $self->filename;</font></div></div><div><div><font face="monospace"> $basename =~ s|\\|/|g;</font></div></div><div><div><font face="monospace"> $basename = ( File::Spec::Unix->splitpath($basename) )[2];</font></div></div><div><div><font face="monospace"> $basename =~ s|[^\w\.-]+|_|g;</font></div></div><div><div><font face="monospace"> return $basename;</font></div></div><div><div><font face="monospace">}</font></div></div></blockquote><div><div><br></div><div>Obviously, we want \w to work on characters, not encoded octets. Decoding the filename should be done -- it's character data.</div><div><br></div><div>Does it make sense to do it in Engine's prepare_uploads?</div><div><br></div><div>For example:</div><div><br></div><div><div><font face="monospace"> my $u = Catalyst::Request::Upload->new(</font></div><div><font face="monospace"> size => $upload->{size},</font></div><div><font face="monospace"> type => scalar $headers->content_type,</font></div><div><font face="monospace"> headers => $headers,</font></div><div><font face="monospace"> tempname => $upload->{tempname},</font></div><div><font face="monospace"> filename => <b>$c->_handle_unicode_decoding($upload->{filename})</b>,</font></div><div><font face="monospace"> );</font></div></div><div><br></div><div><br></div>-- <br><div class="yiv6339117796gmail_signature">Bill Moseley<br><a rel="nofollow" ymailto="mailto:moseley@hank.org" target="_blank" href="mailto:moseley@hank.org">moseley@hank.org</a></div>
</div></div></div></div></div><br>_______________________________________________<br>List: <a ymailto="mailto:Catalyst@lists.scsys.co.uk" href="mailto:Catalyst@lists.scsys.co.uk">Catalyst@lists.scsys.co.uk</a><br>Listinfo: <a href="http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst" target="_blank">http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst</a><br>Searchable archive: <a href="http://www.mail-archive.com/catalyst@lists.scsys.co.uk/" target="_blank">http://www.mail-archive.com/catalyst@lists.scsys.co.uk/</a><br>Dev site: <a href="http://dev.catalyst.perl.org/" target="_blank">http://dev.catalyst.perl.org/</a><br><br><br></div> </div> </div> </div> </div></body></html>