From thg.bot at gmail.com Mon Feb 10 13:22:24 2020 From: thg.bot at gmail.com (Theo Bot) Date: Mon, 10 Feb 2020 14:22:24 +0100 Subject: [Catalyst] Encoding Message-ID: Hi, I created a controller that requests data from a mysql database via another module. The data in this case is a place in Germany called "M?nster". In the module that requests the data from mysql, with the mysql_enable_utf8 flag set to 1, it is still correct just before it is returned to the catalyst controller. But what it arrives in the controller someting has changed. The original character values are: 77 - 252 - 110 - 115 - 116 - 101 - 114 But when they arrive in the controller it has changed into 77 - 195 - 188 - 110 - 115 - 116 - 101 - 114 And becomes unreadable. How do I handle this? -- Kind regards Theo -------------- next part -------------- An HTML attachment was scrubbed... URL: From catalystgroup at unitedgames.co.uk Tue Feb 11 00:37:05 2020 From: catalystgroup at unitedgames.co.uk (Andrew) Date: Tue, 11 Feb 2020 00:37:05 -0000 Subject: [Catalyst] Encoding References: Message-ID: <001f01d5e073$62e60400$6c01a8c0@home> So what's happening in the controller, that might change it? Are you using commands or modules that work only with octets instead of UTF-8? my $octets = encode("UTF-8", $utf8_string); ...converts UTF-8 to Octets. my $utf8_string = decode("UTF-8", $octets); ...converts Octets to UTF-8. I put... use Encode qw(decode encode); ...at the start of my controller to use these functions. My UTF-8 boiler plate for my Catalyst Controllers is: #Always: use strict; use warnings; use Moose; use namespace::autoclean; #UTF-8: use utf8; use v5.30; use warnings ( 'FATAL', #makes anything in this list fatal 'utf8', #utf8 is a warnings category. ); #Specific: use Encode qw( decode encode ); ...and any other specific modules I need, I put in the specific bit. The reason I use... use v5.30; ...is because it enables for following features: unicode_strings (auto-enabled with "use v5.12;" or higher), unicode_eval (auto-enabled with "use v5.16;" or higher), fc (auto-enabled with "use v5.16;" or higher). I guess I could just use v5.16, haha. I use v5.30 because it's fun to be working with the latest Perl, =). There are two webpages I found useful for understanding Perl and Unicode: https://stackoverflow.com/questions/6162484/why-does-modern-perl-avoid-utf-8-by-default/6163129#6163129 https://www.perl.com/pub/2012/04/perlunicook-standard-preamble.html/ However, I do not know if it is a Perl and Unicode issue. It could be a MySQL issue, it could be a really simple mistake somewhere in the code, etc, it could be the other module you're using, etc. Until you share more context, it's difficult to know what the problem could be. Even then, I'm not an expert at character codes, ^_^. I do hope someone in this list can help you though, and everything works out, =). ----- Original Message ----- From: Theo Bot To: The elegant MVC web framework Sent: Monday, February 10, 2020 1:22 PM Subject: [Catalyst] Encoding Hi, I created a controller that requests data from a mysql database via another module. The data in this case is a place in Germany called "M?nster". In the module that requests the data from mysql, with the mysql_enable_utf8 flag set to 1, it is still correct just before it is returned to the catalyst controller. But what it arrives in the controller someting has changed. The original character values are: 77 - 252 - 110 - 115 - 116 - 101 - 114 But when they arrive in the controller it has changed into 77 - 195 - 188 - 110 - 115 - 116 - 101 - 114 And becomes unreadable. How do I handle this? -- Kind regards Theo ------------------------------------------------------------------------------ _______________________________________________ List: Catalyst at lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst at lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From catalystgroup at unitedgames.co.uk Tue Feb 11 01:54:57 2020 From: catalystgroup at unitedgames.co.uk (Andrew) Date: Tue, 11 Feb 2020 01:54:57 -0000 Subject: [Catalyst] Encoding References: <001f01d5e073$62e60400$6c01a8c0@home> Message-ID: <001801d5e07e$4378f7c0$6c01a8c0@home> There is also more info on Unicode Strings and Octets here: http://modernperlbooks.com/books/modern_perl_2016/03-perl-language.html#VW5pY29kZWFuZFN0cmluZ3M ----- Original Message ----- From: Andrew To: The elegant MVC web framework Sent: Tuesday, February 11, 2020 12:37 AM Subject: Re: [Catalyst] Encoding So what's happening in the controller, that might change it? Are you using commands or modules that work only with octets instead of UTF-8? my $octets = encode("UTF-8", $utf8_string); ...converts UTF-8 to Octets. my $utf8_string = decode("UTF-8", $octets); ...converts Octets to UTF-8. I put... use Encode qw(decode encode); ...at the start of my controller to use these functions. My UTF-8 boiler plate for my Catalyst Controllers is: #Always: use strict; use warnings; use Moose; use namespace::autoclean; #UTF-8: use utf8; use v5.30; use warnings ( 'FATAL', #makes anything in this list fatal 'utf8', #utf8 is a warnings category. ); #Specific: use Encode qw( decode encode ); ...and any other specific modules I need, I put in the specific bit. The reason I use... use v5.30; ...is because it enables for following features: unicode_strings (auto-enabled with "use v5.12;" or higher), unicode_eval (auto-enabled with "use v5.16;" or higher), fc (auto-enabled with "use v5.16;" or higher). I guess I could just use v5.16, haha. I use v5.30 because it's fun to be working with the latest Perl, =). There are two webpages I found useful for understanding Perl and Unicode: https://stackoverflow.com/questions/6162484/why-does-modern-perl-avoid-utf-8-by-default/6163129#6163129 https://www.perl.com/pub/2012/04/perlunicook-standard-preamble.html/ However, I do not know if it is a Perl and Unicode issue. It could be a MySQL issue, it could be a really simple mistake somewhere in the code, etc, it could be the other module you're using, etc. Until you share more context, it's difficult to know what the problem could be. Even then, I'm not an expert at character codes, ^_^. I do hope someone in this list can help you though, and everything works out, =). ----- Original Message ----- From: Theo Bot To: The elegant MVC web framework Sent: Monday, February 10, 2020 1:22 PM Subject: [Catalyst] Encoding Hi, I created a controller that requests data from a mysql database via another module. The data in this case is a place in Germany called "M?nster". In the module that requests the data from mysql, with the mysql_enable_utf8 flag set to 1, it is still correct just before it is returned to the catalyst controller. But what it arrives in the controller someting has changed. The original character values are: 77 - 252 - 110 - 115 - 116 - 101 - 114 But when they arrive in the controller it has changed into 77 - 195 - 188 - 110 - 115 - 116 - 101 - 114 And becomes unreadable. How do I handle this? -- Kind regards Theo ---------------------------------------------------------------------------- _______________________________________________ List: Catalyst at lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst at lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ ------------------------------------------------------------------------------ _______________________________________________ List: Catalyst at lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst at lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: