[Catalyst] CSV / UTF-8 / Unicode

Craig Chant craig at homeloanpartnership.com
Tue Jul 2 11:45:51 GMT 2013

Hi Mike,

The data is coming from MS SQL 2008 R2, I have checked the DB and the column is defined as data type nvarchar(max) , so is set to Unicode and when viewing data via SQL Studio Manager, the Unicode characters display fine.

I am accessing the DB via DBI and have set on the connection string.. {'RaiseError' => 1, 'mysql_enable_utf8' => 1}

Though I'm not sure if the utf8 setting has any affect when using an MS ODBC driver as the flag implies it's for mysql ?

I tried to apply  my $chars = decode('UTF-8', $xls); and I got an error saying you cannot decode a string with wide characters.

I don't know what the string is, but when you try to decode you get a wide character error and when you try to encode you get a wide character error.

If I don't do anything to it and just output it, no error, but the data is full of junk chars.

Where do I go from here?


-----Original Message-----
From: Mike Whitaker [mailto:mike at altrion.org]
Sent: 02 July 2013 12:27
To: The elegant MVC web framework
Subject: Re: [Catalyst] CSV / UTF-8 / Unicode

>       # encode UTF8 octet
>        use Encode qw(decode encode);
>        my $octets = encode('UTF-8', $xls);

OUt of curiousity, where's the data in $xls come from, and what do you know about its encoding and bytes vs characters status?
List: Catalyst at lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/
This Email and any attachments contain confidential information and is intended solely for the individual to whom it is addressed. If this Email has been misdirected, please notify the author as soon as possible. If you are not the intended recipient you must not disclose, distribute, copy, print or rely on any of the information contained, and all copies must be deleted immediately. Whilst we take reasonable steps to try to identify any software viruses, any attachments to this e-mail may nevertheless contain viruses, which our anti-virus software has failed to identify. You should therefore carry out your own anti-virus checks before opening any documents. HomeLoan Partnership will not accept any liability for damage caused by computer viruses emanating from any attachment or other document supplied with this e-mail. HomeLoan Partnership reserves the right to monitor and archive all e-mail communications through its network. No representative or employee of HomeLoan Partnership has the authority to enter into any contract on behalf of HomeLoan Partnership by email. HomeLoan Partnership is a trading name of H L Partnership Limited, registered in England and Wales with Registration Number 5011722. Registered office: 26-34 Old Street, London, EC1V 9QQ. H L Partnership Limited is authorised and regulated by the Financial Conduct Authority.

More information about the Catalyst mailing list