<div class="gmail_quote">On Tue, Jun 23, 2009 at 1:28 AM, Aristotle Pagaltzis <span dir="ltr"><<a href="mailto:pagaltzis@gmx.de" target="_blank">pagaltzis@gmx.de</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
* <a href="mailto:seasprocket@gmail.com" target="_blank">seasprocket@gmail.com</a> <<a href="mailto:seasprocket@gmail.com" target="_blank">seasprocket@gmail.com</a>> [2009-06-23 03:00]:<br>
<div>> Thanks for your suggestion, but I'm pretty sure that the data<br>
> is not getting encoded twice. C::V::JSON tests the data before<br>
> it encodes ( Encode::is_utf8() ) and only encodes if this test<br>
> is true. This test only passes if the data is decoded.<br>
<br>
</div>Augh! Augh! Why do people keep reading stuff into the UTF8 flag<br>
that it doesn’t mean. (Yeah, I know why, because it’s called the<br>
UTF8 flag when it should’ve been the UOK flag or something.) You<br>
can have decoded data with the UTF8 flag off, and you can have<br>
encoded data with the UTF8 flag on. </blockquote><div><br>(Sorry to be so slow to reply. I wanted to find time to fully investigate this, but haven't.) <br></div><div><br>The Encode docs state:<br><br># When you encode, the resulting UTF8 flag is always off.<br>
# When you decode, the resulting UTF8 flag is on unless you can unambiguously represent data [as ASCII].<br>
<br>I was interpreting this to apply to all encoding/decoding -- but I now realize that it may only apply to the Encode package. Which really just leaves me more confused .. :)<br> </div>> My suspicion is that I don't really understand what's happening<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div>
> inside sqlite -- I assume it's storing as UTF-8, but I don't<br>
> really know what it's doing.<br>
<br>
</div>Try Devel::Peek to examine the strings that come out of it?</blockquote><div><br>I used Devel::StringInfo and found:<br><br>[info] string: Madrid Alarcón<br>is_utf8: 0<br>octet_length: 15<br>valid_utf8: 1<br>decoded_is_same: 0<br>
decoded:<br> octet_length: 15<br> downgradable: 1<br> char_length: 14<br> string: Madrid Alarc<br> is_utf8: 1<br>raw = <<Madrid Alarcón>><br><br>I did not draw any brilliant conclusions from this, although I'm curious why the decoded version has the non-ASCII char cut off.<br>
<br>At this point, obviously, I need to find the time to dig in further. Thanks for your thoughts!<br><br> </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
<div><div></div><div><br>
Regards,<br>
--<br>
Aristotle Pagaltzis // <<a href="http://plasmasturm.org/" target="_blank">http://plasmasturm.org/</a>><br>
<br>
_______________________________________________<br>
List: <a href="mailto:Catalyst@lists.scsys.co.uk" target="_blank">Catalyst@lists.scsys.co.uk</a><br>
Listinfo: <a href="http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst" target="_blank">http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst</a><br>
Searchable archive: <a href="http://www.mail-archive.com/catalyst@lists.scsys.co.uk/" target="_blank">http://www.mail-archive.com/catalyst@lists.scsys.co.uk/</a><br>
Dev site: <a href="http://dev.catalyst.perl.org/" target="_blank">http://dev.catalyst.perl.org/</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>==========================<br><a href="http://www.bikewise.org" target="_blank">http://www.bikewise.org</a><br><br>2People citizen's network for climate action: <a href="http://www.2people.org" target="_blank">http://www.2people.org</a><br>
<br>Greater Seattle Climate Dialogues: <a href="http://www.climatedialogues.org" target="_blank">http://www.climatedialogues.org</a><br>==========================<br>