last day (14 days later) » 

13:51
0
Q: Python printing JSON data containing special characters?

SudoGaronError:(Python 2) File "test.py", line 49, in wow_rankings print rank_name, rank_class, rank_spec, rank_total, rank_duration, rank_fightID, rank_reportID, rank_guild, rank_server, rank_ilevel UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-6: ordinal not in range(128...

Maybe you could put the print statement in a try/except? That way you can handle special characters in the except block without doing a check on each value.
that's a good idea for processing, but still not sure how to get it to print afterwards.
Seems like the characters erroring out are Unicode. Not sure but you should be able to print by prefixing the string with 'u' - see stackoverflow.com/questions/10569438/… for more details.
What is the value of name you want??
It's in the source above, "name": "\u0422\u044d\u0439\u043a\u0430\u0445\u0445". I set these a variable in the code above for example rank_name. and then print them out to test that it's working. I just need a way to see what the value actually is when it is printed / returned instead of the raw unicode.
13:51
The name includes non ascii data, you need to decode
doing a print rank_name.decode('utf8') fails, along with decode('utf-8'). Wouldn't it already be unicode so I'd need to re-encode?
what does print u"\u0421\u0432\u0435\u0436\u0435\u0432\u0430\u0442\u0435\u043b\u044c \u0414\u0443\u0448" output?
print u"\u0421\u0432\u0435\u0436\u0435\u0432\u0430\u0442\u0435\u043b\u044c \u0414\u0443\u0448" UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-10: ordinal not in range(128)
what OS are you using?
Linux Ubuntu 14.x, this is on the cloud IDE at Koding.com
13:51
what does import locale;locale.getpreferredencoding() output?
>>> import locale;locale.getpreferredencoding() 'ANSI_X3.4-1968'
try import sys;reload(sys); sys.setdefaultencoding("utf-8") then try printing again
same error as before with the above print statement.
I just checked it out, the locale is set to POSIX. From bsh do export LC_ALL="en_US.UTF-8" then try again. After setting the locale it should now be utf-8 and it will work.
after the export, I now get a print output of (Kйкаs, Т, Yazuki) instead of (Kecks, Тэйкахх, Yazuki)
13:51
what does the json look like after making the changes?
You will also need to sudo echo export LC_ALL="en_US.UTF-8" >> .bashrc to make the change persist.
{u'rankings': [{u'name': u'Kecks'}, {u'name': u'\u0422\u044d\u0439\u043a\u0430\u0445\u0445}, {u'name': u'Yazuki'}] Total: 508 Kйкаs Т Yazuki
That is the json yes? where is Kйкаs Т Yazuki coming from?
Correct, I printed the json data out before pulling the data and printing specifics. that was the output right before (Kйкаs, Т, Yazuki).
what are you printing to get Kйкаs Т Yazuki ?
pastebin.com/6xmtCVEU , this is the code that spits out - Kйкаs Т Yazuki
13:51
@PadraicCunningham: that will not help with printing and is considered harmful. Please don't cargo-cult sys.setdefaultencoding().
@MartijnPieters. I was not recommending it as a fix, I wanted to see if it were possible to set utf-8 on the system. I had no idea what the cloud IDE at Koding.com was. The main problem was actually the locale setting
@PadraicCunningham: right; if you are wondering if a codec is available, then there are better ways to test that, like codecs.lookup()‌​‌​. :-)
@MartijnPieters, well I did not know about that method so I used what I knew, I don't think anyone died trying sys.setdefaultencoding() ;)
@PadraicCunningham: Nope, but the 'trick' is in constant use as cargo-culting "lets see if someone will upvote this if posted" answers, and are of no help what-so-ever. So it is slowly killing Stack Overflow quality when it comes to Python unicode issues.
@MartijnPieters, I would never suggest it as a solution, I just wanted to find out more information, I totally agree with the point it should never be used in actual code, sys having to be reloaded for good reason.
13:51
@PadraicCunningham: Glad we agree on that. :-D
@MartijnPieters, it was bound to happen sometime ;)
@PadraicCunningham: I'm not sure I can handle this much excitement and elation before an election. ;-P
@PadraicCunningham I moved from koding.com to a local OSX box and it works fine. So it has to be something specific to the linux system that runs on the site. I'll stick to a local unix system for now so I can get testing done.
@SudoGaron, yes, I could not see any obvious reason why you were getting strange output so it has to be something specific to how it works on that particular ide.

last day (14 days later) »