0

I'm trying to write a script that'll get tweets from the python twitter api for a given hashtag. Running into some unexpected troubles, though, and from what I'm able to tell it's an encoding problem which I'm not yet equipped to figure out on my own. This is the script, with the bits with my API keys skipped for privacy, held in variable 'auth' referenced in the first line.

twitter_api = twitter.Twitter(auth=auth)

q = '#hillaryclinton'
count = 1

search_results  = twitter_api.search.tweets(q=q, count=count)

statuses = search_results['statuses']

next_results = search_results['search_metadata']['next_results']

kwargs = dict([ kv.split('=') for kv in next_results[1:].split("&") ])

statuses += search_results['statuses']
search_results = twitter_api.search.tweets(**kwargs)

# Show one sample search result by slicing the list...
t = statuses[0]
t1 = statuses[1]
print(t['text'])
print(t1['text'])

When I run this I'd expect the output to be the (tweet's) text from the first tweet in the search query, but instead I'm getting this error message:

UnicodeEncodeError: 'charmap' codec can't encode character u'\u2026 in position 139: character maps to <undefined>

Sometimes the character that it can't encode and the position will be different. How can I ensure that the character gets encoded properly?

e: I was told by @martjin that the problem is my console, but when I try to write the output to a textfile via f = open('tweetfile.txt', 'w') f.write(t['text'])

I get an UnicodeEncodeError: 'ascii' codec can't encode character u'\u2026' in position 139: ordinal not in range(128) error.

Any ideas?

n1c9
  • 2,662
  • 3
  • 32
  • 52
  • 2
    This is your *console*; it cannot handle all of Unicode. – Martijn Pieters Oct 08 '15 at 19:59
  • @MartijnPieters , when I use an IDE that supports unicode I'm still getting the error message 'UnicodeEncodeError: 'ascii' codec can't encode character u'\u2026' in position 139: ordinal not in range(128)' when trying to write the output to a text file. is that also a console issue? if so, how do I resolve it? – n1c9 Oct 08 '15 at 20:22
  • 2
    Don't rely on the default codec when opening files; use a Unicode-suitable code like UTF-8, explicitly: `open(filename, 'w', encoding='utf8')`. – Martijn Pieters Oct 08 '15 at 20:54
  • thanks so much, @MartijnPieters ! – n1c9 Oct 08 '15 at 20:56

0 Answers0