converting tweets to string in Python

Question

I'm using tweepy to collect random tweets, and I want to filter out non-alphanumeric tweets.

But in order to do the checking, I first need to convert the tweets to string. For example,

from tweepy import StreamListener
....

class sListener(StreamListener):
       def on_status(self,status):
            ....
            text = str(status.text)
            if not isAlphanumeric(text):
                ......

However, using str() to convert the tweet to string itself results in an error if the tweet is non-ascii with the following message:

UnicodeEncodeError: 'ascii' codec can't encode character

So I'm stuck in a loop where I need to convert to string to filter non-ascii, but I can't convert to string because of non-ascii....

I don't even know what data type tweets are...

Could anyone please help me out?

Did you try this? (http://stackoverflow.com/a/1207479/1984421) — Alberto Miranda, Jan 17 '13 at 15:44
If you do not want non-ascii tweets, why not just skip those that won't convert using `str()`? — woemler, Jan 17 '13 at 15:48

score 0 · Answer 1 · answered Jan 17 '13 at 15:42

0

It seems your tweets encoding is not ascii

Try

text = unicode(status.text)

instead of

text = str(status.text)

answered Jan 17 '13 at 15:42

Abhijit

62,056
18
131
204

score 0 · Answer 2 · answered Oct 18 '16 at 13:05

0

try

text = status.text.encode('utf8')

answered Oct 18 '16 at 13:05

Fabrizio Monaco

1

score 0 · Answer 3 · answered Oct 18 '16 at 20:13

0

I had a similar issue in the past. See if this works:

tweetText = status.text.encode("utf-8")
tweetText = unicode(tweetText, errors='ignore')

answered Oct 18 '16 at 20:13

user2789945

527
2
6
23

converting tweets to string in Python

3 Answers3