for an university project I try to download tweets and analyze them. Unfortunately, every time I run the code for a high number of tweets, I get the following error:
File "C:\Users\Pasca\Anaconda3\lib\encodings\cp1252.py", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u039f' in position 54: character maps to
I tried various things, but I always get the same error message... It's one of my first times using Python, so I am not really an expert yet.
Thank you all very much in advance!
Following the code I'm using:
for tweet in tweepy.Cursor(api.search,q="#thepurge-filter:retweets",count=1000,
lang="en",
since="2017-04-03").items(1000):
result = re.sub(r"http\S+", "", tweet.text)
result = re.sub(r"@\S+", "", result)
result = result.lower()
ps = PorterStemmer()
word_tokens = word_tokenize(result)
result = [ps.stem(word) for word in word_tokens if not word in stop_words and len(word)>3]
result = ' '.join(result)
senttweet = TextBlob(tweet.text)
if senttweet.sentiment.polarity < 0:
sentiment = "negative"
nt.append(result)
elif senttweet.sentiment.polarity == 0:
sentiment = "neutral"
neut.append(result)
else:
sentiment = "positive"
pt.append(result)
if result not in alltweets:
csvWriter.writerow([tweet.id, tweet.created_at, tweet.source, tweet.favorite_count, tweet.retweet_count, result.encode('UTF-8',errors='ignore'), tweet.text.encode('UTF-8',errors='ignore'), sentiment])
alltweets.append(result)
csvFile.close()
