0

I am downloading all the tweets sent to a user into a csv 

auth = tweepy.OAuthHandler(cons_tok, cons_sec)
auth.set_access_token(app_tok, app_sec)
twitter_api = tweepy.API(auth,retry_delay=5,retry_errors=set([401, 404, 500, 503]),
         wait_on_rate_limit=True )

#Open CSV
myfile = open('teststream.csv', 'w')
wr = csv.writer(myfile, quoting=csv.QUOTE_ALL)

#Search for tweets sent to a particular user
search_results = tweepy.Cursor(twitter_api.search, q="@iamsrk").items(5)
try:
    for results in search_results:
        print(results.text)
        item = (results.text).encode('utf-8').strip()
        wr.writerow([item])

The problem is that the csv contains some weird characters and I am not sure how to fix it.

For the tweet:

.@iamsrk’s breakout film #Baazigar also had this soothing romantic track

The data saved in CSV is this:

RT @RadioMirchi: .@iamsrk’s breakout film #Baazigar also had this soothing romantic track …

Here as you can see ' is getting replaced by ’s

I have seen some more weird replacements for other non-alphanumerical characters. How can I fix this?

Piyush
  • 606
  • 4
  • 16
  • 38
  • How are you examining the CSV file? Also, what OS are you using? Are you using Python2 or Python3? – Robᵩ Jul 20 '16 at 20:29
  • Using Python 3. Mac OS. Opening the CSV with Excel – Piyush Jul 20 '16 at 20:40
  • Related: http://stackoverflow.com/questions/6588068/which-encoding-opens-csv-files-correctly-with-excel-on-both-mac-and-windows – Robᵩ Jul 20 '16 at 20:59
  • @Robᵩ : Ahh seems like the issue with encoding in CSV and not related to some problem with the code. Opening the file with TextEdit seems to solve the problem. I anyways need the csv as a datastore for some text analysis..so I think it should be fine as it is. – Piyush Jul 21 '16 at 06:20

0 Answers0