0

I'm new to sentiment analysis and I'm exploring with TextBlob.

My data is pre-processed Twitter data. It's in a series and each tweet has been cleaned and tokenized:

    0   [new, leaked, treasury, document, full, sugges...
    1   [tommy, robinson, endorsing, conservative, for...
    2   [thanks, already, watched, catch, tv, morning, ]
    3   [treasury, document, check, today, check, cons...
    4   [utterly, stunning, video, hoped, prayed, woul...
    ... ...
    307370  [trump, disciple, copycat]
    307373  [disgusting]
    307389  [wonder, people, vote, racist, homophobe, like...
    307391  [gary, neville, slam, fuelling, racism, manche...
    307393  [brexit, fault, excuseforeverything]

When I run textblob sentiment (using help from Apply textblob in for each row of a dataframe), my result is a column of nan values:

    # Create sentiment column using textblob
    # Source: https://stackoverflow.com/questions/43485469/apply-textblob-in-for-each-row-of-a-dataframe

    def sentiment_calc(text):
try:
    return TextBlob(text).sentiment
except:
    return None

    boris_data['sentiment'] = boris_data['text'].apply(sentiment_calc)

         text   sentiment
    0   [new, leaked, treasury, document, full, sugges...   None
    1   [tommy, robinson, endorsing, conservative, for...   None
    2   [thanks, already, watched, catch, tv, morning, ]    None
    3   [treasury, document, check, today, check, cons...   None
    4   [utterly, stunning, video, hoped, prayed, woul...   None
       ...  ... ...
    307370  [trump, disciple, copycat]  None
    307373  [disgusting]    None
    307389  [wonder, people, vote, racist, homophobe, like...   None
    307391  [gary, neville, slam, fuelling, racism, manche...   None
    307393  [brexit, fault, excuseforeverything]    None
Philip Liu
  • 65
  • 9
  • Your code is set to return `None` when the input is not in string format, as per the link you have attached. – Gary Dec 09 '19 at 19:01
  • When I delete the exception, the error is 'unexpected EOF while parsing' – Philip Liu Dec 09 '19 at 19:03
  • Why don't you convert the `text` values to string (maybe for a couple of rows), and then test the logic. See if the issue arises because `text` is not a string value. – Gary Dec 09 '19 at 19:05
  • Yep, text are not in strings. I realise this is ''.join(), but how do I apply this to all rows instead of just one? – Philip Liu Dec 09 '19 at 20:12
  • 1
    Assuming `boris_data['text']`contains the text column, you can do `boris_data['text']=boris_data['text'].astype(str)` to convert the column to string. – Gary Dec 09 '19 at 20:38

0 Answers0