6

I'm stuck in sentiment analysis and I found Vader solution which is the best I could find so far. My issue is that I don't find any doc on how to feed it with languages other than English.

Has QUIT--Anony-Mousse
  • 76,138
  • 12
  • 138
  • 194
Curcuma_
  • 851
  • 2
  • 12
  • 37
  • 1
    Cool hint: the reason I named Vader Multilingual, not Polyglot, is simply that Vader understands, but doesn't speak :) – Curcuma_ Aug 03 '17 at 13:23

2 Answers2

14

The short answer is "no".

The README file on the github page states

if you have access to the Internet, the demo has an example of how VADER can work with analyzing sentiment of texts in other languages (non-English text sentences).

but if you take a look at what is actually done for this demonstration (beginning at line 552 in the current version of vaderSentiment.py), this is based entirely on using a machine-translation web service to automatically translate the text into English. As such, the results are reliant not only upon the accuracy of the sentiment analysis tool but also upon the accuracy of whichever translation tool you use to create the English version of the input.

Vader only performs sentiment analysis on English texts, but that workaround (automatic translation) may be a viable option. Sentiment analysis is less sensitive to common machine translation problems than other usages*, but you'll certainly still have to keep the limitations in mind if you choose to use that workaround.

*To give an example, the service used in the demo translates "Das Internet funktioniert heute nicht. Ist eine Störung bekannt?" to "The Internet was not working today. Is a disorder known?", which would be more accurately translated as "The internet isn't working today. Is a disruption known?". It's got the tense wrong in the first sentence, and while there are several legitimate translations of "Störung" in this context, "disorder" is an awkward choice at best. Nevertheless, while this makes it quite a bad translation in general, the errors are unlikely to affect a sentiment analysis significantly.

Chris H
  • 790
  • 9
  • 19
  • a downside also about web translation is that the algorithm itself is not self contained, the server may go down, or API may change... Your answer was useful, Surely is eligible for bounty in 14 hours – Curcuma_ Aug 03 '17 at 19:33
  • 1
    Hi, how to see the list of positive, negative and neutral lexicons/words in VADER dictionary? And after getting the list of lexicons, suppose, if I were able to map a certain set of vocabulary (create a ENGLISH to NON-ENGLISH dictionary), how do I calculate the sentiment score for the NON-ENGLISH sentences using the VADER method? – OnePunchMan Jun 16 '19 at 19:15
  • @OnePunchMan information about the lexicon is all available in the README on the Vader github. I didn't write Vader so anything that isn't explained there, I can't help you with. As for getting a sentiment score with non-English texts... maybe you should read the answer you're commenting on. Vader only does English, if you don't want to produce your own lexicon you have to translate the sentences to English before running the analysis. – Chris H Jun 17 '19 at 08:29
2

I tried NLTK Vader in another language. It works fairly well with German - after all, the languages are not too far from each other.

There is some work involved - we can't just translate the lexicon:

  • Change the vader_lexicon.txt
  • Change the NEGATE words in the code
  • Change the BOOSTER words in the code
  • Change SPECIAL_CASE_IDIOMS in the code

In general, negations work, but there are cases which involve some additional work which I haven't figured out yet.

FrancoSwiss
  • 131
  • 3
  • I have downloaded Vader files from Github and modified as you mentioned above. However, how to use the modified folder one. Thank you in advance FrancoSwiss – Setthawut Kulsrisuwan Jul 15 '22 at 10:34