Ruby - Microsoft Translator unexpected token error

Question

I'm using the following method to translate a simple word from English to Russian by calling:

translate("hello")

This is my method:

def translate(text)

    begin

        uri = "http://api.microsofttranslator.com/V2/Ajax.svc/GetTranslations?appId=#{@appid}&text=#{text.strip}&from=en&to=ru&maxTranslations=1"
        page = HTTParty.get(uri).body
        show_info = JSON.parse(page) # this line throws the error

    rescue

        puts $!

    end

end

The JSON output:

{"From":"en","Translations":[{"Count":0,"MatchDegree":100,"MatchedOriginalText":"","Rating":5,"TranslatedText":"Привет"}]}

The error:

unexpected token at '{"From":"en","Translations":[{"Count":0,"MatchDegree":100,"MatchedOriginalText":"","Rating":5,"TranslatedText":"Привет"}]}'

Not sure what it means by unexpected token. It's the only error I'm receiving. Unfortunately I can't modify the JSON output as it's returned by the API itself.

UPDATE:

Looks like the API is returning some illegal characters (bad Microsoft):

'´╗┐{"From":"en","Translations":[{"Count":0,"MatchDegree":0,"Matched OriginalText":"","Rating":5,"TranslatedText":"Hello"}]}'

Full error:

C:/Ruby193/lib/ruby/1.9.1/json/common.rb:148:in `parse': 743: unexpected token at '´╗┐{"From":"en","Translations":[{"Count":0,"MatchDegree":0,"Matched
OriginalText":"","Rating":5,"TranslatedText":"Hello"}]}' (JSON::ParserError)
        from C:/Ruby193/lib/ruby/1.9.1/json/common.rb:148:in `parse'
        from trans.rb:13:in `translate'
        from trans.rb:17:in `<main>'

This is not a Rails question, and it doesn't look like an api or translation question either. You didn't pass on the information about the line throwing the error. Is it the JSON line? Please reduce the example down to the minimal failing problem and re-tag your question. — Mark Thomas, Jan 09 '12 at 14:17
Try JSON.parse(output) in irb with and without the Cyrillic characters. — Mark Thomas, Jan 10 '12 at 12:19
@MarkThomas Even if I tell the API to return English converted text it still returns the same error. — gotnull, Jan 11 '12 at 04:00
Updated the question to reveal that the API is returning some illegal/invalid characters in the returned JSON. — gotnull, Jan 11 '12 at 04:45

score 1 · Accepted Answer · edited May 23 '17 at 12:03

Try ensuring UTF-8 encoding and stripping any leading BOM indicators in the string:

# encoding: UTF-8
# ^-- Make sure this is on the first line!

def translate(text)
  begin
    uri = "http://api.microsofttranslator.com/V2/Ajax.svc/GetTranslations?appId=#{@appid}&text=#{text.strip}&from=en&to=ru&maxTranslations=1"
    page = HTTParty.get(uri).body
    page.force_encoding("UTF-8").gsub!("\xEF\xBB\xBF", '')
    show_info = JSON.parse(page) # this line throws the error
  rescue
      puts $!
  end
end

Sources:

Ruby - Microsoft Translator unexpected token error

1 Answers1