Calling annotate_text from IBM Watson Concept Insights with Python

Question

I wrote a set of Python functions to interact with the Bluemix/Watson Concept Insights API. I am able to generate a token and use it to get a result from the server, but the result stinks: it's nowhere near as good as what I get when I plug the same information into their Swagger testing utility.

I suspect that something is wrong with the way I am sending my request, but I don't know quite what. The code follows. First, from event_insight_lib.py:

def importCredentials(filename='credentials.json'):
    if filename in [f for f in os.listdir('.') if os.path.isfile(f)]:
        data = json.load(open(filename))['concept_insights'][0]['credentials']
        return data

def generateToken(filename='credentials.json'):
    credentials = importCredentials(filename)
    r = requests.get("https://gateway.watsonplatform.net/authorization/api/v1/token\?url=https://stream.watsonplatform.net/concept-insights/api", auth=(credentials['username'], credentials['password']))
    if r.status_code == requests.codes.ok:
        return r.text

def annotateText(text, token, content_type = 'text/plain'):
    base_url='https://watson-api-explorer.mybluemix.net/concept-insights/api/v2/graphs/wikipedia/en-20120601/annotate_text'
    headers = {'X-Watson-Authorization-Token': token, 'Content-Type': content_type}
    r = requests.post(base_url, headers=headers, data={'body': text})
    return r.text

These methods are executed by event_insight.py:

token = event_insight_lib.generateToken()
ret = event_insight_lib.annotateText("""long string being concept-analyzed...""", token)
    print(ret)

A full demonstration of the difference in output is here. The full codebase is here. I'm not very experienced with the Requests library: is there a subtle mistake somewhere on the Pythonic end?

The relevant part of IBM's documentation is here.

You're passing a dict in the `data` parameter of your request, which is going to form encode the data. Try passing just `data=text`. http://docs.python-requests.org/en/latest/user/quickstart/#more-complicated-post-requests — engineerC, Oct 18 '15 at 04:27
You are right! Changing that segment to `data=text.encode(encoding='UTF-8', errors='ignore')` appears to have fixed the issue. — Aleksey Bilogur, Oct 18 '15 at 04:37
We have a Python SDK that we are working on.Concept Insights is not supported there but I will hopefully next week. https://github.com/watson-developer-cloud/python-sdk — German Attanasio, Oct 18 '15 at 14:11
That's great to know! A small question, since you're here: are tokens meant to be reused? I've been generating a new one for every request but the documentation says they live for an hour. — Aleksey Bilogur, Oct 18 '15 at 14:19
You don't need tokens, you can use Basic Auth with a username and password. Tokens are made to be use client-side or with the speech to text websockets — German Attanasio, Oct 18 '15 at 17:09

score 2 · Accepted Answer · answered Mar 15 '16 at 19:16

As @engineerc suggested you are sending a dict() as data. Quoting your comment data=text.encode(encoding='UTF-8', errors='ignore') is the solution for your problem.

On the other hand, please don't use https://watson-api-explorer.mybluemix.net, it's a proxy application we use to host the swagger documentation.
The service url is: https://gateway.watsonplatform.net/concept-insights/api

Also, we have a python-sdk that supports ConceptInsights and the annotate_text call.

It's a pip module so you will do:

pip install watson-developer-cloud

Calling annotate_text it's as simple as:

import json
from watson_developer_cloud import ConceptInsightsV2 as ConceptInsights


concept_insights = ConceptInsights(
    username='YOUR SERVICE USERNAME',
    password='YOUR SERVICE PASSWORD')

annotations = concept_insights.annotate_text('IBM Watson won the Jeopardy television show hosted by Alex Trebek')
print(json.dumps(annotations, indent=2))

Thanks---I will hopefully get to updating [watsongraph](https://pypi.python.org/pypi/watsongraph/) (the product of all of this probing) to using `watson-developer-cloud` sometime in the next month or so. — Aleksey Bilogur, Mar 16 '16 at 00:32

Calling annotate_text from IBM Watson Concept Insights with Python

1 Answers1