Fetching language detection from Google api

Question

I have a CSV with keywords in one column and the number of impressions in a second column.

I'd like to provide the keywords in a url (while looping) and for the Google language api to return what type of language was the keyword in.

I have it working manually. If I enter (with the correct api key): http://ajax.googleapis.com/ajax/services/language/detect?v=1.0&key=myapikey&q=merde I get: {"responseData": {"language":"fr","isReliable":false,"confidence":6.213709E-4}, "responseDetails": null, "responseStatus": 200} which is correct, 'merde' is French.

so far I have this code but I keep getting server unreachable errors:

import time
import csv
from operator import itemgetter
import sys
import fileinput
import urllib2
import json

E_OPERATION_ERROR = 1
E_INVALID_PARAMS = 2

#not working
def parse_result(result):
  """Parse a JSONP result string and return a list of terms"""

  # Deserialize JSON to Python objects
  result_object = json.loads(result)

  #Get the rows in the table, then get the second column's value
  # for each row
  return row in result_object

#not working
def retrieve_terms(seedterm):

  print(seedterm) 
  """Retrieves and parses data and returns a list of terms"""
  url_template = 'http://ajax.googleapis.com/ajax/services/language/detect?v=1.0&key=myapikey&q=%(seed)s'
  url = url_template % {"seed": seedterm}


  try:
    with urllib2.urlopen(url) as data:
      data = perform_request(seedterm)
      result = data.read()
  except:
    sys.stderr.write('%s\n' % 'Could not request data from server')
    exit(E_OPERATION_ERROR)

  #terms = parse_result(result)
  #print terms
  print result

def main(argv):
  filename = argv[1]

  csvfile = open(filename, 'r')
  csvreader = csv.DictReader(csvfile)

  rows = []
  for row in csvreader:
    rows.append(row)

  sortedrows = sorted(rows, key=itemgetter('impressions'), reverse = True)

  keys = sortedrows[0].keys()

  for item in sortedrows:
    retrieve_terms(item['keywords'])

  try:
    outputfile = open('Output_%s.csv' % (filename),'w')
  except IOError:
    print("The file is active in another program - close it first!")
    sys.exit()  
  dict_writer = csv.DictWriter(outputfile, keys, lineterminator='\n')
  dict_writer.writer.writerow(keys)
  dict_writer.writerows(sortedrows)
  outputfile.close()

  print("File is Done!! Check your folder") 

if __name__ == '__main__':
  start_time = time.clock()
  main(sys.argv)
  print("\n")
  print time.clock() - start_time, "seconds for script time"

Any idea how to finish the code so that it will work? Thank you!

score 1 · Accepted Answer · edited May 23 '17 at 10:26

Try to add referrer, userip as described in the docs:

An area to pay special attention to relates to correctly identifying yourself in your requests. Applications MUST always include a valid and accurate http referer header in their requests. In addition, we ask, but do not require, that each request contains a valid API Key. By providing a key, your application provides us with a secondary identification mechanism that is useful should we need to contact you in order to correct any problems. Read more about the usefulness of having an API key

Developers are also encouraged to make use of the userip parameter (see below) to supply the IP address of the end-user on whose behalf you are making the API request. Doing so will help distinguish this legitimate server-side traffic from traffic which doesn't come from an end-user.

Here's an example based on the answer to the question "access to google with python":

#!/usr/bin/python
# -*- coding: utf-8 -*-
import json
import urllib, urllib2
from pprint import pprint

api_key, userip = None, None
query = {'q' : 'матрёшка'}
referrer = "https://stackoverflow.com/q/4309599/4279"

if userip:
    query.update(userip=userip)
if api_key:
    query.update(key=api_key)

url = 'http://ajax.googleapis.com/ajax/services/language/detect?v=1.0&%s' %(
    urllib.urlencode(query))

request = urllib2.Request(url, headers=dict(Referer=referrer))
json_data = json.load(urllib2.urlopen(request))

pprint(json_data['responseData'])

Output

{u'confidence': 0.070496580000000003, u'isReliable': False, u'language': u'ru'}

Another issue might be that seedterm is not properly quoted:

if isinstance(seedterm, unicode):
   value = seedterm
else: # bytes
   value = seedterm.decode(put_encoding_here)
url = 'http://...q=%s' % urllib.quote_plus(value.encode('utf-8'))

Fetching language detection from Google api

1 Answers1

Output