2

Here is my code below where I used long and lat coordinates in locations variable and attached it to the URL via coordinates_str. SInce I have CSV file which has latitude and longitude coordinates of around many locations and then call that CSV file as a input to this API(that needs authentication). How do I input CSV file into this code instead of locations variable?

import requests
import pprint

locations = [(13.84, -12.57), (12.21, -14.69)]

coordinates_str = ','.join(map(lambda a: ' '.join(f'{f:.3f}' for f in a), locations))

# Replace "poi-settings" with the endpoint you would like to call.

URL = f'https://ubiconnect-eu.ubimet.com:8090/pinpoint-data?coordinates={coordinates_str}'
TOKEN = 'TOKEN KEY'

# Create session object that can be used for all requests.
session = requests.Session()
session.headers['Authorization'] = 'Token {token}'.format(token=TOKEN)

# Send GET request to UBIconnect.
res = session.get(URL)
res.raise_for_status()

# Decode JSON response.
poi_info = res.json()
pprint.pprint(poi_info, indent=2, compact=True)

Then I tried this way: in place of coordinates_str I did this

import requests
import pprint
import pandas as pd 

df = pd.read_csv(r'E:\route_points.csv')
print(df)


# Replace "poi-settings" with the endpoint you would like to call.
URL = f'https://ubiconnect-eu.ubimet.com:8090/pinpoint-data?'
TOKEN = 'API TOKEN'
params= {'coordinates':(df)}

# Create session object that can be used for all requests.
session = requests.Session()
session.headers['Authorization'] = 'Token {token}'.format(token=TOKEN)

# Send GET request to UBIconnect.
res = session.get(URL, params= params)
res.raise_for_status()

# Decode JSON response.
poi_info = res.json()
pprint.pprint(poi_info, indent=2, compact=True)

Still not working.

Format needed to call the API from Documentation is:

# Replace "poi-settings" with the endpoint you would like to call.
URL = 'https://ubiconnect-eu.ubimet.com:8090/poi-settings'
TOKEN = '<YOUR TOKEN GOES HERE>'

so I replaced the poi-settings by pinpoint-data

URL = 'https://ubiconnect-eu.ubimet.com:8090/pinpoint-data?coordinates=longitude<space<latitude'

For Example: I put one coordinate set into API URL

URL = 'https://ubiconnect-eu.ubimet.com:8090/pinpoint-data?coordinates=132.85 12.84'

then with above URL I get the weather data for that location.

Shubham
  • 23
  • 5

2 Answers2

0

If you just want to submit a block of coordinates at a time from your CSV file then something like the following should suffice:

from itertools import islice
import requests
import pprint
import csv

def grouper(n, iterable):
    it = iter(iterable)
    return iter(lambda: tuple(islice(it, n)), ())


block_size = 10   # how many pairs to submit per request
TOKEN = 'TOKEN KEY'

# Create session object that can be used for all requests.
session = requests.Session()
session.headers['Authorization'] = 'Token {token}'.format(token=TOKEN)

with open('coordinates.csv', newline='') as f_input:
    csv_input = csv.reader(f_input)
    header = next(csv_input)        # skip the header
    
    for coords in grouper(block_size, csv_input):
        coordinates = ','.join(f'{float(long):.3f} {float(lat):.3f}' for long, lat in coords)
        print(coordinates)
        
        URL = f'https://ubiconnect-eu.ubimet.com:8090/pinpoint-data?coordinates={coordinates}'

        # Send GET request to UBIconnect.
        res = session.get(URL)
        res.raise_for_status()
        
        # Decode JSON response.
        poi_info = res.json()
        
        pprint.pprint(poi_info, indent=2, compact=True)

(obviously this was not tested - no token). Make sure there are no blank lines in your CSV file.


To output to a file add an output file:

with open('coordinates.csv', newline='') as f_input, open('output.json', 'w', encoding='utf-8')  as f_output:

and use this in the pprint() call:

pprint.pprint(poi_info, f_output, indent=2, compact=True)
f_output.write('\n')    # add blank line if needed
Martin Evans
  • 45,791
  • 17
  • 81
  • 97
  • Thank you, It is working!! But it would take so much time for 41,000 coordinates. Is there any alternative for inputting all the coordinates as a list or something like [(13.84, -12.57), (12.21, -14.69),......................] as you can see in the first snippet of my code? – Shubham Feb 07 '21 at 17:29
  • That's why the documentation was needed. There is probably a maximum amount per call? – Martin Evans Feb 07 '21 at 17:33
  • I checked the account limitation by 'global': { 'per_day': {'current': 21186, 'max': None}, 'per_hour': {'current': 745758, 'max': None}, 'per_month': {'current': 211523, 'max': None}}} So I think there is no monthly limitation.. – Shubham Feb 07 '21 at 17:53
  • I've updated it to show how to submit in groups of a given block size – Martin Evans Feb 07 '21 at 18:33
  • Thank you so much again for the input. Its improved but still taking time, but when I gave input of block as 41000, it crashed. and if I work with small block size,it would take time. – Shubham Feb 07 '21 at 18:50
  • I don't know what the maximum length of a URL is. Not surprised something would crash if you tried everything in one go. Try 1000 or 500 until it works – Martin Evans Feb 07 '21 at 18:56
  • Ok. I will work with different combinations and see how it goes. Thank you so much for this help!! – Shubham Feb 07 '21 at 19:05
  • It took 1.5 hours to get the data. The output console has the data. I dont want to run it again and copy, paste into Notepad doesnt work. Do you know the way to get the entire data into one file? – Shubham Feb 07 '21 at 20:39
  • Much better, and probably quicker would be to not print anything but write it directly to a file. No suggestion for salvaging it – Martin Evans Feb 07 '21 at 20:43
0

Hope this is what you are looking for

import csv
locations = list()
with open("foo.csv") as csvf:
    csvreader = csv.DictReader(csvf)    
    for row in csvreader:
        locations.append((float(row["lat"]), float(row["long"])))
# now add your code
coordinates_str = ','.join(map(lambda a: ' '.join(f'{f:.3f}' for f in a), locations))
Akash
  • 100
  • 1
  • 10
  • It is supposed to work but strangely its giving this error: csvreader = csv.DictReader(csvf) NameError: name 'csv' is not defined – Shubham Feb 07 '21 at 18:32
  • I have edited to include the import statement. – Akash Feb 07 '21 at 18:56
  • This is the error I am getting: File "e:\API_code_edit.py", line 9, in locations.append(row["longitude"], row["latitude"]) TypeError: list.append() takes exactly one argument (2 given) – Shubham Feb 07 '21 at 19:04
  • can you copy the code again and try? I noticed later that I had missed the conversion to tuple, i had added it with previous edit – Akash Feb 07 '21 at 19:06
  • This error: coordinates_str = ','.join(map(lambda a: ' '.join(f'{f:.3f}' for f in a), locations)) ValueError: Unknown format code 'f' for object of type 'str' – Shubham Feb 07 '21 at 19:18
  • My bad, I dont have a console to test the code. The values read from the csv would be of type string and we would need to cast it to float. Hope with this edit it works – Akash Feb 07 '21 at 19:31
  • It will work but it raises HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 414 Client Error: Request-URI Too Large for url, as my csv has 41000 coordinate set – Shubham Feb 07 '21 at 19:32
  • In that case you should know the limit of coordinates you can pass and break and convert the request to go in batches. – Akash Feb 08 '21 at 09:56
  • You can iterate through rows, append one after the other while making the request for every append and keep an active counter to keep an eye on number of coordinates you can append to the URL. – Akash Feb 08 '21 at 10:00