1

I am a hobby programmer and would like to solve a real-world problem I am having. I am a user of a software system that exposes an api that allows one to update/add records using Get. I have the imported the data I would like to modify the system with into Pandas and don't know the best way to iterate over the rows and dynamically create a URL.

I would like to create a URL for each row that is similar to:

http://fakesite.org/servlet/Erp?_form=AD1&_EVT=ADD&_RTN=DATA&FC=Add&_f1=A&_f3=HELLO&f16=EA&_OUT=XML&_EOT=TRUE

below I have included the beginning of my code setup:

import pandas as pd
import requests as r

site = 'http://fakesite.org/servlet/Erp?'

payload = dict()
payload['_FORM'] = 'AD1'
payload['_EVT'] = 'ADD'
payload['_RTN'] = 'DATA'
payload['FC'] = 'Add'
payload['_OUT'] = 'XML'
payload['_EOT'] = 'TRUE'

data = {'F1': ['A','A','A'], 'F3': ['HELLO', 'GOODBYE', 'HELLO_AGAIN'], 'F16': ['EA','BX','CA']}

df = pd.DataFrame(data)

The "payload" dictionary I created is all the parameters that don't change for the request, these are essentially hard coded and not a part of the pandas data frame. I will ultimately combine this dictionary with the values in each row of the data frame before passing it to request.get

I think I need to use either use either apply or itterrows but I can't seem to figure it out.

How should I write this code?

Benjamin Neil
  • 71
  • 1
  • 8

1 Answers1

1

Not sure why you want to use pandas, you can get a list of payloads to iterate through with

F1  = ['A','A','A']
F3  = ['HELLO', 'GOODBYE', 'HELLO_AGAIN']
F16 = ['EA','BX','CA']

payloads = [dict(payload,**{'F1': F1, 'F3': F3, 'F16': F16}) for F1,F3,F16 in zip(F1,F3,F16)]

EDIT: If you must use pandas then, you could use to_dict('records') as discussed in this stackoverflow question. Then, the documents explain how to formulate the actual requests.

payloads = [dict(payload, **params) for params in df.to_dict('records')]
gets = [requests.get(site, params=payload) for payload in payloads]
# to check that the urls were as you thought
for r in gets:
    print r.url
Community
  • 1
  • 1
michael_j_ward
  • 4,369
  • 1
  • 24
  • 25
  • Thank you for your answer. To continue this exercise, how would you pass the "payloads" to the Requests library so that it produces a request per dictionary? Also, could you do this with the Pandas function "to-dict('records')"? I am using Pandas to manipulate the data, then I am getting it into the dataframe that will be used for the request variable parameters. http://stackoverflow.com/questions/21190323/passing-records-parameter-to-pandas-to-dict-method – Benjamin Neil Jun 07 '16 at 13:53