0

I have got a use case to hit a RESTful API which returns response in XML format. I need to access that response, access the data in it and store that data into a CSV file. All of above needs to be done using Python. I am a noob in Python, but after some googling I got to know that 'requests', 'xml.etree' etc are the packages which are useful in it but I am getting confused between all of them.

The problem is that XML response is very large, and when you hit the API from browser, it shows only first 10 records out of thousands. So I don't know how to get the full response in one go OR in increment fashion.

Below is the code which hits the API and prints the response. 

import requests
from xml.etree import ElementTree

response2 = requests.get("http://www.myapi.com/api/v2_2/eventEditions/1234/participants/companies", auth=('username', 'password'))
print response2

tree = ElementTree.fromstring(response2.content)
print tree 
print response2.content

Can someone suggest what should be my approach in this and the packages, methods to achieve it?

Sahil Doshi
  • 621
  • 1
  • 9
  • 15
  • Did you take a look at : http://stackoverflow.com/questions/24124643/parse-xml-from-url-into-python-object and : http://stackoverflow.com/questions/8331469/python-dictionary-to-csv ? :) – Maxime Flament May 16 '17 at 13:30
  • Thanks @MaximeFlament, that really helpis. But can you suggest how to get the full XML response as I am getting only 10 records – Sahil Doshi May 17 '17 at 05:39
  • Hmm I really don't know. I never faced this problem before, but one of my colleague may have. I'll ask him and come back to you – Maxime Flament May 17 '17 at 08:37
  • Okay no problem, thanks a lot man!! – Sahil Doshi May 17 '17 at 08:46
  • Okay, he doesn't know how to do it. Maybe you could provide a valid URL to the API, or at least a screenshot of what you receive. Moreover, are you sure the problem comes from the request ? Maybe it could be a problem of data structures you're using to store the data you receive : if your data structure isn't iterable, then you'll never be able to iterate over your set of data. Please take a look at this link to learn more about iterable structures : http://stackoverflow.com/questions/9884132/what-exactly-are-pythons-iterator-iterable-and-iteration-protocols – Maxime Flament May 17 '17 at 13:27
  • Moreover, if the API returns only 10 records when you access it through browser, then it means the API is set to return this number of records. And you'll never be able to overcome this by using Python or any kind of languages. – Maxime Flament May 17 '17 at 13:30
  • Yes @MaximeFlament, the problem was from API, so I overcame it by appending two parameters in the URL. 'startRecord' and 'rpp' where rpp stands for records per page. for example : www.myurl.com?startRecord=1&rpp=100 will return first 100 records – Sahil Doshi May 18 '17 at 06:02

0 Answers0