Write json responce for each request into a file

Question

I wrote a code which is making a request to API and recieving output in JSON. So my question is how to write output for each request in file. Now my code is doing the last one request.

import requests
import json

with open("query4.txt", "rt") as file:
        data_file = file.read()

for line in data_file.split("\n"):
        drX, drY, fromX, fromY, dist = line.split(",")

        url = "https://api.openrouteservice.org/directions?"
        params = [
                ["api_key", "my_api_key"],
                ["coordinates", "%s,%s|%s,%s" % (drY, drX, fromY, fromX)],
                ["profile", "driving-car"]
        ]
        headers = {
                "Accept": "application/json, application/geo+json,"
                          "application/gpx+xml, img/png; charset=utf-8"}
        responce = requests.get(url=url, params=params, headers=headers)
        # print(responce.url)
        # print(responce.text)
        result = json.loads(responce.text)
        # print(result)
with open("result.txt", "w+") as f_o:
        for rows in result["routes"]:
                f_o.writelines(json.dumps(rows["summary"]["distance"]))  # depending on how do you want the result
                print(result["routes"])

I have an output like this:

{'routes': [{'warnings': [{'code': 1, 'message': 'There may be restrictions on some roads'}], 'summary': {'distance': 899.6, 'duration': 102.1}, 'geometry_format': 'encodedpolyline', 'geometry': 'u~uiHir{iFb@SXzADTlAk@JOJ]@_@CWW{AKo@k@eDEYKo@y@{EGc@G]GYCOa@gCc@iCoBsLNGlAm@VK^Sh@Un@tD', 'segments': [{'distance': 899.6, 'duration': 102.1, 'steps': [{'distance': 22.1, 'duration': 5.3, 'type': 11, 'instruction': 'Head south', 'name': '', 'way_points': [0, 1]}, {'distance': 45.4, 'duration': 10.9, 'type': 1, 'instruction': 'Turn right', 'name': '', 'way_points': [1, 3]}, {'distance': 645.5, 'duration': 52.3, 'type': 0, 'instruction': 'Turn left onto Партизанська вулиця', 'name': 'Партизанська вулиця', 'way_points': [3, 21]}, {'distance': 114.4, 'duration': 20.6, 'type': 1, 'instruction': 'Turn right', 'name': '', 'way_points': [21, 26]}, {'distance': 72.1, 'duration': 13, 'type': 1, 'instruction': 'Turn right', 'name': '', 'way_points': [26, 27]}, {'distance': 0, 'duration': 0, 'type': 10, 'instruction': 'Arrive at your destination, on the left', 'name': '', 'way_points': [27, 27]}]}], 'way_points': [0, 27], 'extras': {'roadaccessrestrictions': {'values': [[0, 1, 0], [1, 3, 2], [3, 27, 0]], 'summary': [{'value': 0, 'distance': 854.2, 'amount': 94.95}, {'value': 2, 'distance': 45.4, 'amount': 5.05}]}}, 'bbox': [38.484536, 48.941171, 38.492904, 48.943022]}], 'bbox': [38.484536, 48.941171, 38.492904, 48.943022], 'info': {'attribution': 'openrouteservice.org | OpenStreetMap contributors', 'engine': {'version': '5.0.1', 'build_date': '2019-05-29T14:22:56Z'}, 'service': 'routing', 'timestamp': 1568280549854, 'query': {'profile': 'driving-car', 'preference': 'fastest', 'coordinates': [[38.485115, 48.942059], [38.492073, 48.941676]], 'language': 'en', 'units': 'm', 'geometry': True, 'geometry_format': 'encodedpolyline', 'instructions_format': 'text', 'instructions': True, 'elevation': False}}}
{'routes': [{'summary': {'distance': 2298, 'duration': 365.6}, 'geometry_format': 'encodedpolyline', 'geometry': 'u~a{Gee`zDLIvBvDpClCtA|AXHXCp@m@bBsBvBmC`AmAtIoKNVLXHPb@c@`A_AFENGzAc@XKZCJ?PDLBH@F?T?PC~CcATOt@Sd@QLKBCBAb@]ZG|@OY_DQ}IE{DC_DAg@Eg@q@aFgBuH^GjBFj@

I did NeverHopeless answer, but i've got the same:

        result = json.loads(responce.text)
        i = 0
with open(f"result-{i}.txt", "w+") as f_o:
        i += 1
        for rows in result["routes"]:
                f_o.writelines(json.dumps(rows["summary"]["distance"]))  # depending on how do you want the result
                print(result["routes"])

Output now looks like this 899.622982138.832633191.8 I'm expecting to get this:

Every value is a distance from different requests so i need to have each on the new line.

Possible duplicate of [How do I write JSON data to a file?](https://stackoverflow.com/questions/12309269/how-do-i-write-json-data-to-a-file) — F Blanchet, Sep 12 '19 at 09:39

Booboo · Accepted Answer · 2019-09-12T10:09:34.913

2

You need to open and keep open the output file before your loop:

import requests
import json

with open("query4.txt", "rt") as file:
        data_file = file.read()

with open("result.txt", "w") as f_o:
    for line in data_file.split("\n"):
            drX, drY, fromX, fromY, dist = line.split(",")

            url = "https://api.openrouteservice.org/directions?"
            params = [
                    ["api_key", "my_api_key"],
                    ["coordinates", "%s,%s|%s,%s" % (drY, drX, fromY, fromX)],
                    ["profile", "driving-car"]
            ]
            headers = {
                    "Accept": "application/json, application/geo+json,"
                              "application/gpx+xml, img/png; charset=utf-8"}
            responce = requests.get(url=url, params=params, headers=headers)
            # print(responce.url)
            # print(responce.text)
            result = json.loads(responce.text)
            # print(result)
            for rows in result["routes"]:
                    print(rows["summary"]["distance"], file=f_o)  # depending on how do you want the result
                    # print(result["routes"])

edited Sep 12 '19 at 10:09

answered Sep 12 '19 at 09:44

Booboo

38,656
3
37
60

Alright it's doing write, can you tell me please how to put each new value on new row? Because my result is doing not so well now xD ```899.622982138.832633191.8``` – Semyon Kolesnikov Sep 12 '19 at 09:47
1

Describe what comes back exactly with each request and what you would like the output to be for each request. – Booboo Sep 12 '19 at 09:52
For each request i'm gaining a distance from summary as you can see, your code is doing well and writing data from all requests, but i need to do every value from each request on a new line. – Semyon Kolesnikov Sep 12 '19 at 09:55
So then i can start thinking about how to make this script to do 10-20-30 request per minute to not to kill my API key Like it's doing 20 requests and then stops for a minute. Do the same again but with the next values from when it stopped until it reaches the end – Semyon Kolesnikov Sep 12 '19 at 09:56
1

Just edit your original question and show what **one** request looks like and what the expected output should be so that there are no misunderstandings. – Booboo Sep 12 '19 at 09:59

score 2 · Answer 2 · answered Sep 12 '19 at 09:50

2

I think it's better to write results in different files with timestamp. in this way you don't rewrite on your older file and also you can find them easier.

current_time = time.strftime("%m_%d_%y %H_%M_%S", time.localtime())

with open(current_time + ".txt", "w+") as f_o:
        for rows in result["routes"]:
                f_o.writelines(json.dumps(rows["summary"]["distance"]))  # depending on how do you want the result
                print(result["routes"])

answered Sep 12 '19 at 09:50

0xM4x

460
1
8
19

For sure i need to do a 1 file with distance from all requests and to transfer dist from an input file for each line, this is for statistics) – Semyon Kolesnikov Sep 12 '19 at 09:53

NeverHopeless · Answer 3 · 2019-09-12T09:47:10.583

1

You need to make this filename "result.txt" dynamic. Currently it is overwriting content.

Perhaps like this:

i = 0   # <----- Keep it outside your for loop or it will be always set to zero
with open(f"result-{i}.txt", "w+") as f_o:
    i += 1

Or instead of integers, you may better use timestamp in filename.

edited Sep 12 '19 at 09:47

answered Sep 12 '19 at 09:38

NeverHopeless

11,077
4
35
56

I did it, and i got the same. – Semyon Kolesnikov Sep 12 '19 at 09:40
@SemyonKolesnikov, Move` i = 0` outside for loop. – NeverHopeless Sep 12 '19 at 09:43

Write json responce for each request into a file

3 Answers3