API call While loop not doing all lines of csv

Question

While loop doesn't send all line parameters before stopping thus finishes after second pass and sometimes directly after first pass.

i have tried reducing the number of steps but plays little difference

import requests
import rek #request script
import time
import pandas as pd
dfbig= pd.read_csv ('C:\\Users\\libraries2.csv')



step = 19
init = 0; final = init + step
while (final<= dfbig.shape[0]):
    df = dfbig.iloc [ init : final ]
    rek.req (df , init, final) 
    init = final
    final = init + step
    time.sleep (60)

It just stops before completing all lines of csv

I'd say a data issue, can you send a sample of csv file. – Richard Jun 24 '19 at 15:53 — Richard, Jun 24 '19 at 15:53
https://pastebin.com/d30q7jDP I have pasted the data here – George Jun 24 '19 at 16:15 — George, Jun 24 '19 at 16:15

score 0 · Answer 1 · answered Jun 28 '19 at 10:58

Your code is incrementing final at the end of the loop. On the 2nd iteration final exceeded dfbig.shape[0] (ie: 19+19+19 <= 43 is false) thereby ending the loop. This has the effect of skipping over the last 5 rows.

To resolve:

step = 19
init = 0
final = init + step
while (init <= dfbig.shape[0]):
    df = dfbig.iloc[init:final]
    rek.req (df , init, final) 
    init = final
    final = init + step

Building on this chunking idea you could do this to simplify:

rows = dfbig.shape[0]
for i in range(0, rows, step):
    df = dfbig.iloc[i:i + step]
    rek.req(df, i, i + step)

API call While loop not doing all lines of csv

1 Answers1