-2

I am trying to write a python code to read a set of URLs from a CSV file and download the content in that URL. To read data from the CSV file, I am using pandas. And data is stored in data frames.Now I want to pass these values in the data frame(URLs) as an argument one by one to a function that uses the GET method to go to that particular URL and downloads the file. I am stuck in how to pass the values stored in a data frame in a loop as an argument. Any helps or any alternate methods are appreciated. Thanks in advance

Note: The data frame holds around 500 URLs.

Edit: I am using url = pd.read_csv(file_name, usecols=[26]) to read data. My question is how to pass values in url to a function in loop

karolch
  • 184
  • 4
  • 19
Vinyas Bhat
  • 44
  • 5

1 Answers1

0

Not sure I understand your question, but maybe this is an answer to it:

d = {'URL': ['URL1', 'URL2','URL3','URL4','URL5']}

df = pd.DataFrame(data=d)

for k in range(len(df)):
    url = df.at[k,'URL']
    out = do_something_with_url(url)
asere
  • 26
  • 5
  • url = pd.read_csv(file_name, usecols=[26]) I am using this command to read from the .csv file. Now I want to pass the values in **url** to a function as argument. I will try the code you have written. Thanks – Vinyas Bhat Feb 12 '20 at 10:43
  • in that case, your ```url``` variable is the same as my ```df```. In the line ```url = df.at[k,'URL']```, ```'URL'```should be the name of the column that holds your URLs – asere Feb 12 '20 at 11:02
  • Thanks. I am able to download files. Few more query. Is it possible to write the paths of these downloaded files back to CSV file? And now, files are getting downloaded to the path where I have kept .py file. Any idea how to change it? – Vinyas Bhat Feb 12 '20 at 11:37
  • Save the paths as a string of paths separated by commas, like so: ```list_of_paths = 'path1,path2,path3'```. Then create a csv file, open it and save line by line. See [this link](https://stackoverflow.com/questions/37289951/python-write-to-csv-line-by-line/37290105) – asere Feb 12 '20 at 12:54
  • As for your second question, depends on how you're downloading them. Probably in the function you're using there's an option to specify the folder in which you want to save the downloaded files – asere Feb 12 '20 at 12:54