1

I am using Threading to speed up collecting data from a website via RESTful API. I am storing the results in a list that I will walk later. I have made the list global, however when I try to print(list) outside of a thread, I see no results. Within, I can print(list) and see that it is appending correctly with all data. What's the best way to collect this data?

global global_list
global_list = []

pool = ActivePool()
s = threading.Sempahore(3)

    def get_data(s, pool, i, list):
        with s:
            name = threading.currentThread().getName()
            pool.makeActive(Name)
            data = []
            session = get_session(login info and url)#function for establishing connection to remote site
            request = {API Call 'i'}
            response = session.post(someurl, json=request)
            session.close()
            data = response.json()
            global_list.append(data)
            pool.makeInactive(name)

def main():
    for i in api_list:
        t = threading.Thread(target=get_data, name=i['uniqueID'], args=(s, pool, i, global_list))
    print(global_list)

if __name__ == '__main__':
    main()
martineau
  • 119,623
  • 25
  • 170
  • 301
Trogdor
  • 11
  • 2

1 Answers1

0

I was able to use Queue! Per the below Question, I stored the data I needed with put() within the thread and then storage it in a list I appended with get() outside of the threads.

Return Value From Thread

Community
  • 1
  • 1
Trogdor
  • 11
  • 2