Lately I've been playing around a little with the python 3 async features. Overal I'm quit happy with the 3.6 syntax and of course the performance boost you gain. One of the exciting projects evolving around the ASGI standard in my opinion is starlette. I've got a sample app running where I'm reading data from a hdf5 file. h5py does not support asynchronous I/O yet. Which leaves me with the question: does what I'm doing here make any sense at all? To my understanding this code runs synchronously after all. What is the recommended way to do I/O in async contexts?
async def _flow(indexes):
print('received flow indexes %s ' %indexes)
# uses h5py under the hood
gr = GridH5ResultAdmin(gridadmin_f, results_f)
t = gr.nodes.timeseries(indexes=indexes)
data = t.only('s1').data
# data is a numpy array
return data['s1'].tolist()
@app.route('/flow_velocity')
async def flow_results(request):
indexes_list = [[2,3,4,5], [6,7,8,9], [10,11,12,13]]
tasks = []
loop = asyncio.get_event_loop()
t0 = datetime.datetime.now()
for indexes in indexes_list:
print('Start getting indexes %s' % indexes)
# Launch a coroutine for each data fetch
task = loop.create_task(_flow(indexes))
tasks.append(task)
# Wait on, and then gather, all data
flow_data = await asyncio.gather(*tasks)
dt = (datetime.datetime.now() - t0).total_seconds()
print('elapsed time: {} [s]'.format(dt))
return JSONResponse({'flow_velocity': flow_data})
Logging:
INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
Start getting indexes "[2, 3, 4, 5]"
Start getting indexes "[6, 7, 8, 9]"
Start getting indexes "[10, 11, 12, 13]"
received flow indexes [2, 3, 4, 5]
received flow indexes [6, 7, 8, 9]
received flow indexes [10, 11, 12, 13]
elapsed time: 1.49779 [s]
