aiofiles is a python library that is used to handle the local disk files present in asyncio applications. Use this tag for questions related to the python library only. Do not use the tag for APL I/O files.
Questions tagged [python-aiofiles]
36 questions
28
votes
3 answers
ASGI 'lifespan' protocol appears unsupported
I have an asynchronous code running on fastapi & aiofiles
i'm trying to load and save my info from a .json file but every time I shut down the program, it save only the keys of the dict and showing me "ASGI 'lifespan' protocol appears unsupported"…
yuval hayun
- 281
- 1
- 3
- 4
14
votes
1 answer
Writing files asynchronously
I've been trying to create a server-process that receives an input file path and an output path from client processes asynchronously. The server does some database-reliant transformations, but for the sake of simplicity let's assume it merely puts…
Eli Korvigo
- 10,265
- 6
- 47
- 73
5
votes
1 answer
async version runs slower than the non async version
My program does the following:
Takes folder of .txt files
For each file:
2.1. read the file
2.2 sort the contents as a list and pushes the list to a master list
I did this without any async/await and these are the time statistics
real …
rrgirish
- 331
- 2
- 3
- 15
3
votes
1 answer
AIOFiles Take Longer Than Normal File Operation
I have a question I'm new to the python async world and I write some code to test the power of asyncio, I create 10 files with random content, named file1.txt, file2.txt, ..., file10.txt
here is my code:
import asyncio
import aiofiles
import…
SinaMobasheri
- 729
- 8
- 17
3
votes
1 answer
Async file copy - why is the file descriptor bad?
Would like to implement Python code to read and write (i.e. copy) a file. The goal is to read and write concurrently, so it reduces the time to execute the file copy. This is a learning exercise for me to get familiar with the async/await…
Kosa
- 57
- 5
3
votes
1 answer
Why is reading and calling an API from a file slower using Python async than synchronously?
I have a large file, with a JSON record on each line. I'm writing a script to upload a subset of these records to CouchDB via the API, and experimenting with different approaches to see what works the fastest. Here's what I've found to work fastest…
James Kelleher
- 1,957
- 3
- 18
- 34
3
votes
1 answer
websocket data collection design
I have a more open-ended design oriented question for you here. I have background in Python but not in web nor async programming. I am writing an app to save data collect from websockets 24/24, 7/7 with the aim to minmise data loss.
My initial…
jam123
- 129
- 1
- 7
1
vote
1 answer
Reading pdf in fully asynchronous mode in python
I'm really struggling to read my pdf files asynchronously. I tried using aiofiles which is open-source on GitHub. I want to extract the text from pdfs. I want to do it with pdfminer because pypdf is not rendering math (greek letters) or double…
Quentin
- 45
- 1
- 7
1
vote
0 answers
How to implement produce- consumer model with aiohttp and aiofiles
Problem:
I'd like to use aiohttp producers to download large amount of files and aiofiles consumers pull files from the pool.
Here's my draft structure, any thoughts please?
async def producer() #downloader
async with session.get(url) as r:
…
Round
- 69
- 7
1
vote
1 answer
Python Error in Reading Files using Aiofiles
Simple Problem statement. I want to read files asynchronously . My problem is when I try to read a file using aiofiles.open it just errors out with the cryptic messsage
AttributeError: __enter_
The crux of the problem can be demonstrated with the…
Saugat Mukherjee
- 778
- 8
- 32
1
vote
1 answer
aiofiles - Delete files in folder asynchronously based on certain criteria
I have folder with around 50 000 HTML files.
I'm trying to write script which opens file and if title contains certain string than file should be deleted.
This is my attempt so far:
import aiofiles
import glob
from natsort import natsorted
import…
Hrvoje
- 13,566
- 7
- 90
- 104
1
vote
1 answer
awaiting for aiofiles.os.remove() returns TypeError: object NoneType can't be used in 'await' expression
I'm trying to remove a file from a local directory asynchronously; however, I get the following error:
object NoneType can't be used in 'await' expression ()
I'm using ver aiofiles 0.5.0 and Python 3.6.5
my code is as straightforward as such:
async…
user1528465
- 53
- 1
- 8
1
vote
1 answer
How do I async pickle a lot of files with aiofiles?
I have a list I'd like to write out, data, one file for each item like so:
for i,chunk in enumerate(data):
fname = ROOT / f'{i}.in'
with open(fname, "wb") as fout:
dill.dump(chunk, fout)
Since the data list can be quite long and I'm…
Mittenchops
- 18,633
- 33
- 128
- 246
1
vote
1 answer
How to limit the number of concurrent read / write with aiofiles?
My program would concurrently download about 10 million pieces of data with aiohttp and then write the data to about 4000 files on disk.
I use the aiofiles library because I want my program to also do other stuff when it is reading/writing a…
Duh Huh
- 139
- 1
- 1
- 7
1
vote
1 answer
Seeking explanation of the advantages of async with/for
I'm new to asyncio. I came across AIOFiles (https://github.com/Tinche/aiofiles) recently and saw in the documentation that it supports 'async with' and 'async for.' I wanted to learn about it but there isn't much good coverage on it besides PEP 492…
D. Anderson
- 55
- 6