aiofiles - Delete files in folder asynchronously based on certain criteria

Question

I have folder with around 50 000 HTML files. I'm trying to write script which opens file and if title contains certain string than file should be deleted.

This is my attempt so far:

import aiofiles
import glob
from natsort import natsorted
import asyncio
from bs4 import BeautifulSoup
import os

    async def main():
        i=0
        htmls = glob.glob("CarsPages" + "//*.html")
        for html in natsorted(htmls):
            async with aiofiles.open(html, mode='r', encoding='UTF-8',  errors='strict', buffering=1) as f:
                contents = await f.read()
                soup = BeautifulSoup(contents, features="lxml")
                if "Best portal" in soup.title.get_text():
                    i+=1
                    os.close(html) 
                    os.remove(html)
                    print("removing: ", html)
        print("Removed: ", i, " pages")
    
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

But I'm getting :

os.close(html) TypeError: an integer is required (got type str)

Don't know which functions to use for close and remove once it's opened with aiofiles?

EDIT - WORKING CODE BASED ON @joao answer

import aiofiles
import glob
from natsort import natsorted
import asyncio
from bs4 import BeautifulSoup
import os

async def main():
    i=0
    htmls = glob.glob("CarsPages" + "//*.html")
    for html in natsorted(htmls):
        async with aiofiles.open(html, mode='r', encoding='UTF-8',  errors='strict', buffering=1) as f:
            contents = await f.read()
            soup = BeautifulSoup(contents, features="lxml")
        if "Best portal" in soup.title.get_text():
            i+=1
            os.remove(html)
            print("removed: ", html)
    print("Removed: ", i, " pages")

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

`os.close()` accepts the file descriptor, try with `os.close(f)` — Krishna Chaurasia, Apr 05 '21 at 15:04
@KrishnaChaurasia tried that too but got: `os.close(f) TypeError: an integer is required (got type AsyncTextIOWrapper)` — Hrvoje, Apr 05 '21 at 23:14

score 1 · Accepted Answer · answered Apr 05 '21 at 15:53

I'm assuming you're using python >= 3.5, you're using aiofiles.open as a context manager, so you shouldn't worry about closing the file yourself. What you need to do is simply exit the context manager block, when your condition has determined that the file should be removed, and then remove the file after the context manager block (and yes, os.remove is the right function for the job, just make sure you don't need an absolute path).

Unfortunately, you can't use break with a context manager, but this question shows various methods for achieving the same result.

Works great, I've edited my question to show final working code. Thank you! — Hrvoje, Apr 05 '21 at 23:24

aiofiles - Delete files in folder asynchronously based on certain criteria

1 Answers1