I have been using Airflow on AWS(MWAA) for a couple of months now and I noticed that occasionally some Airflow tasks fail with no discernable reason and with no log messages in Cloudwatch. I often have to clear and retry the tasks multiple times for the task to eventually succeed.
Does anyone know the reason why this would be happening? I'm thinking with the autoscaling feature of MWAA this should not be an issue of resources. Anyone with any experience dealing with something like this or with any idea would be greatly appreciated. Thanks
