I have a long running script which gets a (long) array of folders (with subarray of files in that folder) where I have to do several actions on each file.
What is the best way to make sure I make all actions successful? And how to handle unsuccessful actions?
Lets say what will happen if my mysql server is unavailable or like the Amazon S3 API is not working correctly.
pseudocode of my script:
- starting script with folders / files array
- looping through each folder
- looping through each file in that folder
- open file (from external server) and try converting it to custom object (only continue if file is a valid "object")
- extract some parts of file and save them to Amazon S3 bucket
- extract some other parts of file and save them to another Amazon S3 bucket
- extract metadata / text of file and insert into elasticsearch
- update mysql record