Python script to run FME workbench

Question

I have more than 500 xml files and each xml file should processed on FME workbench individually (iteration of FME workbench for each xml file). For such a propose i have to run a python file (loop.py) to iterate FME workbench for each xml file.

The whole process was working in past on other PC without any problem. Now Once i run Module i got the following error:

Traceback (most recent call last):E:\XML_Data File "E:\XML_Data\process\01_XML_Tile_1.py", line 28, in if "Translation was SUCCESSFUL" in open(path_log + "\" + data + ".log").read(): IOError: [Errno 2] No such file or directory: 'E:\XML_Data\data_out\log_01\re_3385-5275.xml.log'

Attached the python code(loop.py).

Any help is greatly appreciated.

import os
import time
# Mainpath and Working Folder:
#path_main = r"E:\XML_Data"
path_main = r"E:\XML_Data"
teil = str("01")

# variables
path_in = path_main + r"\data_in\03_Places\teil_" + teil                # "Source folder of  XML files"
path_in_tile10 = path_main + r"\data_in\01_Tiling\10x10.shp"            # "Source folder of Grid shapefile"
path_in_commu = path_main + r"\data_in\02_Communities\Communities.shp"  # "Source folder of Communities shapefile"
path_out = path_main + r"\data_out\teil_" + teil                        # "Output folder of shapefiles that resulted from XML files (tile_01 folder)"
path_log = path_main + r"\data_out\log_" + teil                         # "Output folder of log files for each run(log_01 folder)"
path_fme = r"%FME_EXE_2015%"                                            # "C:\Program Files\FME2015\fme.exe"
path_fme_workbench = path_main + r"\process\PY_FME2015.fmw"             # "path of FME workbench"
datalists = os.listdir(path_in)
count = 0

# loop each file individually in FME
for data in datalists:
    if data.find(".xml") != -1:
        count +=1
        print ("Run-No." + str(count) + ": with data " + data)
        os.system (path_fme + " " + path_fme_workbench + " " + "--SourceDataset_XML"+ " " + path_in + "\\" + data + " " + "--SourceDataset_SHAPE" + " " + path_in_tile10 + " " + "--SourceDataset_SHAPE_COMU" + " " + path_in_commu + " " + "--DestDataset_SHAPE" +" " +path_out + " " +"LOG_FILENAME" + " " + path_log + "\\" + data + ".log" )
        print ("Data processed: " + data)
        shape = str(data[19:28]) + "_POPINT_CENTR_UTM32N.shp"
        print ("ResultsFileName: " + shape)
        if "Translation was SUCCESSFUL" in open(path_log + "\\" + data + ".log").read():
            # Translation was successful and SHP file exists:
            if os.path.isfile(path_out + "\\" + shape):
                write_log = open(path_out + "\\" + "result_xml.log", "a")
                write_log.write(time.asctime(time.localtime()) + " " + shape + "\n")
                write_log.close()
                print("Everything ok")
            #Translation was successful, but SHP file does not exist:
            else:
                write_log = open(path_out + "\\" + "error_xml.log", "a")
                write_log.write(time.asctime(time.localtime()) + " Data: " + shape + " unavailable.\n")
                write_log.close()      
        # Translation was not successful:
        else:
            write_log = open(path_out + "\\" + "error_xml.log", "a")
            write_log.write(time.asctime(time.localtime()) + " Translation " + Data + " not successful.\n")
            write_log.close()
        
print ("Number of calculated files: " + str(count))

score 1 · Accepted Answer · answered Mar 08 '19 at 16:35

1

Most likely, the script failed at the os.system line, so the log file was not created from the command. Since you mentioned a different computer, it could be caused by many reasons, such as a different version of FME (so the environment variable %FME_EXE_2015% would not exist).

answered Mar 08 '19 at 16:35

prusswan

6,853
4
40
61

Thank you @prusswan. The proplem is Definition of environment variables for FME batch - from Windows Start Menu. Adding new variable: FME_EXE_2015 and the value: "C:\Program Files (x86)\FME2015\fme.exe" – Alfa Oct 21 '21 at 12:33

score 0 · Answer 2 · answered Oct 09 '20 at 00:56

0

Use a workspace runner transformer to do this.

answered Oct 09 '20 at 00:56

Alex Oulton

199
1
4
12

score 0 · Answer 3 · answered Sep 20 '21 at 14:59

0

The FME version is outdated.so first check the version whether it is creating the problem.

answered Sep 20 '21 at 14:59

Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-ask). – Community Sep 20 '21 at 20:10
This does not provide an answer to the question. Once you have sufficient [reputation](https://stackoverflow.com/help/whats-reputation) you will be able to [comment on any post](https://stackoverflow.com/help/privileges/comment); instead, [provide answers that don't require clarification from the asker](https://meta.stackexchange.com/questions/214173/why-do-i-need-50-reputation-to-comment-what-can-i-do-instead). - [From Review](/review/late-answers/29874338) – isopach Sep 20 '21 at 20:28

score -1 · Answer 4 · edited Feb 03 '23 at 13:21

-1

subprocess.call(["C:/Program Files/fme/FMEStarter/FMEStarter.exe", "C:/Program Files/fme/fme20238/fme.exe", "/fmefile.fmw" "LOG_FILENAME","logfile"], stdin=None, stdout=None, stderr=None, shell=True, timeout=None)

edited Feb 03 '23 at 13:21

General Grievance

4,555
31
31
45

answered Jan 30 '23 at 10:26

matth11

1

As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Feb 01 '23 at 09:36

Python script to run FME workbench

4 Answers4