Python code failed in Azure ML Experiments but runs in conda environment

Question

I have created custom conda environment in my Azure ML compute instance and has verified that the python code runs in the environment. However, when I submit the .py file in Azure ML Experiment, the run fails even with the same conda environment set for the experiment.

This is how I submit the experiment:

ws = Workspace.from_config()
compute_name = os.environ.get("AML_COMPUTE_CLUSTER_NAME", "mycompute_cluster")
compute_target = ws.compute_targets[compute_name]
env = Environment.from_existing_conda_environment('expEnv', "myEnv")
experiment = Experiment(workspace=ws, name='exp')
config = ScriptRunConfig(source_directory='./',
                             script='exp1.py',
                             compute_target=compute_target)

config.run_config.environment = env
run = experiment.submit(config)
aml_url = run.get_portal_url()
print(aml_url)

I also tried with creating Azure ML Environments from the conda YAML file and using it when submitting the experiment, but I still get the same error.

Error:

UserError: module 'tensorflow.python.training.experimental.mixed_precision' has no attribute '_register_wrapper_optimizer_cls'

Run the following command **pip3 install tf-nightly** – Harshitha Veeramalla Dec 22 '21 at 08:17 — Harshitha Veeramalla, Dec 22 '21 at 08:17

score 1 · Accepted Answer · answered Jan 05 '22 at 05:32

It seems the required module is missing or renamed. You need to make sure you are using the same module version in your code which has required attribute.

Additionally, you might need to install module tf-nightly using pip3 install tf-nightly command as suggested by @HarshithaVeeramalla-MT in comment section.

Python code failed in Azure ML Experiments but runs in conda environment

1 Answers1