0

I would like to use an Azure Machine Learning Compute Cluster as a compute target but do not want it to containerize my project. Is there a way to deactivate this "feature" ?

The main reasons behind this request is that :

  1. I already set up a docker-compose file that is used to specify 3 containers for Apache Airflow and want to avoid a Docker-in-Docker situation. Especially that I already tried to do so but failed so far (here's the link my other related SO question).
  2. I prefer not to use a Compute Instance as it is tied to an Azure account which is not ideal for automation purposes.

Thanks in advance !

Downforu
  • 317
  • 5
  • 13

1 Answers1

0

Use the provisioning_configuration method of the AmlCompute class to specify configuration parameters.

In the following example, a persistent compute target provisioned by AmlCompute is created. The provisioning_configuration parameter in this example is of type AmlComputeProvisioningConfiguration, which is a child class of ComputeTargetProvisioningConfiguration.

from azureml.core.compute import ComputeTarget, AmlCompute
   from azureml.core.compute_target import ComputeTargetException

   # Choose a name for your CPU cluster
   cpu_cluster_name = "cpu-cluster"

   # Verify that cluster does not exist already
   try:
       cpu_cluster = ComputeTarget(workspace=ws, name=cpu_cluster_name)
       print('Found existing cluster, use it.')
   except ComputeTargetException:
       compute_config = AmlCompute.provisioning_configuration(vm_size='STANDARD_D2_V2',
                                                              max_nodes=4)
       cpu_cluster = ComputeTarget.create(ws, cpu_cluster_name, compute_config)

   cpu_cluster.wait_for_completion(show_output=True)

Refer - https://learn.microsoft.com/en-us/python/api/azureml-core/azureml.core.compute.amlcompute(class)?view=azure-ml-py

Abhishek K
  • 3,047
  • 1
  • 6
  • 19
  • I've just looked at the `provisioning_configuration` method but nothing here indicates the possibility to tell Azure to not building a docker container by default. Do you refer to a particular parameter in your answer ? – Downforu May 30 '22 at 18:30