What service is used for triggering an Azure Pipeline that is assigned a Machine Learning task?

Question

I have a model trained on SVM with dataset as CSV uploaded as blob in blob storage. How can I update the CSV and how the changes can be used to trigger the pipeline that re-train the ML model.

Answer 1

If you mean trigger the build/release pipeline in Azure DevOps, then you need to set CI/CD for the build/release pipeline. Thus the pipeline will be triggered when a new commit/changeset is pushed to the repository.

In your scenario seems you stored the csv file in blob storage but not the normal repository. So, you cannot trigger the pipeline directly.

However as a workaround you can try to create a new build pipeline (e.g Pipeline A) and run commands/scripts in a command line task to update the CSV file, then use this build pipeline (e.g Pipeline A) to trigger another pipeline (e.g Pipeline B). Thus Pipeline B will be triggered when you updated the CSV file successfully in Pipeline A.

Not familiar with Machine Learning, however find the following articles, hope that helps:

• Machine Learning DevOps (MLOps) with Azure ML [Enabling CI/CD

• for Machine Learning project with Azure Pipelines

Answer 2

If you don't want the csv upload to happen in a pipeline you can write an Azure Function or Azure Logic App. Those can be triggered on changes or creations of blobs. Inside you could do a rest call to either start your pipeline like here api-for-automating-azure-devops-pipelines or retrain your model.