I am following the tutorial at https://learn.microsoft.com/en-us/azure/machine-learning/tutorial-pipeline-python-sdk to create a training pipeline. Instead of using a file from a web path, I am using a CSV saved at the same location as notebook. Once the training pipeline is created, I publish it. Now I want to resubmit the training pipeline but with a new training data. I overwrote the previous training CSV with a new training CSV and clicked on Resubmit for the pipeline endpoint. But instead of running the training using the new data, the pipeline is still training with the previous data. How do I update the Data Asset to a new version with the latest CSV and how do I make sure my pipeline run should pick up this latest data instead of continuing to use the data set which was used for the first time the pipeline was run and published.
I want to replicate scenarios in production where the training pipeline will be invoked when there is new training data available. If pipeline continues to train with the previous data, it does not make sense. I Cannot create a new pipeline every time the data changes. The pipeline should be able to pick up the new data.
