0

I have a requirement to execute SQL script files that are either version controlled in a git repo OR versioned within Azure Data Factory itself. I have seen example pipelines online about executing files on ADF that are sitting in Azure Blob Storage, but we dont have version controlling in there and there is no interest in introducing LakeFS, I would rather execute files sitting in a git repo (Or the script files that are stored within Azure Data Factory itself) using ADF pipelines.

Can someone please tell me if this is possible?

Thank you lots in advance.

Justin Thobela
  • 1
  • 1

1 Answers1

2

You can use any API call that will let you get the content from your file with a Web Activity, and then pass the content to either a Script activity or a Copy Activity.

Here is an example: enter image description here

enter image description here

Here I call a web activity to get the file (in this example reading from Github with a Personal Access Token, using it in the headers). Then I reference the contents from the Script activity. You can do the same with a repo in Azure DevOps.

You could also use the contents in a copy activity to execute a code that will return data as a Source.

Martin Esteban Zurita
  • 3,161
  • 12
  • 23