-1

I want to copy files from one folder to another folder in data lake using ADF pipelines. Ex : a/b/c/d. TO a/b Here a,b,c,d are folders here I don't want to copy c,d folders .I have to copy the files inside those folders to 'b' folder.

I created a pipeline using Get Metadata , For each and in For Each I used copy activity.But here I am able to copy files with folder itself .I'm failing to remove folders.

Daniel Mann
  • 57,011
  • 13
  • 100
  • 120
Sowmya thota
  • 1
  • 2
  • hi sowmya, have you checked PratikLad's post? If it helps you could [accept it as an answer](https://meta.stackexchange.com/questions/5234/how-does-accepting-an-answer-work/5235#5235) – Ceeno Qi-MSFT Nov 15 '22 at 06:58
  • Please do not use the tag [azure-pipelines] for questions related to Azure Data Factory pipelines. [azure-pipelines] is for questions related to Microsoft's Azure DevOps pipelines feature. – Daniel Mann Jan 09 '23 at 15:41

1 Answers1

0

I reproduced your scenario follow the below steps:

  • In my demo container I have nested folders like a/b/c/d under d folder I have 3 files as below. enter image description here

  • To copy files from folder to folder I took Get metadata activity to get list of files from folder. Dataset for Get Metadata: enter image description here Get Metadata settings: enter image description here

  • Then I took for-each activity and passed the output of Get Metadata activities output to it.

    @activity('Get Metadata1').output.childItems

enter image description here

  • Then created copy activity inside for each activity and created source dataset with filename parameter enter image description here In file name gave dynamic value as @dataset().filename enter image description here In copy activity source gave dynamic value for dataset property filename as @item().name enter image description here Now created sink dataset with a/b directories only enter image description here and passed it to sink enter image description here

Output

  • files copied under b folder without coping c and d folder enter image description here
Pratik Lad
  • 4,343
  • 2
  • 3
  • 11
  • Thank you for this. But my issue was there are some more folders in the 'C' and 'd' but I don't want to copy those folders .I want to copy only files in those folders .Like ---> I have d&e folders in the 'c' folder . I want to copy only files in those d&e folders . – Sowmya thota Nov 15 '22 at 06:15
  • Do you mean like your d folder has files along with folders and you want to copy only folder? – Pratik Lad Nov 16 '22 at 14:11
  • No .My question was I have some child folders inside parent folders and in that child folders I'm having some more child folders .Just I want to copy the files in those folders to target source without copying folders – Sowmya thota Nov 17 '22 at 05:45