Azure Data Factory File copying from two different datasets to one folder with two subfolders using lookup value

Question

Everyone I'm trying to copy files from two different tables from postgres database, for which I created two datasets, one linked service, one output dataset (parametrized folder name,filename)

step1: My Pipeline is connecting a lookup table which contains column rownumber,lookup e.g. lookup column values ['output1','output2']

step2: I used an iterator (Foreach Activity) to fetch lookup array items

step3: changing data type array items

step4: switch activity, expression if foreach item value match with let's say 'output1' then it will execute copy operation and fetch file from one table and will place in output location which will also dynamically create a folder 'output1' and will dump copied file with name output.csv.

but it seems all operation is running fine only second case from switch operation is not working. It Seems i couldn't find any reason why is it not copying to second location, if condition is false

here is the expression I've used for switch operation:

 @if(equals(variables('var'),'output1'),'True','False')

switch activity setting Pipeline Inside Loop:

Pipeline debug results:

Can you please post the error you get? Also, please note that running foreach in parallel may confuse the set variable activity, and you will not get a different value for each loop. — Chen Hirsh, Apr 16 '23 at 08:33

score 0 · Answer 1 · answered Apr 17 '23 at 12:31

i couldn't find any reason why is it not copying to second location, if condition is false

I have also reproduced the similar scenario as you and its working fine. Till switch activity your all the values and pipeline is correct. the issue you are facing with copy activity may be.

As per you ask you want to copy from two different datasets to one folder with two subfolders.

To achieve that you need to specify dynamic content correctly:

Here in source of copy activity I added the source of file from where have to copy the file.
Here in the sink we added the dynamic expression to copy file in logs folder under that it will create Output1 folder and under that it will place the output.csv file with expression @concat('logs/',variables('string')).