1

I want to join two datasets that are coming from the same source. And I got this concatenation disallowed error.

Below are some details about my problem scope:

  1. For the source: I am extracting from REST API, after parsing the JSON output, there will be 2 response groups - Response 200 (contains PK_Response200), and Behaviors(contains FK_Response200), and in each group, I have the system automatically generated PK and FK. Since these two groups are coming from one API call, that's why I say that I am attempting to join two datasets from one source. source details

  2. For the joiner: I want to use the PK_Response200 and FK_Response200 to join these two response groups together, and only take out several columns from the joiner result.

Different methods I tried:

  1. First save the two output groups to two flat file (csv files), and then create another mapping which has two sources that reads from these two flat file csv. This one actually can work, but I still want to know if there are methods to realize all these in only one mapping so I don't have too much dependencies and also don't need to worry about deleting the local csv files created in the middle steps.

  2. I tried to use two expression transformation to hold these two output groups, and then join them, like the photo below:two expression transformation to one joiner transformation But then I receive error like this:error message

  3. I noticed this might be caused by two passive transformations (expression transformation), so I deleted them and changed to sorter transformation since it is suggested in the log file that I should 'Choose sorted input for Joiner to join data from the same source.', however, I still got the same error message. two sorter transformation to one joiner transformation

Anyone know how to solve this error? Thank you so much!

La. Li
  • 41
  • 6

1 Answers1

0

You are one right track. Solution would be -

  1. Keep sorter #1 and remove the sorter #2. Make sure you sort by the key columns you are going to use in joiner in sorter 1.
  2. join the out put of sorter 1 to master and to details.

Pls refer to below pic.

enter image description here

Koushik Roy
  • 6,868
  • 2
  • 12
  • 33
  • Hi Koushik, thank you so much for helping me! Since my source have two output groups so I did not find the way to connect all two of them to the same sorter, let me know if I misunderstood you method ;) But I tried another method which is to still use expression transformation, and then in the joiner choose sorted output and it worked. – La. Li May 08 '23 at 13:05
  • i got from your picture, that youre trying to sort same source data so i tried to answer like that. Yes, if you have 2 sources, order them using different sorter to sort and then join but if they r diff source you dont need sorter. – Koushik Roy May 08 '23 at 14:41