How to use pivot component in clover dx

Question

Unable to use pivot component , please let me know with example how to use pivot and concatenation components in clover etl

Answer 1

I always describe Concatenate as boarding queues at the airport. You have multiple queues (edges), in all of them people are standing (same records, ie. metadata), all of them get boarded by 1) queue (army first, first class then, economy...), and 2)order in the queue (who is standing first in the queue goes first, second, third...)

I wasn't using Pivot component too much, but I think its kind of a denormalize component. You have bunch of simple records (think key, value pairs from json or NoSQL database) and want to merge them into one wide record, where fields are keys of incoming tuples, values are values from incoming records. Its a grouping component, used when you have multiple records for same group key (customer_id for example) and want to produce one wide record with all available properties.

I prefer Denormalize component as it gives me more control, but it requires a bit more of CTL, for easier things you might be fine with Pivot.

Concatenate component serves totally different use case (collecting same looking records from multiple input sources - similar components SimpleGather, Merge) then Pivot (transforming simple records into denormalized, wide one - similar components Denormalizer).