Chaining InputOperations in Rhino-Etl

Question

I've just recently started using Rhino-Etl for very simple ETL processes and have had great success with it. I have a slightly more complicated scenario to address now and I didn't find the ConventionInputCommandOperation behaving the way I expected.

I've done up a very simplified example of what I'm trying to do. Basically I have two systems involved and I don't know what I want to get from system 2 until I first query system 1. I thought registering an InputOperation immediately after another InputOperation would behave like a loop. So that each row in operation 1 would be fed to operation 2. The below code fails with "Failed to execute operation DetailReader: Must declare the scalar variable @PlanetAbbrv." So my question is how are you meant to handle situations where the input operation is dependent a previous input operation?

Thanks, Brian

using System;
using Rhino.Etl.Core;
using Rhino.Etl.Core.ConventionOperations;

namespace ETLTest
{
    class Program
    {
        static void Main()
        {
            new MainProcess().Execute();
            Console.ReadLine();
        }
    }

    public class MainProcess : EtlProcess
    {
        protected override void Initialize()
        {
            Register(new MainReader());
            Register(new DetailReader());
        }

        protected override void PostProcessing()
        {
            foreach (var exception in GetAllErrors())
            {
                throw exception;
            }
        }
    }

    public class MainReader : ConventionInputCommandOperation
    {
        public MainReader() : base("Galactic1")
        {
            Command = @"select * from Planet";
        }
    }

    public class DetailReader : ConventionInputCommandOperation
    {
        public DetailReader() : base("Galactic2")
        {
            Command = @"select * from Delivery where DeliveryPlanetAbbrv = @PlanetAbbrv";
        }
    }
}

score 0 · Answer 1 · answered Jan 03 '14 at 15:46

You'll need to have your DetailReader select all rows (take out the where operation). Then use a JoinOperation to match the details to the main information.

Register(new JoinPlanets()
                     .Right(new MainReader())
                     .Left(new DetailReader()));


public class JoinPlanets: JoinOperation
{
    protected override Row MergeRows(Row leftRow, Row rightRow)
    {
        Row row = leftRow.Clone();
        foreach (var column in leftRow.Columns)
            row[column] = leftRow[column];
        return row;
    }

    protected override void SetupJoinConditions()
    {
        FullOuterJoin.Left("PlanetAbbrv")
                     .Right("DeliveryPlanetAbbrv");
    }
}

Chaining InputOperations in Rhino-Etl

1 Answers1