Is it possible to do a Lookup use Kiba

Question

Is it possible to do a "Lookup" with Kiba.

Since it's quite a normal process in a etl.

Could you show a demo if yes, thanks.

Answer 1

Yes, a lookup can be done with Kiba!

For a tutorial, see this live coding session I recorded, I create a lookup transform to lookup extra fields using a given fields by tapping in the MovieDB database.

Leveraging this example, you could for instance implement a simple ActiveRecord lookup using a block transform:

# assuming you have a 'country_iso_2' field in the row above
transform do |row|
  country = Country.where(iso_2: row['country_iso_2']).first
  row['country_name'] = country.try(:name) || 'Unknown'
  row
end

or you could extract a more reusable class transform that you would call like this:

transform ActiveRecordLookup, model: Country, 
  lookup_on: 'country_iso_2', 
  fetch_fields: { 'name' => 'country_name' }
transform DefaultValue, 'name' => 'Unknown'

Obviously, if you have the need for large volumes, you will have to implement some improvements (e.g. caching, bulk reading).

Hope this helps!