Design - Ruby- Create multiple CSV files as per User defined header From same data source

Question

I am working on requirements we have data in hash around 100+ keys. we need to generate CSV file as per user-defined header with some transformation, we may end up having 100+ template Main changes will be 1) Change column name such as Fname –> First name 2) Data transformation like Full name – > First name + Last name (adding 2 column) 3) Fixing the position of a column – Fname should be at 35 positions etc.

please suggest is it possible to define declarative way or any gem available. Can you let me know any design pattern we can apply here?

Some sample scenarios

I have input like this with many columns (100+)

[ {:employee_id=>"001", :first_name=>"John",:last_name=>"Dee" :date_of_birth=>"10/10/1983", :salary=>"100000",:bounus =>"50000",......},
  {:employee_id=>"002", :first_name=>"Alex",:last_name=>"Peck" :date_of_birth=>"11/01/1988", :salary=>"120000",:bounus =>"70000", .........},
 ]

Some customer need CSV as

Employee ID, First Name, Last Name, Date of birth, Salary, Bonus
001,John,Dee,10/10/1983,100000,50000,...
002,Alex,Peck,11/01/1988,120000,70000,...

Others (only header change)

ID, FName, LName, Dob, Salary, Bounus
001,John,Dee,10/10/1983,100000,50000,...
002,Alex,Peck,11/01/1988,120000,70000,...

another (merge of colum FName, LName -> Fullname)

ID, Fullname, Dob, Salary, Bounus
001,John Dee,10/10/1983,100000,50000,...
002,Alex Peck,11/01/1988,120000,70000,...

anothers (merge of column Salary, Bonus -> Salary+ Bonus)

ID, FName, LName, Dob, Salary
001,John,Dee,10/10/1983,150000,...
002,Alex,Peck,11/01/1988,190000,...

anothers ( column order changed also insted of Dob need age)

FName, LName, ID, age, Salary
John,Dee,001,36,150000,...
Alex,Peck,003,32,190000,...

Like many variations with the same input

Thanks for help

Answer 1

What you need is the presenter design pattern.

Your controller will request the data and store it in a local variable, and then your will have to load a presenter for your client passing it the data variable. In response you'll get the final CSV to return to the client.

Let's say you clients have uniq codes, so that a Client model instance has a code attribute which is a string.

So your controller would looks like this:

app/controllers/exports_controller.rb

class ExportsController < ApplicationController
  def export
    data = MyService.fetchData # <== data contains the data you gave as an example

    # Gets the right presenter, initialise it, and build the CSV
    csv = PresenterFactory.for(current_client).new(data).present

    respond_to do |format|
      format.html
      format.csv { send_data csv, filename: "export-name-for-#{current_client.code}.csv" }
    end
  end
end

The PresenterFactory class would be something like that:

app/models/presenter_factory.rb

class PresenterFactory
  def self.for(client)
    # For client with code "ABCD" it will return Presenters::Abcd class
    "Presenters::#{client.code.capitalize}".constantize
  end
end

The factory return the client's presenter class

And here is an example for a client's presenter class, for a client having the code ABCD:

app/models/presenters/abcd.rb

module Presenters
  class Abcd
    def initialize(data)
      @data = data
    end

    def present
      CSV.generate(headers: true) do |csv|
        # Here is the client's specific CSV header
        csv << [
          'Employee ID',
          'First Name',
          # ...
        ]

        @data.each do |row|
          # Here is the client's specific CSV row
          csv << [
            row[:employee_id],
            row[:first_name],
            # ...
          ]
        end
      end
    end
  end
end

Answer 2

You can achieve your objective by constructing a transformation hash whose keys are the names of the columns in the desired CSV file, in order, and whose values are procs, which when called with an argument equal to an element of the given array of hashes, returns an element to be written in a row of the CSV file in the column corresponding to the key.

Code

require 'csv'

def construct_csv(fname, arr, transform)
  CSV.open(fname, "wb") do |csv|
    keys = transform.keys  
    csv << keys  
    arr.each { |h| csv << keys.map { |k| transform[k].call(h) } }
  end  
end  

Examples

I will now illustrate how this method is used with various transformations.

Common data

arr = [{:employee_id=>"001", :first_name=>"John", :last_name=>"Dee",
        :date_of_birth=>"10/10/1983", :salary=>"100000", :bonus=>"50000" },
       {:employee_id=>"002", :first_name=>"Alex", :last_name=>"Peck",
        :date_of_birth=>"11/01/1988", :salary=>"120000", :bonus=>"70000" }]

FName = 'temp.csv'

Write a CSV file with the same keys, in the same order, and the same values

keys = arr.first.keys
  #=> [:employee_id, :first_name, :last_name, :date_of_birth, :salary, :bonus] 
transform = keys.each_with_object({}) { |k,g| g[k] = ->(h) { h[k] } }
  #=> {:employee_id=>#<Proc:0x00005bd270a0e710@(irb):451 (lambda)>,
  #    :first_name=>#<Proc:0x00005bd270a13260@(irb):451 (lambda)>,
  #    ...
  #    :bonus=>#<Proc:0x00005bd270a19cc8@(irb):451 (lambda)>} 
construct_csv(FName, arr, transform)

Let's see what was written.

puts File.read(FName)
employee_id,first_name,last_name,date_of_birth,salary,bonus
001,John,Dee,10/10/1983,100000,50000
002,Alex,Peck,11/01/1988,120000,70000

Write a CSV file with the columns reordered¹

col_order = [:last_name, :first_name, :employee_id, :salary, :bonus,
             :date_of_birth]
keys = arr.first.keys
order_map = col_order.each_with_object({}) { |k,h| h[k] = keys.index(k) }
  #=> {:last_name=>2, :first_name=>1, :employee_id=>0, :salary=>4,
  #    :bonus=>5, :date_of_birth=>3} 
transform = col_order.each_with_object({}) { |k,g|
  g[k] = ->(h) { h[keys[order_map[k]]] } }
  #=> {:last_name=>#<Proc:0x00005bd270f8e5a0@(irb):511 (lambda)>,
  #    :first_name=>#<Proc:0x00005bd270f8e550@(irb):511 (lambda)>,
  #    ...
  #    :date_of_birth=>#<Proc:0x00005bd270f8e3c0@(irb):511 (lambda)>} 
construct_csv(FName, arr, transform)

puts File.read(FName)
last_name,first_name,employee_id,salary,bonus,date_of_birth
Dee,John,001,100000,50000,10/10/1983
Peck,Alex,002,120000,70000,11/01/1988

Write a CSV file with a subset of keys, renamed and reordered

keymap = { :FirstName=>:first_name, :LastName=>:last_name, :ID=>:employee_id,
           :Salary=>:salary, :Bonus=>:bonus } 
transform = keymap.each_with_object({}) { |(new,old),g| g[new] = ->(h) { h[old] } }
  #=> {:FirstName=>#<Proc:0x00005bd270d50298@(irb):391 (lambda)>,
  #    :LastName=>#<Proc:0x00005bd270d50220@(irb):391 (lambda)>,
  #    ...
  #    :Bonus=>#<Proc:0x00005bd270d830f8@(irb):391 (lambda)>} 
construct_csv(FName, arr, transform)

puts File.read(FName)
FirstName,LastName,ID,Salary,Bonus
John,Dee,001,100000,50000
Alex,Peck,002,120000,70000

Write a CSV file after removing keys and adding keys whose values are computed

keys_to_remove = [:first_name, :last_name]
keys_to_add = [:full_name, :compensation]
keys = arr.first.keys + keys_to_add - keys_to_remove
  #=> [:employee_id, :date_of_birth, :salary, :bonus, :full_name,
  #    :compensation] 
transform = keys.each_with_object({}) do |k,h|
  h[k] =
  case k
  when :full_name
    ->(h) { h[:first_name] + " " + h[:last_name] }
  when :compensation
    ->(h) { h[:salary].to_i + h[:bonus].to_i }
  else
    ->(h) { h[k] }
  end
end
  #=> {:employee_id=>#<Proc:0x00005bd271001000@(irb):501 (lambda)>,
  #    :date_of_birth=>#<Proc:0x00005bd271000f88@(irb):501 (lambda)>,
  #    :salary=>#<Proc:0x00005bd271000f10@(irb):501 (lambda)>,
  #    :bonus=>#<Proc:0x00005bd271000ec0@(irb):501 (lambda)>,
  #    :full_name=>#<Proc:0x00005bd271000e20@(irb):497 (lambda)>,
  #    :compensation=>#<Proc:0x00005bd271000dd0@(irb):499 (lambda)>} 
construct_csv(FName, arr, transform)

puts File.read(FName)
employee_id,date_of_birth,salary,bonus,full_name,compensation
001,10/10/1983,100000,50000,John Dee,150000
002,11/01/1988,120000,70000,Alex Peck,190000

^{1. I don't understand the reason for doing this but it was mentioned as a possible requirement.}