0

We have multiple very old ISAM COBOL files on mainframe containing historical data. I have to build an ETL process to read/migrate this data (stored in the ISAM files) with Pentaho DI and load the same to PostgreSQL database with little or no transformations.

How do I achieve this with Pentaho DI ?

One workaround solution could be to write a Cobol program to convert the ISAM files to regular datasets in mainframe and then ftp/read the converted files easily. However, please let me know if there is a solution to read directly from the ISAM files with Pentaho without having to develop another cobol program. Also, please let me know any tips about the migration.

mzy
  • 1,754
  • 2
  • 20
  • 36
Raj Aryan
  • 59
  • 3
  • 12
  • Do you really mean ISAM, or do you really mean VSAM? I doubt anyone would support anything with Mainframe ISAM files, they'd not expect anyone to still be using them 30+ years after VSAM existed, – Bill Woodger Mar 16 '16 at 21:09
  • 1
    One possible alternative is to find a JDBC driver for your files (whatever their type). On first look though, I only see commercial offerings. – Brian.D.Myers Mar 16 '16 at 22:37

1 Answers1

3

You are not going to be able to directly import ISAM (VSAM ???) files directly into Pentaho.

You do not need to write a program to unload an ISAM (VSAM ???) files, existing Mainframe utilities (sort etc) can be used to do the unload. The unloaded file could then be transmitted to your system (it is probably going to be a binary-ebcdic file), wether you can process the file is a different question.

Files:

Complicated files (e.g. have Cobol Redefines, Occurs Depending), are going to require Cobol.

For simple files:

  • a Mainframe-Sort can convert the Binary files to Text if needed.
  • From memory Pentaho has some Cobol capabilities (based on cobol2j / cb2xml)
  • there is a legstar plugin for cobol processing.
  • There is a Cobol-to-Csv programs in JRecord that can be used convert Cobol files to CSV.

But:

  • The files may not be sufficient, you may need other related details which could require additional Cobol extract programs.
  • The Structure of the Current Tables are probably not ideal for your new purpose. Some times, it might be easier to do some transformation of the Mainframe because you have all the related Tables / DB's present on the Mainframe.

Things to watch out for

  • Cobol Redefines - needs to be sorted out in Cobol on the Mainframe (or in Java~JRecord step). A Redefines clause is a major problem when Pic-x/9 fields redefine a binary (comp/comp-3) fields.
  • Occurs Depending On - Fix in Cobol before transmission.
  • Multi-Record files

Other approaches

Disclosure: I wrote JRecord and worked on cb2xml.

Community
  • 1
  • 1
Bruce Martin
  • 10,358
  • 1
  • 27
  • 38
  • 1
    What a shame it was not accepted as an answer by the OP. – PepitoSh Apr 14 '21 at 16:27
  • Thank you @PepitoSh the way it goes. Not may people use Cobol not many reads / upvotes. On the positive side there is not as much competition to answer questions – Bruce Martin Apr 14 '21 at 23:56