0

I am trying to convert utf-8 files from a directory listing on USS into ebcdic files BEFORE getting them into z/OS datasets.

Using a helper function which I found on stackoverflow (thanks for this!) I can issue shell-commands from within the python script:

def r(cmd_line): 
     return Popen(cmd_line.split(), stdout=PIPE).communicate()[0]

With this I can allocate and populate mainframe datasets from USS-files, using 

 r("tso alloc DSNAME(...) etc.")    # to allocate a mainframe DS and
 r("tso oget ...")                  # to populate the mainframe DS

However: some files need to be converted first, which in a shellscript I would simply code with

iconv -f UTF-8 -t IBM-1141 $utf8_file > $ebcdic_file

and I am totally at a loss of how to do this in python (2.7)?

Can't ask anybody in my shop since python was newly installed and I am currently the only one interested in it. Anyone an idea? Thanks a lot in advance!

Dirk
  • 1
  • 2
  • 1
    See https://stackoverflow.com/questions/191359/how-to-convert-a-file-to-utf-8-in-python. – meat Jan 29 '20 at 19:26
  • Is there any reason you have to use iconv? – Kevin McKenzie Jan 30 '20 at 03:52
  • @meat: thanks, that looks good; although that runs me into a probable encoding issue with the sourcefile: UnicodeDecodeError: 'utf8' codec can't decode byte 0x78 in position 2: invalid continuation byte – Dirk Jan 30 '20 at 07:09
  • @ Kevin: what would be an alternative way? – Dirk Jan 30 '20 at 07:10
  • If it cannot decode it and the file _looks_ like it is in UTF-8 it might be ISO-8859-1. You can also consider using file tagging which would automatically convert the file when it is opened. – meat Jan 30 '20 at 12:06
  • tried out the following, but the print output is 0 - `sourceFileName = "/u/arb07/myscripts/python/utf8file" ` `targetFileName = "/u/arb07/myscripts/python/ebcdicfile" ` How can I redirect the converted file from stdout to a file? – Dirk Jan 30 '20 at 12:56
  • OK, thanks everybody! I think I got it: if I use a file-object and redirect `stdout=myTargetFile` in subprocess.call() I do get a converted file. – Dirk Jan 30 '20 at 13:24

1 Answers1

0

Although not in the true spirit of python, you can do what you want by wrapping USS commands in a python script. Here is an example:

#!/bin/env python

from cStringIO import StringIO
import os
import sys

def r(cmd):
    import subprocess
    return subprocess.Popen(cmd, stdout=subprocess.PIPE).communicate()[0]

def allocate_dataset(dsName):
    name = "'" + dsName + "'"
    out = r(['/bin/tso', 'alloc',  'ds(' + name + ')',  'space(6000 2000)',  'track', 'lrecl(80)',
             'dsntype(library)', 'blksize(3200)',  'recfm(f b)',  'dir(2)',  'new'])
    for line in out.split():
        print line

def not_allocated(dsName):
    name = "'" + dsName + "'"
    out = r(['/bin/tsocmd', 'listds ' + name])
    for line in out.split():
        if "NOT IN CATALOG" in out:
            return True
    return False

def ascii_to_ebcdic(from_codepage, to_codepage, fileName):
    os.system('iconv -f' + from_codepage + ' -t' + to_codepage + ' <' + fileName + ' >ebcdic_' + fileName)

def copy_to_dataset(fileName, dsName, memberName):
    dsn = "//'" + dsName + '(' + memberName + ")'"
    os.system('cp -T ' + fileName + ' "' + dsn + '"')

def main():
    dsName = "HLQ.MY.PYTHON"
    if not_allocated(dsName):
        print("Allocating '" + dsName + "' data set")
        allocate_dataset(dsName)
    ascii_to_ebcdic("UTF-8", "IBM-1047", "test.txt")
    copy_to_dataset("ebcdic_test.txt", "HLQ.MY.PYTHON", "TXT")
    member = "//'HLQ.MY.PYTHON(TXT)'"
    os.system('cat -v "' + member + '"')

main()
Milos Lalovic
  • 554
  • 3
  • 10
  • that was pretty nifty! Thanks for the example - esp. for showing how quoting within quoting would work **(' " some var " ')**. It's really tedious trying to figure that out by trial and error. ;-) – Dirk Jan 31 '20 at 18:28