Substitue hex characters in z/OS UNIX File

Question

I need to convert single LF (x'0A') character in a z/OS UNIX file to couple CRLF (x'0D'x'0A') characters using sed noninteractive stream editoR invoked from a z/OS JCL step.

Wandering in this web site I've found other posts and I've tried this JCL step:

//SEDSTEP  EXEC PGM=BPXBATCH,REGION=0M                              
//STDERR   DD SYSOUT=2                                              
//STDOUT   DD SYSOUT=2                                              
//STDPARM  DD *                                                     
SH sed 's/\x0A/\x0Dx0A/g' </u/sftp/zwnmsft/E/wnmcapdf/arxiu_unix.txt
>/u/sftp/zwnmsft/E/wnmcapdf/arxiu_unix_sed.txt                      
/*

Unfortunately it does not work as expected.

Any help would be greatly appreciated.

Thanks in advance

After reading your comments I add this new information intended to clarify the issue:

1 - The file I try to convert is located witin z/OS UNIX but contains characters ASCII undestandable; not EBCDIC.

2- This file does not contains any native EBCDIC newline character X'15'.

3 - The file I try to convert looks like this after executing od -cx command:

ZXDALMA@CIGC:/u/sftp/zwnmsft/E/wnmcapdf> od -cx arxiu_unix.txt              
0000000000   060 061 062 063 064 065 066 067 070 071 012 071 070 067 066 065
                3031    3233    3435    3637    3839    0A39    3837    3635
0000000020   064 063 062 061 060 012                                        
                3433    3231    300A                                        
0000000026

4 - IF I execute an awk alternative like this:

//AWK      EXEC PGM=BPXBATCH,REGION=0M                                  
//STDERR   DD SYSOUT=2                                                  
//STDOUT   DD SYSOUT=2                                                  
//STDPARM  DD *                                                         
SH awk '{ORS=""; gsub("\x0A","\x0D\x0A"); print}'                       
/u/sftp/zwnmsft/E/wnmcapdf/arxiu_unix.txt >                             
/u/sftp/zwnmsft/E/wnmcapdf/arxiu_unix_bis.txt                           
/*

I get the desired results which are:

ZXDALMA@CIGC:/u/sftp/zwnmsft/E/wnmcapdf> od -cx arxiu_unix_bis.txt          
0000000000   060 061 062 063 064 065 066 067 070 071  \r 012 071 070 067 066
                3031    3233    3435    3637    3839    0D0A    3938    3736
0000000020   065 064 063 062 061 060  \r 012                                
                3534    3332    3130    0D0A                                
0000000030

5 and last - If I exeute the sed command what I get is as follows:

ZXDALMA@CIGC:/u/sftp/zwnmsft/E/wnmcapdf> od -cx arxiu_unix_sed.txt          
0000000000   060 061 062 063 064 065 066 067 070 071 012 071 070 067 066 065
                3031    3233    3435    3637    3839    0A39    3837    3635
0000000020   064 063 062 061 060 012  \n                                    
                3433    3231    300A    1500                                
0000000027

I hope these explantions help understand the situation I'm facing.

New addition of information from the comment posted by Hogstrom:

Here is the output of ls -H command:

ZXDALMA@CIGC:/u/sftp/zwnmsft/E/wnmcapdf> ls -H arxiu_unix.txt             
-rwxrwxrwx  bin    1 ZWNMSFT  G@PROJ        22 Sep  5 12:17 arxiu_unix.txt

Regarding enviroment variable here is the output if I execute env command:

ZXDALMA@CIGC:/u/sftp/zwnmsft/E/wnmcapdf> env                                    
_BPX_TERMPATH=OMVS                                                              
PATH=/bin:/usr/bin:/usr/lib:/usr/lpp/dfsms/bin:/usr/lpp/java/J7.1_64/bin:/u/zxda
lma::/usr/bin:/usr/lib:/usr/lpp/dfsms/bin                                       
SHELL=/bin/sh                                                                   
PS1=$LOGNAME@$SYS:$PWD>                                                         
COLUMNS=80                                                                      
PS2=>>                                                                          
SYS=CIGC                                                                        
_BPX_SPAWN_SCRIPT=YES                                                           
_=/bin/env                                                                      
_BPXK_SETIBMOPT_TRANSPORT=TCPIP                                                 
STEPLIB=none                                                                    
LOGNAME=ZXDALMA                                                                 
TERM=dumb                                                                       
_BPX_SHAREAS=YES                                                                
HOME=/u/zxdalma                                                                 
LINES=20                                                                        
TZ=MET-1DMET-2,M3.5.0/02:00:00,M10.5.0

One more important detail is that I've recently realized that in our z/OS UNIX installlation the ESCape character is [, not the traditional .

I've seen this detail watching at the bottom part of the terminal, I see this:

ZXDALMA@CIGC:/u/sftp/zwnmsft/E/wnmcapdf>                                        
 ===>                                                                           
                                                                          INPUT 
ESC=[   1=Help      2=SubCmd    3=HlpRetrn  4=Top       5=Bottom    6=TSO       
        7=BackScr   8=Scroll    9=NextSess 10=Refresh  11=FwdRetr 12=Retrieve

I've also tried my sed command with instead of \; but, unfortunately, I get the same results

I tried this out with a file that had two \0x0a and ran your sample. Here is the results looking at the files as hexdumps. infile=0a0a outfile=0a0a15 Is this your problem. — Hogstrom, Sep 04 '18 at 14:43
Should it be \x0D\x0A in a substitution portion of your sed command? — Philipp Grigoryev, Sep 04 '18 at 22:38
I think the problem is that sed's `s`-command only does substitution *within* a line - and the newline character is neither part of the previous nor of the next line. Al least that's what I read from the answers to https://stackoverflow.com/questions/1251999/how-can-i-replace-a-newline-n-using-sed — piet.t, Sep 05 '18 at 07:28
Hello David; I cannot use awk because there is a hidden flaw in this code excerpt. As you may know awk programming language is suitable for text files and, in principle, it is not intended to work with binary files. In my case I'm managing a file imported from an external server into my z/OS UNIX working platform and this file does not contains any \n which is the standard RS record separator variable. So, the code you've just seen works fine if the file I process is small but fails if the file is larger than LINE_MAX .So, I'd to know how to do this with sed or other editor in z/OSUNIX. — FRANCESC XAVIER DALMAU PUJOL, Sep 11 '18 at 12:25
Can you issue an `ls -H` against the file that your trying the conversion on. In addition, can you run the command `export` and provide the `_BPXK_*` variables. The answer to your question is complicated depending on the encoding, file attributes and env settings as `sed` is dependent on external settings in how it handles automatic conversions. — Hogstrom, Sep 15 '18 at 16:17
Thanks Hogstrom for your post; I've expanded the question with the information you have asked. — FRANCESC XAVIER DALMAU PUJOL, Sep 19 '18 at 16:13
Is there a reason you don't use `dos2unix` and `unix2dos` to switch between line endings in a text file? — Makyen, Oct 09 '18 at 20:16

David Crayford · Accepted Answer · 2018-09-12T08:50:00.317

I know this doesn't answer the original question WRT using sed. I understand your dilemma. You're trying to convert an ASCII UNIX text file to a Windows text file on a mainframe using tools designed for EBCDIC! It's trivial to write your own filter in C.

#include <stdio.h>
#include <stdlib.h>

static void put_char(int c) {
    if (putchar(c) == EOF) {
        fputs("Error: putchar() failed with unexpected EOF", stderr);
        exit(EXIT_FAILURE);
    }
}

int main()
{
    int c;
    while ((c = getchar()) != EOF) {
        if (c == '\x0A') put_char('\x0D');
        put_char(c);
    }
    return 0;
}

Test

DOC:/u/doc/src: >printf "hello word\x0ahello again\x0a" | trlf | hexdump
00000000 88859393 9640A696 99840D0A 88859393 |hello word..hell|
00000010 96408187 8189950D 0A                |o again..       |

Thanks David; I'm sure that this should work but in our installation I cannot use this. — FRANCESC XAVIER DALMAU PUJOL, Sep 19 '18 at 16:15

Substitue hex characters in z/OS UNIX File

1 Answers1