Removing the trailing delimiters dynamically

Question

I am new to scripting and I need to write a script to remove unwanted delimiter from the last line of the flat file which will be changing dynamically.

EG:

1|AK|1,asd|CHN|TN|IN|600001
2|AK|1,asd|CHN|TN|IN|600001
AK|10-01-2019|2||||

Like this different files are there and row and column will be varying for each file. So how to remove the trailing pipelines only from the last row (I,e) after 2

Thanks and Regards,
Aravind Kumar Sekar

Answer 1

The documentation for Perl's split() function says this:

If LIMIT is omitted (or, equivalently, zero), then it is usually treated as if it were instead negative but with the exception that trailing empty fields are stripped (empty leading fields are always preserved)

So, the default behaviour of split() is to remove trailing empty fields. We can then use join() to regenerate the original record minus any trailing delimiters.

So it can be as simple as:

# Note that '|' has a special meaning in regexes, so we
# need to escape it, using '\'.
my $output_line = join '|', split /\|/, $input_line;

Or, putting it in a complete program that reads from STDIN and writes to STDOUT:

#!/usr/bin/perl

use strict;
use warnings;
use feature 'say';

while (<>) {
  chomp;
  say join '|', split /\|/;
}

If you put that in a file called rmtrailing, then it can be run from a command line like this:

$ perl rmtrailing < your_input_file.txt > your_output_file.txt

Answer 2

The following will delete the trailing pipes from all lines:

perl -ple's/\|+\z//'

If you want to leave trailing pipes in place on all but the last line, you can use the following:

perl -nle'
   print($prev) if defined($prev);
   $prev = $_;
   END {
      if (defined($prev)) {
         $prev =~ s/\|+\z//;
         print($prev);
      }
   }
'

Specifying file to process to Perl one-liner

Answer 3

This sed one-liner will do the job:

sed '$s/|*$//' file

Test:

kent$  cat f
a|b
a|b||||||
||||
foo|bar||||||

kent$  sed '$s/|*$//' f
a|b
a|b||||||
||||
foo|bar

You see that only the last line was changed.