How do I remove the last 2 columns from a tsv file?

Question

I am coding using bash on terminal through a docker container on my mac. I am struggling to figure out how to remove the last 2 columns on my TSV file. It has 7 total and the last 2 are not needed for my work and are required to be removed.

Edit: The first picture is the original data file, the second is what the code is doing and it is deleting some random entries from the column. The third picture is what the end result of this program should do. The month and year columns I am struggling with also but I deleted the code and tried to simplify the data first.

I tried using awk and using NF = NF - 2 which does remove the last 2 columns but for some reason deletes some of the data I have in my 5th column which I need. So whilst I got the column deletion I needed, the code did a little extra. Here is the code:

preprocess() {
 31     input_file="$1"
 32 
 33     # Extract the base name of the input file
 34     base_name=$(basename "$input_file" .tsv)
 35 
 36     # Create the new output file name
 37     output_file="${base_name}_clean.tsv"
 38 
 39     awk -F'\t' 'BEGIN{OFS=FS} 
 40     {
 41         NF = NF - 2
 42 
 43         print
 44     }' "$input_file" > "$output_file"
 45 }

I Have a few other lines but they shouldn't cause any issues. They just check the file exists etc.

Answer 1

With AWK a script like this works:

Inputdata (separated by ; for clarity here, but could be tab also).

F1;F2;F3;F4
V11;V12;V13;V14
V21;V22;V23;V24

Program to convert. Comments to everyone can follow, even those new to awk.

BEGIN{
   FS=";"    # Convert til "\t" for TAB separation
   OFS=";"   # set as desired
   skipcolcount=2
}

{ 
  # In each line, loop over the fields
  for (i=1;i<=NF-skipcolcount;i++) {   
     printf $i                   # reference field by index variable  
     if (i<NF-skipcolcount) {    # no separator after last field
       printf OFS
     }
  }
  printf "\n"                    # linefeed after each line
}

Result:

F1;F2
V11;V12
V21;V22

Answer 2

I tried using awk and using NF = NF - 2 which does remove the last 2 columns but for some reason deletes some of the data I have in my 5th column which I need.

This is unexpected for me, I did run your code using GNU Awk 5.1.0 and it works fine, however you are using

docker

so maybe this force usage of erratic version of awk? Anyway, if your task is given as

how to remove the last 2 columns on my TSV file. It has 7 total and the last 2 are not needed for my work and are required to be removed.

this might be simplified to: get first 5 columns of tab-separated file, which can be expressed in awk as

awk 'BEGIN{FS=OFS="\t"}{print $1,$2,$3,$4,$5}' file.tsv

Please run it and write if output is as desired.

Answer 3

The easiest will be rev/cut/rev combination

$ rev inputfile | cut -f3- | rev > output.file

Answer 4

You are close.

Given the following TSV file:

cat file
1   2   3   4   5
6   7   8   9   10
11  12  13  14  15

You can do this in awk:

awk 'BEGIN{FS=OFS="\t"}
{NF=3} 1
' file 

Prints:

1   2   3
6   7   8
11  12  13

Or Ruby:

ruby -ne 'puts $_.split("\t")[0..2].join("\t")' file
# same

Or Perl:

perl -nE 'say join("\t", (split "\t")[0..2])' file
# same

Answer 5

One way could be to use sed:

sed 's/\t[^\t]*\t[^\t]*$//' "$input_file" > "$output_file"

Match explained:

• \t - a tab character
• [^\t]* - zero or more non-tab characters
• \t - a tab character
• [^\t]* - zero or more non-tab characters
• $ - end of line anchor

Substitute (the // part) with an empty string.

An alternative:

sed -E 's/(\t[^\t]*){2}$//' "$input_file" > "$output_file"

Here the match \t[^\t]* is in a group (...) which is repeated twice {2}.