remove the first 15 characters from every other line in a file

Question

I have some txt files that look like this (they contain DNA sequences and sample codes):

>SRR1502445.1
GACTACACGTAGTATACGAGTGCGTTCCTGCGCTTATTGATATGCTTAAGTTCAGCGGGTAGTCTCACCCGATTTGAGGTCAAGGTTTGTGGGTCGAGTCACAACTCGAACATCGTCTTTGTACAAAGACGGTTGGAAGCGGGTTCCAAGGCAACACAGGGGGATAGGNNNNNNNNNNNNNNNNNNNNNNN
>SRR1502445.2
GACTACACGTAGTATACGAGTGCGTTCCTGCGCTTATTGATATGCTTAAGTTCAGCGGGTAGTCTCACCCGATTTGAGGTCAAGGTTTGTGGGTCGAGTCACAACTCGAACATCGTCTTTGTACAAGACGGTTGGAAGCGGGTTCCAAGGCACACAGGGGATAGGNNN
>SRR1502445.3
GACTACACGTAGTATACGAGTGCGTTCCTGCGCTTATTGATATGCTTAAGTTCAGCGGGTAGTCTCACCCGATTTGAGGTCAAGGTTTGTGGGTCGAGTCACAACTCGAACATCGTCTTTGTACAAAGACGGTTGGAAGCGGGTTCCAAGGCACACAGGGGATAGGNNN
>SRR1502445.4
GACTACACGTAGTATACGAGTGCGTTCCTGCGCTTATTGATATGCTTAAGTTCAGCGGGTAGTCTCACCCGATTTGAGGTCAAGGTTTGTGGGTCGAGTCACAACTCGAACATCGTCTTTGTACAAAGACGGTTGGAAGCGGGTTCCAAGGCACACAGGGGATAGGNNNNNNNNNNN

I would like to remove the first 15 characters of every other line in the file. This would remove the string GACTACACGTAGTAT from the second, fourth, sixth, eighth lines (etc).

For instance the cut command can remove the first 15characters of every line:

cut -c 1-15 /path/to/file.txt

I'd like to apply to to only every other line, starting with the second.

Answer 1

If you don't mind using sedand assuming other line starts with > then the following will remove the first 15 contiguous uppercase characters "A-Z" of the other lines:

sed 's/^[A-Z]\{15\}//' file > new_file

Or, in place edit (GNU sed) use -i:

sed -i 's/^[A-Z]\{15\}//' file

Or, in place edit (BSD sed) use -i '':

sed -i '' 's/^[A-Z]\{15\}//' file

Or, back it up:

sed -i.bak 's/^[A-Z]\{15\}//' file

Example:

$ cat file
>SRR1502445.1
GACTACACGTAGTATACGAGTGCGTTCCTGCGCTTATTGATATGCTTAAGTTCAGCGGGTAGTCTCACCCGATTTGAGGTCAAGGTTTGTGGGTCGAGTCACAACTCGAACATCGTCTTTGTACAAAGACGGTTGGAAGCGGGTTCCAAGGCAACACAGGGGGATAGGNNNNNNNNNNNNNNNNNNNNNNN
>SRR1502445.2
GACTACACGTAGTATACGAGTGCGTTCCTGCGCTTATTGATATGCTTAAGTTCAGCGGGTAGTCTCACCCGATTTGAGGTCAAGGTTTGTGGGTCGAGTCACAACTCGAACATCGTCTTTGTACAAGACGGTTGGAAGCGGGTTCCAAGGCACACAGGGGATAGGNNN
>SRR1502445.3
GACTACACGTAGTATACGAGTGCGTTCCTGCGCTTATTGATATGCTTAAGTTCAGCGGGTAGTCTCACCCGATTTGAGGTCAAGGTTTGTGGGTCGAGTCACAACTCGAACATCGTCTTTGTACAAAGACGGTTGGAAGCGGGTTCCAAGGCACACAGGGGATAGGNNN
>SRR1502445.4
GACTACACGTAGTATACGAGTGCGTTCCTGCGCTTATTGATATGCTTAAGTTCAGCGGGTAGTCTCACCCGATTTGAGGTCAAGGTTTGTGGGTCGAGTCACAACTCGAACATCGTCTTTGTACAAAGACGGTTGGAAGCGGGTTCCAAGGCACACAGGGGATAGGNNNNNNNNNNN
$ sed 's/^[A-Z]\{15\}//' file
>SRR1502445.1
ACGAGTGCGTTCCTGCGCTTATTGATATGCTTAAGTTCAGCGGGTAGTCTCACCCGATTTGAGGTCAAGGTTTGTGGGTCGAGTCACAACTCGAACATCGTCTTTGTACAAAGACGGTTGGAAGCGGGTTCCAAGGCAACACAGGGGGATAGGNNNNNNNNNNNNNNNNNNNNNNN
>SRR1502445.2
ACGAGTGCGTTCCTGCGCTTATTGATATGCTTAAGTTCAGCGGGTAGTCTCACCCGATTTGAGGTCAAGGTTTGTGGGTCGAGTCACAACTCGAACATCGTCTTTGTACAAGACGGTTGGAAGCGGGTTCCAAGGCACACAGGGGATAGGNNN
>SRR1502445.3
ACGAGTGCGTTCCTGCGCTTATTGATATGCTTAAGTTCAGCGGGTAGTCTCACCCGATTTGAGGTCAAGGTTTGTGGGTCGAGTCACAACTCGAACATCGTCTTTGTACAAAGACGGTTGGAAGCGGGTTCCAAGGCACACAGGGGATAGGNNN
>SRR1502445.4
ACGAGTGCGTTCCTGCGCTTATTGATATGCTTAAGTTCAGCGGGTAGTCTCACCCGATTTGAGGTCAAGGTTTGTGGGTCGAGTCACAACTCGAACATCGTCTTTGTACAAAGACGGTTGGAAGCGGGTTCCAAGGCACACAGGGGATAGGNNNNNNNNNNN
$ 

Answer 2

You can try

sed '0~2s/^.\{15\}//g' filename

0~2 takes every 2nd line

^.\{15\}

looks for the first 15 characters

The sed command replaces them with nothing!

Answer 3

The following script might help you, it takes two arguments: 1. Original file (from which to make a conversion) 2. File where to save results.

#!/bin/bash
# call this script and pass two arguments:
# ./script FROM_FILE TO_FILE
FROM=$1
TO=$2

i=1;
while IFS=$'\n' read line; do
    ((i++)); 
    # skip 2,4,6, ..., nth lines 
    [ $((i % 2)) -eq 0 ] && (echo -n $line >> $TO; continue);
    echo ${line:15} >> $TO
done < $FROM

Answer 4

you need erase first bases of file fasta and qual for analysis, while i find a solution with QIIME, a solution using python and biopython:

from Bio import SeqIO

file_fasta = open("test.fasta")
file_qual = open("test.qual")

iterator_fasta = SeqIO.parse(file_fasta, "fasta")
iterator_qual = SeqIO.parse(file_qual, "qual")

size_trim = 15

output_fasta = open("trim.fasta","w")
for seq in iterator_fasta:
  if len(seq) <= size_trim:
    raise NameError('len seq less or equal than trim size')
  seq.seq = seq.seq[size_trim:]
  output_fasta.write(seq.format("fasta"))

output_fasta.close()

output_qual = open("trim.qual","w")
for seq_qual in iterator_qual:
  if len(seq_qual.letter_annotations['phred_quality']) <= size_trim:
    raise NameError('len qual less or equal than trim size')
  seq_qual.letter_annotations['phred_quality'] = seq_qual.letter_annotations['phred_quality']
  output_qual.write(seq_qual.format("qual"))

output_qual.close()

you get in trim.fasta

>SRR1502445.1
ACGAGTGCGTTCCTGCGCTTATTGATATGCTTAAGTTCAGCGGGTAGTCTCACCCGATTT
GAGGTCAAGGTTTGTGGGTCGAGTCACAACTCGAACATCGTCTTTGTACAAAGACGGTTG
GAAGCGGGTTCCAAGGCAACACAGGGGGATAGGNNNNNNNNNNNNNNNNNNNNNNN
>SRR1502445.2
ACGAGTGCGTTCCTGCGCTTATTGATATGCTTAAGTTCAGCGGGTAGTCTCACCCGATTT
GAGGTCAAGGTTTGTGGGTCGAGTCACAACTCGAACATCGTCTTTGTACAAGACGGTTGG
AAGCGGGTTCCAAGGCACACAGGGGATAGGNNN
>SRR1502445.3
ACGAGTGCGTTCCTGCGCTTATTGATATGCTTAAGTTCAGCGGGTAGTCTCACCCGATTT
GAGGTCAAGGTTTGTGGGTCGAGTCACAACTCGAACATCGTCTTTGTACAAAGACGGTTG
GAAGCGGGTTCCAAGGCACACAGGGGATAGGNNN
>SRR1502445.4
ACGAGTGCGTTCCTGCGCTTATTGATATGCTTAAGTTCAGCGGGTAGTCTCACCCGATTT
GAGGTCAAGGTTTGTGGGTCGAGTCACAACTCGAACATCGTCTTTGTACAAAGACGGTTG
GAAGCGGGTTCCAAGGCACACAGGGGATAGGNNNNNNNNNNN

EDIT:

using qiime, I recommend to use split_libraries, it does the trim and check the quality .... truncate_fasta_qual_files.py only select first B bases, trim last base doing otherwise to expected.

Answer 5

Use regular expressions and either perl or awk,

perl (write a script, and expand it to detect other regular expressions,

my $pattern=$ARGV[1]||"GACTACACGTAGT";
#provide any gene sequence prefix, and pattern removes that prefix
while (<>) {
    #explicit check for non-gene/header pattern
    if( $_ =~ /^[\>\;]/ ) {
        print $_;
    }
    #check for the specific header pattern provided, for example
    elsif( $_ =~ /^SRR1502445/ ) {
        print $_;
    }
    #check for the gene pattern given
    elsif( $_ =~ /^$pattern(.*)/ ) {
        print "$1\n";
    }
    else {
        print $_;
    }
}

perl -lane,

perl -lane 'if( $_ =~ /^GACTACACGTAGT(.*)/ ) {print "$1\n";} else {print $_; }'

awk,

/SRR1502445/ { print $0; }
/^GACTACACGTAGTAT/ { print substr($0,16); }

---

Works on any linux/unix box, and cygwin also.

---

The file format seems to be FASTA, which is described here FASTA Specification

Answer 6

A one-liner alternative to sed is awk.

Given an alternating-lined-element FASTA file called foo.fa, you can strip the first 15 characters of the sequence strings with substr():

$ awk '/^#/ {next} /^>/ { print $0 } /^[^>]/ { print substr($0, 16, length($0) - 15) }' foo.fa > foo.filtered.fa

As awk uses 1-based indexing, the start position argument in substr() is 16.

In addition to offering code to process alternating lines separately, another advantage of awk is that it can sometimes run faster than sed. Yet another advantage is portability, given differences in sed between common bioinformatics platforms.

So if you plan on doing this a lot or on "whole genome"-scale files, you might investigate this approach, as well.