Using sed to split a file by a sequence. bash

Question

I want to cut a 211,548,559 lines file into 10 smaller files. So, the first file, for example will have 1st to 21154856th line

I would like to write a for loop with a seq that allows me to automatize the process.

I tried to create a function first and then a loop with seq.

run_sed(){
    sed -n $1p Bar08_depth_chr1.txt > Bar8_d_c1_$1.txt
}
for pos in seq 1 10 211548559
do
    run_sed $pos
done

This script didn't work. I believe its because the $1 in sed -n 1$p But I don't know how to solve it

do yeah mean manually split the 10 files? Yeah, I can do that, but I have other files I need to do it in 100 or even more. — Cebs, May 26 '19 at 17:29
No, he was talking about the command `split`. You can read its manual by running the command `man split`. — alfunx, May 26 '19 at 17:33

Dennis Williamson · Accepted Answer · 2019-05-27T12:34:39.153

2

For GNU split:

split -nl/10 --additional-suffix=.txt -d Bar08_depth_chr1.txt Bar8_d_c1_

Which will create 10 files named Bar8_d_c1_00.txt through Bar8_d_c1_09.txt which will likely not need to be renamed.

For split under MacOS:

split -l $(( (211548559 - 9) / 10 )) Bar08_depth_chr1.txt Bar8_d_c1_

Which will create 10 files named Bar8_d_c1_aa through Bar8_d_c1_aj which can be renamed to the name pattern you need.

The calculation shown causes the number of lines per file to be rounded up in order to avoid a very small 11th file.

edited May 27 '19 at 12:34

answered May 26 '19 at 21:24

Dennis Williamson

Why not just `split -nl/10`? – vintnes May 27 '19 at 07:37
@vintnes: The version of `split` that I was looking at doesn't support that. I'll add that version to my answer. Thanks. – Dennis Williamson May 27 '19 at 12:30

1 Answers1