6

From this question, I found the split utilty, which takes a file and splits it into evenly sized chunks. By default, it outputs these chunks to new files, but I'd like to get it to output them to stdout, separated by a newline (or an arbitrary delimiter). Is this possible?

I tried cat testfile.txt | split -b 128 - /dev/stdout

which fails with the error split: /dev/stdoutaa: Permission denied.

Looking at the help text, it seems this tells split to use /dev/stdout as a prefix for the filename, not to write to /dev/stdout itself. It does not indicate any option to write directly to a single file with a delimiter. Is there a way I can trick split into doing this, or is there a different utility that accomplishes the behavior I want?

ecapstone
  • 269
  • 5
  • 14

5 Answers5

7

It's not clear exactly what you want to do, but perhaps the --filter option to split will help out:

   --filter=COMMAND
          write to shell COMMAND; file name is $FILE

Maybe you can use that directly. For example, this will read a file 10 bytes at a time, passing each chunk through the tr command:

split -b 10 --filter "tr [:lower:] [:upper:]" afile

If you really want to emit a stream on stdout that has separators between chunks, you could do something like:

split -b 10 --filter 'dd 2> /dev/null; echo ---sep---' afile

If afile is a file in my current directory that looks like:

the quick brown fox jumped over the lazy dog.

Then the above command will result in:

the quick ---sep---
brown fox ---sep---
jumped ove---sep---
r the lazy---sep---
 dog.
---sep---
larsks
  • 277,717
  • 41
  • 399
  • 399
  • The last example is the behavior I was going for. Though this works, I eventually found it easier to allow split to produce new files as it normally does, then loop through them, then delete them. – ecapstone Aug 01 '16 at 15:28
3

From info page : `--filter=COMMAND' With this option, rather than simply writing to each output file, write through a pipe to the specified shell COMMAND for each output file. COMMAND should use the $FILE environment variable, which is set to a different output file name for each invocation of the command.

split -b 128 --filter='cat ; echo ' inputfile
Sam Daniel
  • 1,800
  • 12
  • 22
  • Split stdout: | split -b 64 --filter="cat;echo" - 'cat' is to print the split chunk and 'echo' is to add a new line. – clarkttfu Dec 02 '20 at 10:01
1

Here is one way of doing it. You will get each 128 character into variable "var".

You may use your preferred delimiter to print or use it for further processing.

#!/bin/bash

cat yourTextFile | while read -r -n 128 var  ; do
    printf "\n$var"
done

You may use it as below at command line:

while read -r -n 128 var  ; do printf "\n$var" ; done < yourTextFile
Robert Ranjan
  • 1,538
  • 3
  • 19
  • 17
0

No, the utility will not write anything to standard output. The standard specification of it says specifically that standard output in not used.

If you used split, you would need to concatenate the created files, inserting a delimiter in between them.

If you just want to insert a delimiter every N th line, you may use GNU sed:

$ sed '0~3a\-----\' file

This inserts a line containing ----- every 3rd line.

Kusalananda
  • 14,885
  • 3
  • 41
  • 52
0

To divide the file into chunks, separated by newlines, and write to stdout, use fold: 

cat yourfile.txt | fold -w 128

...will write to stdout in "chunks" of 128 chars.

fear_no_eval
  • 1
  • 1