2

I understand I can:

ssh archive_server -l user -n "cat text.csv"|tee -a text1.csv|tee -a text2.csv|tee....|tee -a text10.csv

Is there a way to do it a a loop?

for i in `seq 1 10`; do
  echo $i
  tee ???
done
olekb
  • 638
  • 1
  • 9
  • 28
  • Do you just want to duplicate `text.csv` multiple times? – hek2mgl Feb 27 '17 at 20:40
  • 1
    why not `cp file copy$i`in the loop? – karakfa Feb 27 '17 at 20:55
  • BTW, for cases unlike `tee` where you *really do* need to create a dynamically determined number of pipeline elements, see [Handling long edit lists in XMLStarlet](http://stackoverflow.com/questions/9898939/handling-long-edit-lists-in-xmlstarlet) – Charles Duffy Feb 27 '17 at 21:12
  • similarly, were it not for `tee` being a special case, this would arguably be an outright duplicate of [how do I code an arbitrarily long chain of pipes?](http://stackoverflow.com/questions/25212663/how-do-i-code-an-arbitrarily-long-chain-of-pipes) – Charles Duffy Feb 27 '17 at 21:13
  • @hek2mgl modified question. I ssh data from archive server. – olekb Feb 27 '17 at 21:15
  • I would simply scp(!) the file from the remote server and then duplicate it via `cp` on the local machine. Is that too simple? :) – hek2mgl Feb 27 '17 at 21:19
  • @hek2mgl I write it into a pipe and load to database. no time to wait - file is too large – olekb Feb 27 '17 at 21:23
  • But why do you need to duplicate the file? – hek2mgl Feb 27 '17 at 21:44

2 Answers2

4

Assuming your shell is really bash (not /bin/sh), you can build an array (and use a C-style for loop, which unlike the nonstandard external command seq, is guaranteed to be available everywhere bash is):

#!/bin/bash
filenames=( )
for ((i=1; i<=10; i++)); do    # note that the "10" here can be a variable name
  filenames+=( "file$i.txt" )
done
tee -a -- "${filenames[@]}" <text.csv

If you need compatibility with /bin/sh, it gets a little bit more verbose:

#!/bin/sh
tee_between() (
  prefix=$1; suffix=$2; start=$3; end=$4
  set --
  i=$(( $start - 1 )); while [ $(( ( i += 1 ) <= end )) -ne 0 ]; do
    set -- "$@" "file$i.txt"
  done
  exec tee -a -- "$@"
)

tee_between "file" ".txt" 1 10 <text.csv

Note:

  • set -- modifies the current process's (or, in this case, the current function's) argument list, using that as an array we can dynamically modify.
  • tee_between() ( ) instead of tee_between() { } means that the function runs in a subshell -- a completely separate forked-off process. In consequence of this choice, the exec command will replace only that subshell with a copy of tee, and not the parent process.
Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
3

You don't need a loop. tee can be given multiple filename arguments, so just give all the output files at once:

cat text.csv | tee -a text{1..10}.csv

If the number of files is dynamic, you can use a loop in $() to insert the filenames into the command line:

cat text.csv | tee -a $(
    for i in $(seq 1 $filecount); do
        echo text$i;
    done)

Just make sure that you don't have any whitespace in the output filename prefix, as the spaces will be treated as argument delimiters.

Barmar
  • 741,623
  • 53
  • 500
  • 612
  • @hek2mgl True, but that's what the question has. – Barmar Feb 27 '17 at 20:41
  • At least in the headline it says a *variable* number, which would change the game a bit. Probably I'm too pedantic.... – hek2mgl Feb 27 '17 at 20:42
  • OK, added that. – Barmar Feb 27 '17 at 20:43
  • 2
    The dynamic version is fragile with respect to filenames containing whitespace and such; I'd build an array. – chepner Feb 27 '17 at 20:45
  • @chepner The output filename isn't dynamic, there is no whitespace. – Barmar Feb 27 '17 at 20:54
  • 2
    @Barmar, sure, but we're providing a learning resource here. Something that's only safe when conditions A, B and C are met needs to be called out as such explicitly, or it'll be reused outside of that set of circumstances. By contrast, teaching something that's unambiguously safe moots that obligation. – Charles Duffy Feb 27 '17 at 20:56
  • OTOH, teaching people that whitespace has no place in a file name is timeless. – William Pursell Feb 27 '17 at 20:57
  • @CharlesDuffy Sure, but sometimes you can overengineer things. If you control the input, you can easily ensure that it doesn't run into weird cases that you need to work around. – Barmar Feb 27 '17 at 20:57
  • 1
    @Barmar, granted that overengineering is a thing -- but at bare minimum, if you're using something that has known limitations, those known limitations need to be disclosed, or you're leaving software with hidden pitfalls to those who don't know the language and toolset well enough to recognize them (a set of folks which is overwhelmingly likely to overlap with our audience here). – Charles Duffy Feb 27 '17 at 20:58
  • OK, I've added the cavest. – Barmar Feb 27 '17 at 21:00
  • @Barmar It does not work with pipes. if text{1..10}.csv are pipes the process that reads from them is stuck (gets no data). But as files they work – olekb Feb 27 '17 at 21:39
  • 1
    @olekb, it certainly will work with pipes, but only if there's a reader attached to all 10 of them. Since `tee` doesn't buffer (it's documented not to!), then if any of those pipelines doesn't have an active reader attached, that blocks everything else. – Charles Duffy Feb 27 '17 at 21:45
  • 2
    @olekb, ...that said, I'm starting to wonder if your *real* problem is one best answered in a manner involving [process substitution](http://wiki.bash-hackers.org/syntax/expansion/proc_subst)... or putting [`buffer`](https://linux.die.net/man/1/buffer) in place on the other end of each of your pipelines. – Charles Duffy Feb 27 '17 at 21:46