How can I prepend a string to the beginning of each line in a file?

Question

I have the following bash code which loops through a text file, line by line .. im trying to prefix the work 'prefix' to each line but instead am getting this error:

rob@laptop:~/Desktop$ ./appendToFile.sh stusers.txt kp
stusers.txt
kp
./appendToFile.sh: line 11: /bin/sed: Argument list too long
115000_210org@house.com,passw0rd

This is the bash script ..

#!/bin/bash

file=$1
string=$2

echo "$file"
echo "$string"

for line in `cat $file`
do
    sed -e 's/^/prefix/' $line
    echo "$line"
done < $file

What am i doing wrong here?

Update: Performing head on file dumps all the lines onto a single line of the terminal, probably related?

rob@laptop:~/Desktop$ head stusers.txt
rob@laptop:~/Desktop$ ouse.com,passw0rd

You probably have invalid line endings in the file. Perhaps you want `dos2unix` — William Pursell, Nov 27 '12 at 14:34
@WilliamPursell i tried running dos2unix against the file but did not insert the new lines for me, easiest solution was copy file contents into a new file — bobbyrne01, Nov 27 '12 at 15:10
[DontReadLinesWithFor](https://mywiki.wooledge.org/DontReadLinesWithFor) is _deeply_ on-point. `for line in` is misleading, because it's not actually lines that are iterated over. — Charles Duffy, Dec 19 '22 at 20:30

score 77 · Accepted Answer · answered Nov 27 '12 at 15:27

77

a one-line awk command should do the trick also:

awk '{print "prefix" $0}' file

answered Nov 27 '12 at 15:27

nullrevolution

3,937
1
18
20

score 30 · Answer 2 · edited Jul 29 '19 at 02:01

Concerning your original error:

./appendToFile.sh: line 11: /bin/sed: Argument list too long

The problem is with this line of code:

sed -e 's/^/prefix/' $line

$line in this context is file name that sed is running against. To correct your code you should fix this line as such:

echo $line | sed -e 's/^/prefix/'

(Also note that your original code should not have the < $file at the end.)

William Pursell addresses this issue correctly in both of his suggestions.

However, I believe you have correctly identified that there is an issue with your original text file. dos2unix will not correct this issue, as it only strips the carriage returns Windows sticks on the end of lines. (However, if you are attempting to read a Linux file in Windows, you would get a mammoth line with no returns.)

Assuming that it is not an issue with the end of line characters in your text file, William Pursell's, Andy Lester's, or nullrevolution's answers will work.

A variation on the while read... suggestion:

while read -r line; do  echo "PREFIX " $line; done < $file

This could be run directly from the shell (no need for a batch / script file):

while read -r line; do echo "kp" $line; done < stusers.txt

1) turn off field splitting so you don't strip leading and trailins blanks, and 2) quote your variables for MANY reasons: `while IFS= read -r line; do echo "kp" "$line"; done < stusers.txt`. You could also argue for using print instead of echo. — Ed Morton, Nov 29 '12 at 04:33

score 15 · Answer 3 · answered Nov 29 '13 at 15:40

15

The entire loop can be replaced by a single sed command that operates on the entire file:

sed -e 's/^/prefix/' $file

answered Nov 29 '13 at 15:40

Jack

5,801
1
15
20

2

Is it also possible to check wether the line is empty or not? – Julian F. Weinert Apr 03 '15 at 06:26
Do you want to leave empty lines empty? – Jack Jun 10 '16 at 15:10
2

It's so long ago... It would be nive to know both options – Julian F. Weinert Jun 10 '16 at 23:11
The simplest way is to add another sed command to replace all lines with just the prefix with blank lines: sed -e 's/^/prefix/' -e 's/^prefix$//' $file – Jack May 18 '17 at 01:53
@JulianF.Weinert : sed -e 's/^$.$/prefix \1/' file – Julien Sep 01 '17 at 21:46

score 5 · Answer 4 · answered Nov 27 '12 at 14:47

A Perl way to do it would be:

perl -p -e's/^/prefix' filename

or

perl -p -e'$_ = "prefix $_"' filename

In either case, that reads from filename and prints the prefixed lines to STDOUT.

If you add a -i flag, then Perl will modify the file in place. You can also specify multiple filenames and Perl will magically do all of them.

score 4 · Answer 5 · answered Nov 27 '12 at 14:29

4

Instead of the for loop, it is more appropriate to use while read...:

while read -r line; do
do
    echo "$line" | sed -e 's/^/prefix/'
done < $file

But you would be much better off with the simpler:

sed -e 's/^/prefix/' $file

answered Nov 27 '12 at 14:29

William Pursell

204,365
48
270
300

the first code block did not update the file with the new prefixes and the second block only updated the first line of the file – bobbyrne01 Nov 27 '12 at 15:36

score 2 · Answer 6 · answered Aug 05 '16 at 17:04

2

Use sed. Just change the word prefix.

sed -e 's/^/prefix/' file.ext

If you want to save the output in another file

sed -e 's/^/prefix/' file.ext > file_new.ext

answered Aug 05 '16 at 17:04

Ahmad Awais

33,440
5
74
56

score 0 · Answer 7 · answered Nov 27 '12 at 16:04

0

You don't need sed, just concatenate the strings in the echo command

while IFS= read -r line; do
    echo "prefix$line"
done < filename

Your loop iterates over each word in the file:

for line in `cat file`; ...

answered Nov 27 '12 at 16:04

glenn jackman

238,783
38
220
352

score 0 · Answer 8 · answered Nov 13 '21 at 00:15

0

sed -i '1a\
Your Text' file1 file2 file3

answered Nov 13 '21 at 00:15

user1034533

1,054
9
9

The question specifies "beginning _of each line_", not just beginning of the file. – Charles Duffy Dec 19 '22 at 20:35

reddot · Answer 9 · 2022-12-19T19:57:02.533

0

A solution without sed/awk and while loops:

xargs -n1 printf "$prefix%s\n" < "$file"

edited Dec 19 '22 at 19:57

answered Dec 19 '22 at 16:45

reddot

764
7
15

`xargs -n1` is _wildly_ inefficient; you're copying a whole new copy of `/usr/bin/printf` for every line of the file; and substituting the data _into the format string_ introduces its own bugs in addition. – Charles Duffy Dec 19 '22 at 20:30
A less incorrect way to do this (with the caveat that it requires GNU xargs) would be `xargs -d $'\n' printf '%s%s\n' "$prefix"` – Charles Duffy Dec 19 '22 at 20:32
...to support BSD xargs as well, perhaps `tr '\n' '\0' <"$file" | xargs -0 printf '%s%s\n' "$prefix"` – Charles Duffy Dec 19 '22 at 20:32
Note that both of those have a bug insofar as they can print a line with just the prefix and nothing else when input is empty -- but then, your original answer had that bug too; it's solvable with a GNU extension, but not portably. – Charles Duffy Dec 19 '22 at 20:33

How can I prepend a string to the beginning of each line in a file?

9 Answers9

Linked

Related