45

I am looking for a bash or sed script (preferably a one-liner) with which I can insert a new line character after a fixed number of characters in huge text file.

rangalo
  • 5,448
  • 8
  • 46
  • 66
  • Dupe of http://stackoverflow.com/questions/525592/find-and-replace-inside-a-text-file-from-a-bash-command among many, many others –  Jul 27 '09 at 08:36
  • 8
    I disagree by the dupe comment here, Neil: this is not a simple search and replace like the question in your link.. – Kristian Jul 27 '09 at 09:18

6 Answers6

62

How about something like this? Change 20 is the number of characters before the newline, and temp.text is the file to replace in..

sed -e "s/.\{20\}/&\n/g" < temp.txt
Kristian
  • 6,357
  • 4
  • 36
  • 37
  • 2
    Fails if the file contains a '~' character – William Pursell Jul 27 '09 at 10:05
  • 1
    This inserts a newline after *every* 20 characters (per line of the original). If the original contains no newlines and you want a newline after only the first 20 characters, leave out the "g" (global) at the end. If you want this and the original contains newlines, you'll have to use a different solution. – Dennis Williamson Jul 27 '09 at 10:45
  • I had the same problem but in OSX it inserts an "n" rather than a line break? I checked other posts with the same problem but I could not understand how to fix it? Anyone please? – JM88 Feb 26 '14 at 17:21
  • @JM88, Unix/Linux use the line-feed character (`\n`) for line breaks. Mac uses the carriage-return character for line breaks (`\r`), and Windows uses a combination of the two (`\r\n`) for line breaks. Change the command to `sed -e "s/.\{20\}/&\r/g" < temp.txt`, and you should get what you need. See also [this Stack-O](http://stackoverflow.com/questions/7013034/does-windows-carriage-return-r-n-consist-of-two-characters-or-one-character) and [this blog](http://www.danielmiessler.com/study/crlf/) about it. – David May 19 '14 at 14:04
  • A lot of sed implementations limit the number of selected characters to 255. Prefer the Steven Penny's response. – Benoît Sauvère Aug 14 '14 at 17:19
  • Does not work on OSX with a file.containing a base64 encoded string. –  May 04 '15 at 13:24
  • I think the use of `fold` in is more elegant. – laindir Dec 29 '15 at 16:04
3

Let N be a shell variable representing the count of characters after which you want a newline. If you want to continue the count accross lines:

perl -0xff -pe 's/(.{'$N'})/$1\n/sg' input

If you want to restart the count for each line, omit the -0xff argument.

William Pursell
  • 204,365
  • 48
  • 270
  • 300
2

Because I can't comment directly (to less reputations) a new hint to upper comments:

I prefer the sed command (exactly what I want) and also tested the Posix-Command fold. But there is a little difference between both commands for the original problem: If you have a flat file with n*bytes records (without any linefeed characters) and use the sed command (with bytes as number (20 in the answer of @Kristian)) you got n lines if you count with wc. If you use the fold command you only got n-1 lines with wc! This difference is sometimes important to know, if your input file doesn't contain any newline character, you got one after the last line with sed and got no one with fold

gmu
  • 74
  • 4
0

if you mean you want to insert your newline after a number of characters with respect to the whole file, eg after the 30th character in the whole file

gawk 'BEGIN{ FS=""; ch=30}
{
    for(i=1;i<=NF;i++){
        c+=1
        if (c==ch){
            print "" 
            c=0           
        }else{
            printf $i
        }
    }
    print ""
}' file

if you mean insert at specific number of characters in each line eg after every 5th character

gawk 'BEGIN{ FS=""; ch=5}
{
    print substr($0,1,ch) "\n" substr($0,ch)
}' file
ghostdog74
  • 327,991
  • 56
  • 259
  • 343
0

Append an empty line after a line with exactly 42 characters

sed -ie '/^.\{42\}$/a\
' huge_text_file
Chen Levy
  • 15,438
  • 17
  • 74
  • 92
  • 2
    I think sed is often is better without the `-i` flag when used on the command line as you won't accidentally overwrite your original file with unwanted output if you make a typo when writing your command. I feel the answer should at least warn people what `-i` does in case they are not familiar with sed and don't first check the flags they use. – Mehmet Karatay May 17 '20 at 07:40
0

This might work for you:

echo aaaaaaaaaaaaaaaaaaaax | sed 's/./&\n/20'
aaaaaaaaaaaaaaaaaaaa
x
potong
  • 55,640
  • 6
  • 51
  • 83
  • 1
    literally only works for the first line (eg inserts newline after 20 chars and then quits). Not suitable for "a huge text file". – michael Dec 30 '17 at 16:24