Bash script, taking values from a file and adding them to another file in order

Question

I'm trying to make a script that gets number values from a file (multiple lines) and adds said values to another file in order of appearance (if that makes sense). I'm pretty green when it comes to this. There's probably some way to use sed, or maybe even omit the file creation and simply make sed take the values from the results of grep?

Dunno if I can explain it better.

Basically I put the results of grep in a file for example the results would be: 101 102 104 108 111 etc now I want to add those results each single one to a specific line in another file for example:

external=200100
external=200100
external=200100

etc

the end result I want is:

external=200100101
external=200100102
external=200100104

Sorry if I'm not being clear enough about this, this is my first time dabbling in this and also I don't really know the proper terminology for everything.

Wanted to clarify that there's other stuff in between the "external=200100" values. So just copying 1 to 1 won't work. It needs to find every "external=200100" value and paste the results of grep at the end of each "external=200100" in proper order.

Clearer example maybe:

1 Results of grep:

100
104

2 Insides of the file I want to edit:

[100]
secret=blah
type=blah
dial=blah
external=200100

[104]
secret=blah
type=blah
dial=blah
external=200100

etc

3 End result

[100]
secret=blah
type=blah
dial=blah
external=200100100

[104]
secret=blah
type=blah
dial=blah
external=200100104

etc

Hope that's clear enough.

I tried some things with sed and awk but I don't really grasp all the options yet so nothing really worked. — Netami, Nov 26 '19 at 10:36
You can do this with plain Bash. Just read the file line by line. Split at `=` then concat the strings. — stephanmg, Nov 26 '19 at 10:39
It's not line by line, there's other stuff between the external= values — Netami, Nov 26 '19 at 10:41
I see, this is something you didn't state before. Then iterate over one file, do your `grep` stuff and merge. — stephanmg, Nov 26 '19 at 10:42

dash-o · Answer 1 · 2019-11-27T08:20:14.247

Restating the ASK: * Read list of values val-1, val-2, ... from one file * Read the second file, and append val[n] to the nth occurrence of a line starting with 'external='

The list of values is generated dynamically by some grep command.

I think awk should be able to do the work

awk '
!SUB {
   # Read values from file1, store in val array.
   val[i1++] = $0 ;
}

SUB {
  # Append value if needed
  if ( $0 ~ /^external=/ ) $0 = $0 val[i2++]
  print 
  next
}

' <(grep ...) SUB=1 input-file

Note: There are no details on the 'grep' command that provides input to awk. Most likely, it will be possible to integrate the grep into awk, and eliminate the extra step. Need OP input for that.

UPDATED 2019-11-27

Newer version (>=4.1) of GNU awk have (not POSIX standard) 'in-place' editing, as per Save modifications in place with awk

awk -i inlpace ...

I can not comment on using it, as I always usually learned the hard way that edit-in-place implementations are not friendly toward bugs and create difficult to debug setup. Up to you.

Option 2, which is usually safe,

awk 'ORIGINAL CODE HERE' < <(grep ...) SUB=1 input-file > new-file && mv input-file old-file && mv new-file input-file

Note the renaming is conditional on successful awk completion.

It's either read from file#2 and paste into file#1 or read from grep and paste to file#1 (no idea if possible to do it directly from grep, either way works). Also sorry where do I put the name of the file ? Never used this kind of command. — Netami, Nov 26 '19 at 12:07
Modified sample code to explicitly reference input-file. Also note command just generate the output to stdout. Do you need to replace original file ? — dash-o, Nov 26 '19 at 12:33
Uhm it should edit/replace the original file I guess. The one I want to have the stuff added from the other file. — Netami, Nov 26 '19 at 12:39

score 0 · Answer 2 · answered Nov 26 '19 at 12:43

0

In sed:

# recreate and pipe the input
cat <<EOF |
[100]
secret=blah
type=blah
dial=blah
external=200100

[104]
secret=blah
type=blah
dial=blah
external=200100
EOF

# the script
sed '
   # remember the part between braces
   /^\[\([0-9]*\)\]$/{
          # hold current line, we are going to change it
          h
      # remove from the line the [ ]
      s//\1/
      # switch hold and pattern space
            x
   }
   # add it to the end of the line we want to
   /^external=[0-9]*$/{
       # grab hold space
       G
       # remove the newline appended by sed when grabing
       s/\n//
   }
'

outputs:

[100]
secret=blah
type=blah
dial=blah
external=200100100

[104]
secret=blah
type=blah
dial=blah
external=200100104

A nicer oneliner:

sed '/^\[\([0-9]*\)\]$/{ h;s//\1/;x }; /^external=[0-9]*$/{ G;s/\n// }'

Tested on repl.

answered Nov 26 '19 at 12:43

KamilCuk

120,984
8
59
111

Hm not quite what I had in mind, inputting 100's of numbers doesn't seem like such a good idea. Also the output should be saved to file not just printed. Unless I'm missing something, which I probably am cause I'm pretty much clueless about this. – Netami Nov 26 '19 at 12:53
Please learn about shell redirections and how to save the output of a command to a file. [This looks like ok introduction](https://mywiki.wooledge.org/BashGuide/InputAndOutput#Redirection). If you have GNU `sed` implementation, it has `-i` switch to allow editing files in place. I don't understand the `inputting 100's of numbers doesn't seem like such a good` part, why do you want to input any numbers and where? – KamilCuk Nov 26 '19 at 12:56
Wait, maybe I completely misunderstood your question. How is the `Results of grep:` relevant to the file? Aren't they generated from the `[104]` lines? So the numebrs in `Results of grep:` differ from the numbers in stored in `[]` inside `Insides of the file I want to edit:` file? – KamilCuk Nov 26 '19 at 12:59
I'm confused by the "cat >>EOF" doesn't that ask for input from me? Actually results of grep are generated from a different line. This is all just an example the files I need to go through got dozens/hundreds of numbers that need changing. – Netami Nov 26 '19 at 13:11
[bash here documents](http://tldp.org/LDP/abs/html/here-docs.html). It's `cat << EOF |` - `<< EOF` starts a heredoc and `|` pipes the output to the next command. `are generated from a different line` - from which line? It would be way faster if you generate all the values with one command, rather then two. – KamilCuk Nov 26 '19 at 13:23
Sorry, I might be stupid but this is too much for me... Definitely need to read more in depth about it, unfortunately I'm strapped for time right now. Was just looking for something to save me hours of typing. Got the rest of the script working just got stuck at this one part, that's way above my current knowledge. – Netami Nov 26 '19 at 13:33

score 0 · Answer 3 · answered Nov 27 '19 at 04:25

If I understand correctly, how about:

#!/bin/bash

mapfile -t values < <(grep -oE "[0-9]+" "file1")
# now the array "values" holds (100 104 ..)

mv -f "file2" "file2.bak"
# make a backup of file2
while IFS= read -r line; do
    [[ $line =~ ^external= ]] && line+="${values[$((i++))]}"
    # if the line starts with the pattern "external=" then append
    # the element of array "values" in order
    echo "$line"
done < "file2.bak" > "file2"

As of now I do not have enough information about the contents of file1 and your grep command. Please tweak the grep line above accordingly.

My assumptions are:

file1:

foo=100
bar=104

file2 (before):

[100]
secret=blah
type=blah
dial=blah
external=200100

[104]
secret=blah blah
type=blah blah
dial=blah blah
external=200100

file2 (after):

[100]
secret=blah
type=blah
dial=blah
external=200100100

[104]
secret=blah blah
type=blah blah
dial=blah blah
external=200100104

Hope this helps.

Bash script, taking values from a file and adding them to another file in order

1 Results of grep:

2 Insides of the file I want to edit:

3 End result

3 Answers3