Using sed to add text after a pattern, but the added text comes from a list file

Question

How can I use sed to locate a string, and add text from another file after the string?

File 1:

stage ('Clone Repo31') {

        steps {
                git credentialsId: '', url: '/stash/scm/'
        }
    }
    stage ('Zip Repo31') {
        steps {
        sh"""
            tar --exclude='*.tar' -cvf .tar *
        """
        }
    }
    steps {
            git credentialsId: '', url: '/stash/scm/'
    }
}
stage ('Zip Repo32') {
    steps {
    sh"""
        tar --exclude='*.tar' -cvf .tar *
    """
    }
}

File 2:

randomRepo.git
differentRandomRepo.git

I want to be able to use sed to read the second file, and add the contents of each line from the second file after each occurance of stash/scm/

Desired output:

       stage ('Clone Repo31') {

        steps {
                git credentialsId: '', url: '/stash/scm/randomRepo.git'
        }
    }
    stage ('Zip Repo31') {
        steps {
        sh"""
            tar --exclude='*.tar' -cvf .tar *
        """
        }
    }
    steps {
            git credentialsId: '', url: '/stash/scm/differentRandomRepo.git'
    }
}
stage ('Zip Repo32') {
    steps {
    sh"""
        tar --exclude='*.tar' -cvf .tar *
    """
    }
}

Can this be done with sed? I'm having issues reading it from a list file and it's confusing since it has a lot of slashes in it. I've been able to use normal sed substitution but I don't know how to do substitution by reading another file.

Answer 1

In the following I present an almost pure sed solution.

sed has an r command to read files, so you could in principle use that to read the file2. However, no subsequent command will affect the lines read from the file, so I cannot think of any way of using the r command effectively to do what you ask.

However, a solution is possible if file1 and file2 are both given in input to sed.

In the following, in order to distinguish the two files, I put a marker line (-----) that I give for granted is not in file2; it could be anywhere in file1 without creating any problems, however.

cat file2 <(echo '-----') file1 | sed -f script.sed

where script.sed is the following:

1{                     # only on line 1
  :a                   # begin while
  /-----/!{            # while the line does not contain the marker
    N                  # append the following line
    ba                 # end while
  }                    # here the pattern space is a multiline containing the list
  s/\n-----//          # remove the last newline and the marker
  h                    # put the multiline in the hold space
  d                    # delete, as we don't want to print anything so far
}                      # that's it, lines from 1 to the marker are processed
/stash\/scm\//{        # for lines matching this pattern
  G                    # we append the full hold space
  s/'\n\([^\n]*\)/\1'/ # and position the first entry in the list appropriately
  x                    # then we swap pattern and hold space
  s/[^\n]*\n//         # remove the first element of the list
  x                    # and swap again
}                      # now the hold space has one item less

Answer 2

This is a bash script that uses sed and reads File_2 (The file containing the replacements) line by line, thus reading one replacement at a time. I then replaced the lines in File_1 with a sed script.

while IFS= read -r line; do
    sed -i "0,/\/stash\/scm\/'/{s|/stash/scm/'|/stash/scm/${line}'|}" File_1.txt
done < File_2.txt

Some tricks used to do this:

1. sed '0,/Apple/{s/Apple/Banana/}' input_filename Replace only the first occurrence in filename of the string Apple with the string Banana
2. Using double quotes for the sed script to allow for variable expansion ${line}
3. Making sure the search string to replace was being changed each iteration. This was done by including the ending single quote char ' for the search argument in the sed script s|/stash/scm/'|
4. Reading a file line by line in a bash script

while IFS= read -r line; do
    echo $line
done < File_2.txt

Read File line by line in bash

Answer 3

You want to have lines like

sed 's#'/stash/scm/'#&something_from_file2#' file1 

You can make these lines with

    # Note:
    # / is not a delimiter, but part of the path
    # % is the delimiter in the current sed-command
    # # is the delimiter in the generated command.

sed 's%.*%s#/stash/scm/#\&&#%' file2

You can generate these commands on the fly and execute them on file1.

sed -f <(sed 's%.*%s#/stash/scm/#\&&#%' file2) file1

One problem left. Both commands will substitute all matches.
I will use the single quote given after the match. When something is put before the single quote in /stash/scm/' this is different than the match files when you look for the /stash/scm/' string including the quote.
You want to generate lines like

s#(/stash/scm/)(')#\1randomRepo.git\2#
s#(/stash/scm/)(')#\1differentRandomRepo.git\2#

Each substition should be done only once, so we consider file2 as one long line using the option -z:

sed -rzf <(sed 's%.*%s#(/stash/scm/)('\'')#\\1&\\2#%' file2) file1