2

While using Sed to search/ insert a config file, I'm greeted by errors. What's causing them, and how can I fix them?

The Heredoc I'm looking to insert can be defined as follows:

read -d '' APPLICATION_ENV_STATE <<'EOF'
defined('APPLICATION_ENV') || define('APPLICATION_ENV',(getenv('APPLICATION_ENV') 
    ? getenv('APPLICATION_ENV') : 'production'));
EOF

While my Sed command uses the variable like this:

sed -i "/\/\/ \*\* MySQL settings \*\* \/\//i$APPLICATION_ENV_STATE" wp-config.php

Which results in:

sed: -e expression #1, char 1: unknown command: `?'

In addition to an extra characters after command error.

However, the following Heredoc works, but results in some less than pretty formatting in my text file:

read -d '' APPLICATION_ENV_STATE <<'EOF'
defined('APPLICATION_ENV') || define('APPLICATION_ENV', (getenv('APPLICATION_ENV') ? getenv('APPLICATION_ENV') : 'production'));
EOF

How do I get the first example to work?

chrisRidgers
  • 23
  • 4

2 Answers2

1

AIUI, it's not the heredoc that's the problem, it's understanding which process is doing what at various times.

In your script that runs the sed command, Bash is substituting the variable before sed even sees it. Being a multi-line string, it would need escaping for sed). From the man page for sed, under the i command:

i \     Insert text, which has each embedded newline preceded by a back-slash.

Personally, I'd recommend using cat or echo if you can (or a scriping language like Python / Ruby / PHP), having broken the template up into atomic elements, so you can simply concatenate the relevant pieces together.

If you do want to continue with the current method though, you'll at least need to replace the newlines with backslashed newlines - try something like:

echo $APPLICATION_ENV_STATE | sed 's/$/\\/'
declension
  • 4,110
  • 22
  • 25
  • That's a brilliant answer thankyou. From that, I got the result i was looking for by modifying my variable/ Heredoc to: `code` defined('APPLICATION_ENV') || define('APPLICATION_ENV', (getenv('APPLICATION_ENV') \\ ? getenv('APPLICATION_ENV') : 'production')); EOF `code` Which worked great with my original Sed command. In this particular example I'm working with config file generated as part of a Wordpress install, it would be more work to separate the sections out into separate variables. Thanks again for that answer :) – chrisRidgers Jun 08 '15 at 10:22
  • Tried to upvote but couldn't and tried to format my code example, and it rejected my changes while I was trying to markdown it. Thanks again though. – chrisRidgers Jun 08 '15 at 10:31
  • Glad it helped. If it solved your question, please mark it as the accepted answer (no real need to upvote, which you probably can't do yet being a new user) – declension Jun 08 '15 at 10:32
1

You're using the wrong tool. The only constructs you should be using in sed are s, g, and p (with -n). Just use awk and avoid all the quoting/escaping nonsense:

$ cat file
foo
// ** MySQL settings ** //
bar

$ awk -v app="defined('APPLICATION_ENV') || define('APPLICATION_ENV',(getenv('APPLICATION_ENV')
    ? getenv('APPLICATION_ENV') : 'production'));" '
{print} index($0,"// ** MySQL settings ** //"){print app}' file
foo
// ** MySQL settings ** //
defined('APPLICATION_ENV') || define('APPLICATION_ENV',(getenv('APPLICATION_ENV')
    ? getenv('APPLICATION_ENV') : 'production'));
bar

Notice that you don't need to escape the RE metachars in the string you want to search for because awk can treat a string as a string and you don't need to escape newlines in the string you're adding and you don't need a here doc with a shell variable, etc.

Your read/sed command as written would fail for various character combinations in your search and/or replacement strings - see Is it possible to escape regex metacharacters reliably with sed for how to robustly search/replace "strings" in sed but then just use awk so you don't have to worry about any of it.

Community
  • 1
  • 1
Ed Morton
  • 188,023
  • 17
  • 78
  • 185
  • That's a strong statement, Why should those be the only Sed features I use? awk looks good as well though. – chrisRidgers Jun 09 '15 at 10:58
  • I've been using sed for 30+ years and awk for about 20 years and all of the other sed constructs are for functionality that awk constructs can do more briefly, simply, robustly, efficiently, extensibly, consistently, etc. sed is an excellent tool for simple substitutions on individual lines but all of those other arcane single-character sed language constructs that you need to arrange in unlikely configurations to have one-time impact on text became obsolete in the mid-1970s when awk was invented. See http://awk.info/?doc/tip/sedInAwk.html for some examples. – Ed Morton Jun 09 '15 at 12:26
  • I'll take a look at that. One of the things I've been trying to do is keep my scripts as tidy as possible, and to me that means storing these strings to insert into config files at the top of the file well away from script's logic. You mentioned that my read command would also likely fail with certain string patterns? How would you go about getting a multi line variable stored like that? (I originally went with `read -d ''` after seeing another stack overflow answer somewhere). – chrisRidgers Jun 09 '15 at 13:11
  • Any time you write a read command in shell you should use `IFS= read -r var` unless you have a specific reason to exclude the `IFS=` or the `-r` and fully understand the effects. Try your current method if the string you want to read starts with white space or contains a backslash and you'll find it strips the leading white space and the backslash. Shell is not intended for manipulating text so it's extremely difficult to do it robustly. The guys who wrote shell also wrote awk for precisely the purpose of manipulating text. I'm not sure what advice to give without an example - new question? – Ed Morton Jun 09 '15 at 13:27
  • Well the example is already there in my question; I read a heredoc into a variable. You mentioned though that the read/sed command would likely fail under certain conditions. I'll have to look up the read command and that IFS variable (which I've used before but it's purpose currently eludes me). – chrisRidgers Jun 09 '15 at 14:16
  • That is demonstrating the XY problem (http://meta.stackexchange.com/questions/66377/what-is-the-xy-problem). You're asking how to implement a solution to a problem instead of how to solve a problem. I'd need to see what the actual problem is before advising you as it's unlikely that creating a bunch of shell variables to use later is the best solution but if it is then how you are using them would be important. – Ed Morton Jun 09 '15 at 14:21
  • I think in response to that I'd argue that my problem was that my sed command was failing; i knew this and was looking for the correct syntax which is what I asked. From there several alternative methods of completing a similar task have been proposed, each with their own merits. Which is a nice bonus, but not my original question. – chrisRidgers Jun 11 '15 at 22:05