1

I am trying to parse some text files with the command line. part of this involves reattaching broken words in some badly-formatted emails. An example:

9,650 330,765.0  16.38% NYSE (000) 1,707,915 272,099.0  18.95%     Commodit=
ies Close Change % Change Crude Oil (Feb) 19.62 0.32  1.66% Heating Oil (Ja=

I want to grab 'Commodities.' I'm using this workaround to sed to get the job done.

I'm using Mac OS X 10.7 and GNU sed version 4.2.1. If at the command line I enter

sed ':a;N;$!ba;s/=\r\n//g' ./filename

sed works correctly. However if I run this bash script:

#!/bin/bash


sed ':a;N;$!ba;s/=\r\n//g' filename

sed doesn't work. However, the same script works under Ubuntu's command line:

9,650 330,765.0  16.38% NYSE (000) 1,707,915 272,099.0  18.95%     Commodities Close Change % Change Crude Oil (Feb) 19.62 0.32  1.66% Heating Oil (Jan)

On my Mac, the simpler script

#!/bin/bash


sed 's/=//g' filename

successfully removes all the equal signs. I'm trying different combinations of characters to backslash out but without much success. Any hints to what the Mac terminal isn't liking?

Community
  • 1
  • 1
RussH
  • 329
  • 2
  • 17
  • The example file contains a single space after the last equal sign on each line. Could that be the reason your command is failing? Also, perhaps try changing your shebang to `#!/bin/sh` for it to work on OSX. Goodluck. – Steve Jan 09 '13 at 07:32
  • @steve thank you for your suggestion. Unfortunately changing /bin/bash to /bin/sh didn't work. :( I updated the example file to remove the spaces, they are not in the actual file. – RussH Jan 09 '13 at 07:39

1 Answers1

0

It's most likely a PATH setting. /bin/bash uses the default $PATH; not sure why, but perhaps that depends on your normal working shell (is that bash), or in which dot-files your PATH settings are.

OS X comes with its own (BSD) sed, which is not the same as the GNU one, and thus doesn't work. Running the sed command in the script will pick up the BSD sed, not your self-installed GNU sed. Use the full path to sed in the script, or set $PATH in your script before. Obviously, you don't have the problem on Ubuntu, since the default sed is GNU.

  • well, I put 'gsed' instead of 'sed' in my script and that seems to work. I must have it installed on MacPorts or something. You're right, the sed called by the script is different. Curiously, when i put `echo $PATH` in the script I was shown the same PATH I got from entering the same command on the command line. – RussH Jan 09 '13 at 07:55
  • 1
    If you can call `sed` form the command line but not from the script, and your `PATH` is the same, perhaps it's an alias that's not recognised/expanded in the script? –  Jan 10 '13 at 09:05
  • Thanks. I looked again, i have alias sed='gsed' in my .bashrc file. I tried moving it to my .bash_profile and .profile files as per http://stackoverflow.com/questions/4477328/bashrc-not-read-when-shell-script-is-invoked-from-desktop-shortcut but my alias still isn't recognized in the script... Still trying to figure it out. – RussH Jan 10 '13 at 21:37
  • Aliases are not expanded in non-interactive shells (such as your script). See e.g. http://unix.stackexchange.com/questions/1496/why-doesnt-my-bash-script-recognize-aliases. –  Jan 11 '13 at 00:36