-5

I am an iOS developer, not familiar with Linux, I searched for a long time but still could not solve my problem. I hope someone can help me, thanks!

Before delete:

Other contents......

#
# Proxy server for iTerm & Terminal Setup Script
# version 0.1
# VincentSit
# Nov 26, 2014
#
function start_proxy {
 export http_proxy='abc.efgh.com:1234'
 export HTTPS_PROXY='abc.efgh.com:1234'
 }

function stop_proxy {
 export http_proxy=''
 export HTTPS_PROXY=''
 }

Other contents......

After delete:

Other contents......
Other contents......

UPDATE

I'm writing a shell script(can not use other languages.). On users system has a text file named abc, content is probably something like this:

# Other comments......
# Other comments......
# Other comments......
Other contents......
# Other comments......

#
# Proxy server for iTerm & Terminal Setup Script
# version 0.1
# VincentSit
# Nov 26, 2014
#
function start_proxy {
 export http_proxy='abc.efgh.com:1234'
 export HTTPS_PROXY='abc.efgh.com:1234'
 }

function stop_proxy {
 export http_proxy=''
 export HTTPS_PROXY=''
 }

# Other comments......
Other contents......
# Other comments......
# Other comments......

Which Other contents ...... is fictional, can not use it as deleted condition.

I just need to delete the following sections:

#
# Proxy server for iTerm & Terminal Setup Script
# version 0.1
# VincentSit
# Nov 26, 2014
#
function start_proxy {
 export http_proxy='abc.efgh.com:1234'
 export HTTPS_PROXY='abc.efgh.com:1234'
 }

function stop_proxy {
 export http_proxy=''
 export HTTPS_PROXY=''
 }

Do not need to delete other contents in this file.

Because it runs on the user's system, I do not know the line number of the contents I want to delete.

Finally look like this:

# Other comments......
# Other comments......
# Other comments......
Other contents......
# Other comments......

# Other comments......
Other contents......
# Other comments......
# Other comments......

I think the answer might look like this:

sed xxxxxx abc.txt

awk xxxxxx abc.txt

Vincent Sit
  • 2,214
  • 1
  • 24
  • 27
  • 2
    Could you explain the problem in a bit more detail? Otherwise I'd recommend plain `nano` or `vi`. – Joachim Isaksson Nov 27 '14 at 07:06
  • see this answer: http://stackoverflow.com/questions/6287755/using-sed-to-delete-all-lines-between-two-matching-patterns . – Arjun Mathew Dan Nov 27 '14 at 07:09
  • 4
    I gave you -1, since its not clear what to delete and how to search for it. You also have not posted what you have tried to solve this. – Jotne Nov 27 '14 at 07:27
  • need a bit more info becaause the content of *Others contents* could be altered via a generic sed/awk, ... . It will be interesting to define and assume some point like 'for the section Proxy server, remove the section coment, all the function after it using _proxy in name, until next comment out of a function...' – NeronLeVelu Nov 27 '14 at 07:29
  • Hello everyone, I added more description, thank you. – Vincent Sit Nov 27 '14 at 07:40
  • @jotne Could I ask you to cancel -1, I've added more detail. And nu11p01n73R solve my problem. Anyway, thank the help of everybody. – Vincent Sit Nov 27 '14 at 07:55
  • So you want to delete all lines starting with `#`, `function start_proxy` block and `function stop_proxy` block regardless of where its found in the file, and nothing else? – Jotne Nov 27 '14 at 08:16
  • @Jotne No, I just need to delete the six line beginning with `#` above the `function start_proxy`. – Vincent Sit Nov 27 '14 at 08:35
  • So how to know what `#` to delete? The 6 first it find? How do we know where in the file they are since you have other text above it??? – Jotne Nov 27 '14 at 08:39
  • So we need to search for `# Proxy server for iTerm & Terminal Setup Script`, then delete it, line above, and 4 below it?. We should not delete all blank lines, only those between `function start` `function stop`? – Jotne Nov 27 '14 at 08:48
  • @Vincent the answer you got will produce the output you expect from THAT specific input file. That does not mean it's a solution to your problem. If I were you I would NOT use that on a clients files given what you've told us so far about their content! You simply haven't told us enough about your problem for us to hep you solve your problem in a robust way. – Ed Morton Nov 27 '14 at 15:27

4 Answers4

1

You can delete unnecessary lines using the line number range.

Example:

sdlcb@Goofy-Gen:~/AMD/SO$ cat File
This is Line 1
This is Line 2
This is Line 3
This is Line 4
This is Line 5
This is Line 6
This is Line 7
sdlcb@Goofy-Gen:~/AMD/SO$ sed -i.bak '3,6d' File
sdlcb@Goofy-Gen:~/AMD/SO$ cat File
This is Line 1
This is Line 2
This is Line 7

Deletes 3rd to 6th lines.

Else:

sed -n '/^Other contents/,/^Other contents/{ /^Other contents/!d; /^Other contents/!d; p; }' File

Details:

-n option => do not print
/a/,/b/ => select lines between the lines matching /a/ and /b/ (inclusive)
/a/!d => delete lines not matching /a/
/b/!d => delete lines not matching /b/
in our case a and b = '^Other contents' which means beginning with 'Other contents'
p => print

Example:

sdlcb@Goofy-Gen:~/AMD/SO$ cat File
Other contents......

#
# Proxy server for iTerm & Terminal Setup Script
# version 0.1
# VincentSit
# Nov 26, 2014
#
function start_proxy {
 export http_proxy='abc.efgh.com:1234'
 export HTTPS_PROXY='abc.efgh.com:1234'
 }

function stop_proxy {
 export http_proxy=''
 export HTTPS_PROXY=''
 }

Other contents......
sdlcb@Goofy-Gen:~/AMD/SO$ sed -n '/^Other contents/,/^Other contents/{ /^Other contents/!d; /^Other contents/!d; p; }' File
Other contents......
Other contents......
Arjun Mathew Dan
  • 5,240
  • 1
  • 16
  • 27
  • Thank you for your help, because I did not clearly describe the problem, cause you may not be accurate answer, but I still give you +1. Thank you for your detailed explanation. – Vincent Sit Nov 27 '14 at 07:51
1

I would do something like

$ sed '/^#/d; /^function/, /}/ d; /^$/d ' input
Other contents......
Other contents......

What it does?

  • d deletes the pattern space.

  • ^# matches lines starting with #

  • /^function/, /}/ specifies a range

  • ^$ matches empty lines

EDIT

$ sed ' /^#$/d; /^# Proxy/, /}/d;  /function stop_proxy/,  /}/ d ' input
# Other comments......
# Other comments......
# Other comments......
Other contents......
# Other comments......



# Other comments......
Other contents......
# Other comments......
# Other comments......

NOTE The above solution has a limitation that it removes empty comments which has no content like

#
nu11p01n73R
  • 26,397
  • 3
  • 39
  • 52
  • @Vincent This also deletes all blank lines trough the whole file. It may be ok or not. (messes with formatting of the file) – Jotne Nov 27 '14 at 08:25
  • This command has a bug, it will remove all lines beginning with `#` in this file, I just need to remove the six line above the `function start_proxy`. – Vincent Sit Nov 27 '14 at 08:30
  • @Jotne Yes! I just found this bug. – Vincent Sit Nov 27 '14 at 08:31
  • @Vincent This also removes all line starting with `#`, not only the 6 lines in top of the file. – Jotne Nov 27 '14 at 08:38
  • But how do you know what `#` to delete. It may be other comments above int the file, sine there are other text above the `#` lines. It may be comments there too. – Jotne Nov 27 '14 at 08:40
  • @Vincent You should remove the check mark OK for this answer, since it still does not give what you want. – Jotne Nov 27 '14 at 08:52
  • @Jotne The answer is the correct answer after a little update. I do not want to nu11p01n73R unhappy. I think he can solve this little problem, and if he can not solve it, then not too late to cancel the mark. – Vincent Sit Nov 27 '14 at 09:00
  • @Vincent :) I have edited my answer. Please note that it removes the empty comments in the file as well, starts with `#` but not containing any content – nu11p01n73R Nov 27 '14 at 09:06
  • @Vincent You are welcome:) Glad that I could help you – nu11p01n73R Nov 27 '14 at 09:34
  • @nu11p01n73R Anyway, thank you very much for your help, other people's solutions even better, we should choose his answer, I think you will agree. – Vincent Sit Dec 02 '14 at 03:34
1

Here's a Perl solution.

perl -0777 -pe 's/\n+(#.*\n)*\n*function\s+start_proxy\s*\{(.*\n)+?function\s+stop_proxy\s*\{(.*\n)+?\s*\}//'

It's somewhat more precise, and also somewhat more memory-intensive, than the previously posted sed solutions, since it reads the whole file into memory before processing it.

The regex tries to look for

  • \n+ Any empty lines before ...
  • (#.*\n)* ... any number of comment lines, starting with #, followed by ...
  • \n* ... optional empty lines before ...
  • function\s+start_proxy\s*\{
  • (.*\n)+? ... and as few lines as possible up to ...
  • function\s+stop_proxy\s\{
  • (.*\n)+? ... and again as few lines as possible up to ...
  • \s*\} ... the following closing brace.

The quantifier +? looks for the shortest possible match, as opposed to just + which will always grab the longest possible match (greedy matching).

tripleee
  • 175,061
  • 34
  • 275
  • 318
  • I think it's safe to assume that's a placeholder for multiple lines of similar function definitions and other configuration directives which cannot be exactly predicted. – tripleee Nov 27 '14 at 07:40
  • Thank you for your help, because I did not clearly describe the problem, cause you may not be accurate answer, but I still give you +1. Thank you! – Vincent Sit Nov 27 '14 at 07:51
1

This will robustly delete precisely what you asked for:

$ cat bad
#
# Proxy server for iTerm & Terminal Setup Script
# version 0.1
# VincentSit
# Nov 26, 2014
#
function start_proxy {
 export http_proxy='abc.efgh.com:1234'
 export HTTPS_PROXY='abc.efgh.com:1234'
 }

function stop_proxy {
 export http_proxy=''
 export HTTPS_PROXY=''
 }

.

$ cat file
# Other comments......
# Other comments......
# Other comments......
Other contents......
# Other comments......

#
# Proxy server for iTerm & Terminal Setup Script
# version 0.1
# VincentSit
# Nov 26, 2014
#
function start_proxy {
 export http_proxy='abc.efgh.com:1234'
 export HTTPS_PROXY='abc.efgh.com:1234'
 }

function stop_proxy {
 export http_proxy=''
 export HTTPS_PROXY=''
 }

# Other comments......
Other contents......
# Other comments......
# Other comments......

.

$ awk -v RS='^$' -v ORS= 'NR==FNR{bad=$0;next} s=index($0,bad){$0=substr($0,1,s-1) substr($0,s+length(bad))} 1' bad file
# Other comments......
# Other comments......
# Other comments......
Other contents......
# Other comments......

# Other comments......
Other contents......
# Other comments......
# Other comments......
$

It uses GNU awk for multi-char RS. It's easily do-able in other awks too if needed but not going to put effort into that until we find out if this is the solution you are looking for.

With bad in a string:

$ awk -v RS='^$' -v ORS= -v bad="#
# Proxy server for iTerm & Terminal Setup Script
# version 0.1
# VincentSit
# Nov 26, 2014
#
function start_proxy {
 export http_proxy='abc.efgh.com:1234'
 export HTTPS_PROXY='abc.efgh.com:1234'
 }

function stop_proxy {
 export http_proxy=''
 export HTTPS_PROXY=''
 }

" 's=index($0,bad){$0=substr($0,1,s-1) substr($0,s+length(bad))} 1' file
# Other comments......
# Other comments......
# Other comments......
Other contents......
# Other comments......

# Other comments......
Other contents......
# Other comments......
# Other comments......
Ed Morton
  • 188,023
  • 17
  • 78
  • 185
  • Thank you for your answer, it looks more professional. But the contents I need to delete can not be stored in another file(Looks like you store it in a file named 'bad'.), you can only use it as a string assigned to a variable in the shell script. How to solve it? – Vincent Sit Nov 27 '14 at 19:52
  • Just assign it to a variable `awk -v bad="..." 's=index()...' file`. – Ed Morton Nov 27 '14 at 20:00
  • Sorry, probably because the contents I want to delete has more than one line, the command fails to run on my computer, and may I ask you update your answer, write a complete shell script to delete the `bad` in the `file`. I hope it can be directly modified file. – Vincent Sit Nov 27 '14 at 20:12
  • my shell script is: awk -v RS='^$' -v bad="# # Proxy server for iTerm & Terminal Setup Script # version 0.1 # VincentSit # Nov 26, 2014 # function start_proxy { export http_proxy='abc.efgh.com:1234' export HTTPS_PROXY='abc.efgh.com:1234' } function stop_proxy { export http_proxy='' export HTTPS_PROXY='' }" s=index($0,bad){$0=substr($0,1,s-1) substr($0,s+length(bad))} 1' bad file – Vincent Sit Nov 27 '14 at 20:12
  • After running, I get the following error: un.sh: line 15: syntax error near unexpected token `(' un.sh: line 15: ` }" s=index($0,bad){$0=substr($0,1,s-1) substr($0,s+length(bad))} 1' bad file' – Vincent Sit Nov 27 '14 at 20:13
  • Sorry, I still can not get the correct results, see: https://gist.github.com/VincentSit/542375c1bdcb956be244 – Vincent Sit Dec 01 '14 at 08:10
  • Is it possible you are not using GNU awk? What does `awk --version` tell you? What OS are you on? Have you tried stripping it down to a very basic script that reproduces the error message to try to debug it? – Ed Morton Dec 01 '14 at 13:07
  • awk version 20070501, Sorry! I was tested on OS X. How to get the right result that on OS X? – Vincent Sit Dec 02 '14 at 02:59
  • Install GNU awk. It has far more features and fewer bugs than OSX awk. – Ed Morton Dec 02 '14 at 03:10
  • I can install it, but requires users to install it is unrealistic. – Vincent Sit Dec 02 '14 at 03:17
  • So you're writing a tool for people to run on some other machine so they can't access gawk from the bin where you install it on your machine? – Ed Morton Dec 02 '14 at 03:18
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/66003/discussion-between-vincent-and-ed-morton). – Vincent Sit Dec 02 '14 at 03:19