bash grep - negative match

Question

I want to show flag places in my Python unittests where I have been lazy and de-activated tests.

But I also have conditional executions that are not laziness, they are motivated by performance or system conditions at time of testing. Those are the skipUnless ones and I want to ignore them entirely.

Let's take some inputs that I have put in a file, test_so_bashregex.txt, with some comments.

!ignore this, because skipUnless means I have an acceptable conditional flag
@unittest.skipUnless(do_test, do_test_msg)
def test_conditional_function():
    xxx

!catch these 2, lazy test-passing
@unittest.skip("fb212.test_urls_security_usergroup Test_Detail.test_related fails with 302")
def sometest_function():
    xxx
@unittest.expectedFailure
def test_another_function():
    xxx

!bonus points... ignore things that are commented out
   # @unittest.expectedFailure

Additionally, I can't use a grep -v skipUnless in a pipe because I really want to use egrep -A 3 xxx *.py to give some context, as in:

grep -A 3 "@unittest\." *.py

test_backend_security_meta.py:    @unittest.skip("rewrite - data can be legitimately missing")
test_backend_security_meta.py-    def test_storage(self):
test_backend_security_meta.py-        with getMultiDb() as mdb:
test_backend_security_meta.py-

What I have tried:

Trying @ https://www.debuggex.com/

I tried @unittest\.(.+)(?!(Unless\()) and that didn't work, as it matches the first 3.

Ditto @unittest\.[a-zA-Z]+(?!(Unless\())

@unittest\.skip(?!(Unless\()) worked partially, on the 2 with skip.

All of those do partial matches despite the presence of Unless.

on bash egrep, which is where this going to end up, things don't look much better.

jluc@explore$ egrep '@unittest\..*(?!(Unless))' test_so_bashregex.txt
egrep: repetition-operator operand invalid

score 1 · Answer 1 · answered Apr 13 '16 at 17:43

1

you could try this regex:

(?<!#\s)@unittest\.(?!skipUnless)(skip|expectedFailure).*

if you don't care if 'skip' or 'expectedFailure' appear you could simplify it:

(?<!#\s)@unittest\.(?!skipUnless).*

answered Apr 13 '16 at 17:43

Scott Weaver

7,192
2
31
43

No, skip and expectedFailure don't really matter. It's only the skipUnless I care about, in order to avoid it. `@unittest\.(?!skipUnless).*` works well in the online test. But on bash, I get the **repetition-operator operand invalid**. I'll add that it's bash on OSX, because I know that grep/egrep differ slightly there, compared to Linux variants. – JL Peyret Apr 13 '16 at 18:13

tinkertime · Answer 2 · 2016-04-13T19:13:55.297

1

How about something like this - grep seems a bit restrictive

items=$(find . -name "*.py")
for item in $items; do
    cat $item | awk ' 
    /^\@unittest.*expectedFailure/{seen_skip=1;}
    /^\@unittest.*skip/{seen_skip=1;}
    /^def/{
        if (seen_skip == 1)
            print "Being lazy at " $1
        seen_skip=0;
    }
    '
done

edited Apr 13 '16 at 19:13

answered Apr 13 '16 at 18:54

tinkertime

2,972
4
30
45

Well, in real life, the lines are not sitting in a test.txt file, they are picked up by *grep xxx* **\*.py**. so I'd wouldn't be getting the filenames and line #s unless I had an extra step piping that info to awk. Your code does work, printing out *'Being lazy at def'* 3 times. – JL Peyret Apr 13 '16 at 19:04
Updated to loop over all py's - should be able to extend that find however you have your test files structures – tinkertime Apr 13 '16 at 19:14
ok, and it does pick the \*.py's apparently, but the output is still just *Being lazy at def*. So **$1** is only 'def', all of the time. No *@unittest* directive, filename, line # or even the actual function code. Appreciate the effort, that's why I upvoted you, but it looks shakier & more complicated so far than my solution. – JL Peyret Apr 13 '16 at 19:21

score 0 · Answer 3 · edited May 23 '17 at 11:50

OK, I'll put up what I found with sweaver2112's help, but if someone has a good single-stage grep-ready regex, I'll take it.

bash's egrep/grep doesn't like ?! (ref grep: repetition-operator operand invalid). end of story there.

What I have done instead is to pipe it to some extra filters: negative grep -v skipUnless and another one to strip leading comments. These 2 strip out the unwanted lines. But, then pipe their output back into another grep looking for @unittest again and again with the -A 3 flag.

If the negative greps have cleared out a line, it won't show in the last pipestage so drops out of the input. If not, I get my context right back.

egrep -A 3 -n '@unittest\.' test_so_bashregex.txt  | egrep -v "^\s*#" | egrep -v "skipUnless\(" | grep @unittest -A 3

output:

7:@unittest.skip("fb212.test_urls_security_usergroup Test_Detail.test_related fails with 302")
8-def sometest_function():
9-  xxx
10:@unittest.expectedFailure
11-def test_another_function():
12- xxx

And my actual output from running it on * *.py*, rather than my test.txt file:

 egrep -A 3 -n '@unittest\.' *.py  | egrep -v "\d:\s*#" | egrep -v "skipUnless\(" | grep @unittest -A 3

output:

test_backend_security_meta.py:77:    @unittest.skip("rewrite - data can be legitimately missing")
test_backend_security_meta.py-78-    def test_storage(self):
test_backend_security_meta.py-79-        with getMultiDb() as mdb:
test_backend_security_meta.py-80-
--
test_backend_security_meta.py:98:    @unittest.skip("rewrite - data can be legitimately missing")
test_backend_security_meta.py-99-    def test_get_li_tag_for_object(self):
test_backend_security_meta.py-100-        li = self.mgr.get_li_tag()
test_backend_security_meta.py-101-

bash grep - negative match

3 Answers3