Why Does This Python Script Freeze Up?

Question

I have a function that tries a list of regexes on some text to see if there's a match.

@timeout(1)
def get_description(data, old):
    description = None
    if old:
        for rx in rxs:
            try:

                matched = re.search(rx, data, re.S|re.M)
                if matched is not None:
                    try:
                        description = matched.groups(1)
                        if description:
                            return description
                        else:
                            continue
                    except TimeoutError as why:
                        print(why)
                        continue
                else:
                    continue
            except Exception as why:
                print(why)
                pass

I use this function in a loop and run a bunch of text files through. In one file, execution keeps stopping:

Traceback (most recent call last):
  File "extract.py", line 223, in <module>
    scrape()
  File "extract.py", line 40, in scrape
    metadata = get_metadata(f)
  File "extract.py", line 186, in get_metadata
    description = get_description(text, True)
  File "extract.py", line 64, in get_description
    matched = re.search(rx, data, re.S|re.M)
  File "C:\Users\Joseph\AppData\Local\Programs\Python\Python36\lib\re.py", line 182, in search
    return _compile(pattern, flags).search(string)
KeyboardInterrupt

It simply hangs on evaluating matched = re.search(rx, data, re.S|re.M). For many other files, when no match is found, it goes on to the next regex. With this file, it does nothing and throws no exception. Any ideas what could be causing this?

EDIT: I'm now trying to detect timeout errors (This is more efficient for me than changing the rx's)

The TimeoutError, borrowed from this question, is triggered but doesn't cause the script to keep running. It simply writes 'Timer expired' and stays frozen.

Seeems like you have an inefficient pattern, possibly encountering catastrophic backtracking. Impossible to say without the actual pattern and string that it is tested against. — Sebastian Proske, Jul 24 '18 at 08:54
@SebastianProske is there a way to detect catastrophic backtracking and simply continue to the next rx? — David J., Jul 24 '18 at 08:55
See https://stackoverflow.com/questions/11901328/how-to-timeout-function-in-python-timeout-less-than-a-second — Sebastian Proske, Jul 24 '18 at 08:57
your pattern match if it return none, it will always goes to else part,and program will always continue. Try to change that — Saranya Sridharan, Jul 24 '18 at 08:57
You'll have to post a MCVE - no one can anwer for sure without it. — bruno desthuilliers, Jul 24 '18 at 09:16
@SebastianProske I've modified it to include a timeout error. It still freezes up even though the timeout error is caught. — David J., Jul 24 '18 at 09:23

Why Does This Python Script Freeze Up?

0 Answers0