2

I'm fairly new to Python and I am having real trouble because I run into it this Segmentation fault: 11 error.

Here is a simple code example that produces this error every time:

    import grequests

    class Url(object):
        pass

    a = Url()
    a.url = 'http://www.heroku.com'
    a.result = 0
    b = Url()
    b.url = 'http://www.google.com'
    b.result = 0
    c = Url()
    c.url = 'http://www.wordpress.com'
    c.result = 0

    urls = [a, b, c]

    rs = (grequests.get(u.url) for i, u in enumerate(urls))
    grequests.map(rs)

What is absolutely bizarre is that if I replace the urls = ... line with this:

urls = [a, b]

Then I get no error, and the script runs fine.

If I change that to just

urls = [c]

Then I also get no error, and the script runs fine.

If I change c.url = ... to

c.url = "http://yahoo.com"

And revert urls = ... back to

urls = [a, b, c]

Then I do get the segmentation fault: 11 error.

Being a memory issue sounds like a possibility though I'm not sure how to fix it.

I've been stuck on this for a number of days, so any help, no matter how small, is greatly appreciated.

For reference, I'm using macOS High Sierra (10.13.5) and installed Python 3.7.0 using Brew.

Jack Robson
  • 2,184
  • 4
  • 27
  • 50
  • What if you only use `[c]`? Maybe the wordpress website is doing something weird and running out of memory. – Peter Wood Jul 14 '18 at 13:04
  • Thank you both for your interest. I've added response to comments in original post. – Jack Robson Jul 14 '18 at 13:12
  • 1
    Works for me on Linux python3.7, pip installed certifi-2018.4.16 chardet-3.0.4 gevent-1.3.4 greenlet-0.4.13 grequests-0.3.0 idna-2.7 requests-2.19.1 urllib3-1.23. Given that 3.7 is very new I'd be suspicious that it might be related. Segfault sounds more like a library/ABI issue than memory, but I'm not an expert on low-level stuff. That's all I've got I'm afraid. – snakecharmerb Jul 14 '18 at 13:22
  • @snakecharmerb that's helpful stuff. Tomorrow I'm going to figure out how to install Python 3.6 on my Mac and see if that helps. – Jack Robson Jul 14 '18 at 13:29

1 Answers1

0

Segmentation fault (violation) is caused by an invalid memory reference. Trying to access an address that should not be accessible for current process (could also be buffer overrun or entirely bogus or uninitialized pointer). Usually it would be indicative of a bug in the underlying code or a problem during binary build (linking).

This problem lies not in your Python script, even though you may be able to trigger it by modifying your python code. Even if you for instance exhausted buffers used in a module or by the interpreter itself, it should still handle that situation gracefully.

Given your script, either gevent (dependency of grequests) or your Python (and/or bits of its standard library) are likely places where a segfault could have occurred (or a library is being used that causes it). Perhaps try rebuilding them? Where there any substantial changes around them on your system since the time you've built them? Perhaps they are trying to run against libraries other than they've been originally built against?

You can also allow your system to dump cores (I presume MacOS being essentially BSD can do that) and inspect (load it into a debugger such as gdb) the coredump to see what exactly crashed and what was going on at the time.

Ondrej K.
  • 8,841
  • 11
  • 24
  • 39