Python: block network connections for testing purposes?

Question

I'm trying to test a package that provides interfaces to a few web services. It has a test suite that is supposed to test most functions without connecting to the internet. However, there are some lingering tests that may attempt to connect to the internet / download data, and I'd like to prevent them from doing so for two reasons: first, to make sure my test suite works if no network connection is available; second, so that I'm not spamming the web services with excess queries.

An obvious solution is to unplug my machine / turn off wireless, but when I'm running tests on a remote machine that obviously doesn't work.

So, my question: Can I block network / port access for a single python process? ("sandbox" it, but just blocking network connections)

(AFAICT, pysandbox doesn't do this)

I'm using py.test so I need a solution that will work with py.test, in case that affects any proposed answers.

Answer 1

Monkey patching socket ought to do it:

import socket
def guard(*args, **kwargs):
    raise Exception("I told you not to use the Internet!")
socket.socket = guard

Make sure this runs before any other import.

Answer 2

Update: There is now a pytest plugin that does the same thing as this answer! You can read the answer just to see how things work, but I strongly recommend using the plugin instead of copying-pasting my answer :-) See here: https://github.com/miketheman/pytest-socket

---

I found Thomas Orozco's answer to be very helpful. Following on keflavich, this is how I integrated into my unit test suite. This works for me with thousands of very different unit test-cases (<100 that need socket though) ... and in and out of doctests.

I posted it here. Including below for convenience. Tested with Python 2.7.5, pytest==2.7.0. (To test for yourself, run py.test --doctest-modules in directory with all 3 files cloned.)

_socket_toggle.py

from __future__ import print_function
import socket
import sys

_module = sys.modules[__name__]

def disable_socket():
    """ disable socket.socket to disable the Internet. useful in testing.

    .. doctest::
        >>> enable_socket()
        [!] socket.socket is enabled.
        >>> disable_socket()
        [!] socket.socket is disabled. Welcome to the desert of the real.
        >>> socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        Traceback (most recent call last):
        ...
        RuntimeError: I told you not to use the Internet!
        >>> enable_socket()
        [!] socket.socket is enabled.
        >>> enable_socket()
        [!] socket.socket is enabled.
        >>> disable_socket()
        [!] socket.socket is disabled. Welcome to the desert of the real.
        >>> socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        Traceback (most recent call last):
        ...
        RuntimeError: I told you not to use the Internet!
        >>> enable_socket()
        [!] socket.socket is enabled.
    """
    setattr(_module, '_socket_disabled', True)

    def guarded(*args, **kwargs):
        if getattr(_module, '_socket_disabled', False):
            raise RuntimeError("I told you not to use the Internet!")
        else:
            # SocketType is a valid public alias of socket.socket,
            # we use it here to avoid namespace collisions
            return socket.SocketType(*args, **kwargs)

    socket.socket = guarded

    print(u'[!] socket.socket is disabled. Welcome to the desert of the real.')


def enable_socket():
    """ re-enable socket.socket to enable the Internet. useful in testing.
    """
    setattr(_module, '_socket_disabled', False)
    print(u'[!] socket.socket is enabled.')

conftest.py

# Put this in the conftest.py at the top of your unit tests folder,
# so it's available to all unit tests
import pytest
import _socket_toggle


def pytest_runtest_setup():
    """ disable the interet. test-cases can explicitly re-enable """
    _socket_toggle.disable_socket()


@pytest.fixture(scope='function')
def enable_socket(request):
    """ re-enable socket.socket for duration of this test function """
    _socket_toggle.enable_socket()
    request.addfinalizer(_socket_toggle.disable_socket)

test_example.py

# Example usage of the py.test fixture in tests
import socket
import pytest

try:
    from urllib2 import urlopen
except ImportError:
    import urllib3
    urlopen = urllib.request.urlopen


def test_socket_disabled_by_default():
    # default behavior: socket.socket is unusable
    with pytest.raises(RuntimeError):
        urlopen(u'https://www.python.org/')


def test_explicitly_enable_socket(enable_socket):
    # socket is enabled by pytest fixture from conftest. disabled in finalizer
    assert socket.socket(socket.AF_INET, socket.SOCK_STREAM)

Answer 3

Building on the very helpful answers from Thomas Orozco and driftcatcher here is a variant that works with Python's unittest and (after a small change) Django.

All you need to do is inherit your test case class from the enhanced NoSocketTestCase class and any access to the network will be detected and raises the SocketAccessError exception.

And this approach also works with Django. You only need to change the NoSocketTestCase class to inherit from django.test.TestCase instead of unittest.TestCase.

While not strictly answering OP's question I think this might be helpful for anyone who wants to block network access in unit tests.

no_sockets.py

import socket
from unittest import TestCase


class SocketAccessError(Exception):
    pass


class NoSocketsTestCase(TestCase):
    """Enhancement of TestCase class that prevents any use of sockets

    Will throw the exception SocketAccessError when any code tries to
    access network sockets
    """

    @classmethod
    def setUpClass(cls):
        cls.socket_original = socket.socket
        socket.socket = cls.guard
        return super().setUpClass()

    @classmethod
    def tearDownClass(cls):
        socket.socket = cls.socket_original
        return super().tearDownClass()

    @staticmethod
    def guard(*args, **kwargs):
        raise SocketAccessError('Attempted to access network')

test_no_sockets.py

import urllib.request
from .no_sockets import NoSocketsTestCase, SocketAccessError


class TestNoSocketsTestCase(NoSocketsTestCase):

    def test_raises_exception_on_attempted_network_access(self):

        with self.assertRaises(SocketAccessError):            
            urllib.request.urlopen('https://www.google.com')

Answer 4

A simple way to put a gag on the requests library:

from unittest import mock

requests_gag = mock.patch(
    'requests.Session.request',
    mock.Mock(side_effect=RuntimeError(
        'Please use the `responses` library to mock HTTP in your tests.'
    ))
)

with requests_gag:
    ...  # no Internet here

Answer 5

httpretty is a small library that solves this problem.

If you are using Django test runner, write a custom test runner where you disable all 3rd party API calls.

# common/test_runner.py

import httpretty
from django.test.runner import DiscoverRunner


class CustomTestRunner(DiscoverRunner):
    def run_tests(self, *args, **kwargs):
        with httpretty.enabled(allow_net_connect=False):
            return super().run_tests(*args, **kwargs)

add this new test runner to your settings

TEST_RUNNER = "common.test_runner.CustomTestRunner"

And from now on all external API calls have to be mocked or httpretty.errors.UnmockedError will be raised.

If you are using pytest, this fixture should work.

@pytest.fixture
def disable_external_api_calls():
    httpretty.enable()
    yield
    httpretty.disable()

Answer 6

I have a pytest solution. pytest-network lybrary help me on this.

# conftest.py
import pytest
import socket

_original_connect = socket.socket.connect

def patched_connect(*args, **kwargs):
    ...
    # It depends on your testing purpose
    # You may want a exception, add here
    # If you test unconnectable situations
    # it can stay like this 
    

@pytest.fixture
def enable_network():
    socket.socket.connect = _original_connect
    yield
    socket.socket.connect = patched_connect

@pytest.fixture
def disable_network():
    socket.socket.connect = patched_connect
    yield
    socket.socket.connect = _original_connect

# test_internet.py
def test_your_unconnectable_situation(disable_network):
    response = request.get('http://stackoverflow.com/')
    response.status_code == 400

Answer 7

Other answers break tests that just create local threads, like pytorch dataloaders. If you want to allow localhost internet connections as well, you can try this code.

    import socket
    orig_connect = socket.socket.connect
    def guard(self, address: tuple, *args: Any, **kwargs: Any) -> None:
        is_local_connection = False
        # Family types documeted at https://docs.python.org/3/library/socket.html#socket-families
        if self.family == socket.AF_UNIX:
            # AF_UNIX is for local connections only
            is_local_connection = True
        elif self.family == socket.AF_INET or self.family == socket.AF_INET6:
            # Check that the host is local
            host = address[0]
            is_local_connection = host  == 'localhost' or host == '127.0.0.1'
        else:
            # Other types are too rare to bother with (e.g. cluster computing)
            pass
        if is_local_connection:
            orig_connect(self, address, *args, **kwargs)
        else:
            assert False, "No internet allowed in unittests"
    socket.socket.connect = guard