Mocking BigQuery Connection in python

Question

I have the following code in a python file. I have to unit test this file. But in order to do that I need to instantiate the object of the class

class BigQuery(metaclass=singleton.Singleton):
    """
    Big Query Class for operations on big query
    Will standardize in future versions.
    """

    def __init__(self):
        """
        Used for initializing client
        """
        try:
            self.client = bigquery.Client.from_service_account_json(
                SERVICE_ACCOUNT_JSON)
        except:
            logging.error("Cannot instantiate bigquery client", exc_info=True)
            raise Exception("Cannot instantiate bigquery client.")

The above class also contains other methods that needs to be tested. How will I mock the object for every method without calling bigquery API??

Have you tried this: https://stackoverflow.com/questions/15753390/how-can-i-mock-requests-and-the-response — Tamir Klein, Dec 31 '19 at 15:14

score 5 · Accepted Answer · answered Jan 01 '20 at 06:14

I got it working. Basically you need to mock the function call to the bigquery client initialization. With the help of mock.patch we can mock the client object or the function from_service_account_json. The following is the code

with patch.object(bigquery.Client, "from_service_account_json") as srv_acc_mock:
            srv_acc_mock.return_value = Mock()

            # do something here....

We need to follow the same pattern for GCS client also but change the bigquery.Client to storage.Client by importing the right modules.

score 5 · Answer 2 · answered Jul 20 '21 at 22:35

While your accepted solution could work, it'll be more complete and robust to mock all of bigquery.Client. This will prevent implementation changes from breaking the mock, and it makes it easy to set return values:

from unittest.mock import patch

@patch('google.cloud.bigquery.Client', autospec=True)
def my_test(mock_bigquery): 
  mock_bigquery().query.return_value = ...

Mocking BigQuery Connection in python

2 Answers2