How to test complicated functions which use requests?

Question

I want to test my code that is based on the API created by someone else, but im not sure how should I do this.

I have created some function to save the json into file so I don't need to send requests each time I run test, but I don't know how to make it work in situation when the original (check) function takes an input arg (problem_report) which is an instance of some class provided by API and it has this problem_report.get_correction(corr_link) method. I just wonder if this is a sign of bad written code by me, beacuse I can't write a test to this, or maybe I should rewrite this function in my tests file like I showed at the end of provided below code.

# I to want test this function
def check(problem_report): 
    corrections = {}
    for corr_link, corr_id in problem_report.links.items():
        if re.findall(pattern='detailCorrection', string=corr_link):
            correction = problem_report.get_correction(corr_link)
            corrections.update({corr_id: correction})
    return corrections

# function serves to load json from file, normally it is downloaded by API from some page.
def load_pr(pr_id): 
    print('loading')
    with open('{}{}_view_pr.json'.format(saved_prs_path, pr_id)) as view_pr:
        view_pr = json.load(view_pr)
    ...
    pr_info = {'view_pr': view_pr, ...}
    return pr_info

# create an instance of class MyPR which takes json to __init__
@pytest.fixture
def setup_pr():
    print('setup')
    pr = load_pr('123')
    my_pr = MyPR(pr['view_pr'])
    return my_pr 

# test function
def test_check(setup_pr):
    pr = setup_pr
    checked_pr = pr.check(setup_rft[1]['problem_report_pr'])
    assert checker_pr

# rewritten check function in test file 
@mock.patch('problem_report.get_correction', side_effect=get_corr)
def test_check(problem_report):
    corrections = {}
    for corr_link, corr_id in problem_report.links.items():
        if re.findall(pattern='detailCorrection', string=corr_link):
            correction = problem_report.get_correction(corr_link)
            corrections.update({corr_id: correction})
    return corrections

Im' not sure if I provided enough code and explanation to underastand the problem, but I hope so. I wish you could tell me if this is normal that some function are just hard to test, and if this is good practice to rewritte them separately so I can mock functions inside the tested function. I also was thinking that I could write new class with similar functionality but API is very large and it would be very long process.

score 1 · Accepted Answer · answered Apr 17 '19 at 21:41

I understand your question as follows: You have a function check that you consider hard to test because of its dependency on the problem_report. To make it better testable you have copied the code into the test file. You will test the copied code because you can modify this to be easier testable. And, you want to know if this approach makes sense.

The answer is no, this does not make sense. You are not testing the real function, but completely different code. Well, the code may not start being completely different, but in short time the copy and the original will deviate, and it will be a maintenance nightmare to ensure that the copy always resembles the original. Improving code for testability is a different story: You can make changes to the check function to improve its testability. But then, exactly the same resulting function should be used both in the test and the production code.

How to better test the function check then? First, are you sure that using the original problem_report objects really can not be sensibly used in your tests? (Here are some criteria that help you decide: What to mock for python test cases?). Now, lets assume that you come to the conclusion you can not sensibly use the original problem_report.

In that case, here the interface is simple enough to define a mocked problem_report. Keep in mind that Python uses duck typing, so you only have to create a class that has a links member which has an items() method. Plus, your mocked problem_report class needs a method get_correction(). Beyond that, your mock does not have to produce types that are similar to the types used by problem_report. The items() method can return simply a list of lists, like [["a",2],["xxxxdetailCorrectionxxxx",4]]. The same argument holds for get_correction, which could for example simply return its argument or a derived value, like, its negative.

For the above example (items() returning [["a",2],["xxxxdetailCorrectionxxxx",4]] and get_correction returning the negative of its argument) the expected result would be {4: -4}. No need to simulate real correction objects. And, you can create your mocked versions of problem_report without need to read data from files - the mocks can be setup completely from within the unit-testing code.

score 0 · Answer 2 · answered Apr 11 '19 at 15:08

Try patching the problem_report symbol in the module. You should put your tests in a separate class.

@mock.patch('some.module.path.problem_report')
def test_check(problem_report):
    problem_report.side_effect = get_corr
    corrections = {}
    for corr_link, corr_id in problem_report.links.items():
        if re.findall(pattern='detailCorrection', string=corr_link):
            correction = problem_report.get_correction(corr_link)
            corrections.update({corr_id: correction})
    return corrections

How to test complicated functions which use requests?

2 Answers2