How to unit test this? Big foreach with lots of db calls

Question

I have to make a program that basically reads x lines from a file and then tries to find it in the db then take some of the info in the file and update the info of the row in the db.

If all else fails insert it as a new row in db.

So basically alot of inserting and updating.

I trying to unit test as I am updating a lot of fields I want to make sure that each piece of info is going to right spot(ie not email address is being set with firstname or something like that).

However I am not sure how to do it as I see this being the program method

// get records from file

foreach record in file
{
   db record = find if it is in db

   if(record != null)
   {
        if(do another logic check)
         {
             // update record
         }
         else if(do another logic check)
         { 
            // update record
         }
        else
        {
           // do some more logic 

            if(do another check)
             {
               // update record
             }
        }
   }
   else
   {
      // do some more logic checks and do inserts.
   }
}

I see this in a void method with maybe some private methods(for say update record part). Now how should I be unit testing this? I want to unit test the first if(do another logic check) and if say some record I send in meets those conditions.

However since those would be private methods I can't unit test them, right now I don't see the the method returning anything since it will go through hundreds of records and I probably will be printing out most of stuff into a error log file or something.

the code I showed you is in a service layer. I would mock out the db method calls with moq.

The application is a console app.

Any suggestions how I could better break this up so I could check the logic?

Answer 1

You could mock db calls, also you could mock record type and pass mocked inside method

Answer 2

The code I showed you is in a service layer. I would mock out the db method calls with moq.

That's the key. You should abstract all database dependencies behind an interface that you could mock in your unit test. You would define expectations on the methods. As far as reading the file is concerned, you should abstract the method so that it doesn't take a filename but a Stream or StreamReader instead which would allow you to feed a sample file data in the unit test instead of relying on the existence of an actual file.

So your class could now become:

public class SomeClass
{
    private readonly IRepository _repo;
    public SomeClass(IRepository repo)
    {
        _repo = repo;
    }

    public void SomeMethod(StreamReader reader)
    {
        string line;
        while ((line = reader.ReadLine()) != null)
        {
            // put your logic and use the _repo to update your database.
        }
    }
}

Answer 3

Your class may end up looking like this from a high level:

// get records from file

foreach record in file
{
   dbRecord = _db.Find(record)

   if(dbRecord != null)
   {
        if(logicCheck(dbRecord))
        {
            _db.Update(dbRecord)
        }
        else if(logicCheck2(dbRecord))
        { 
            _db.Update3(dbRecord)
        }
        else
        {
            // do some more logic 
        }
        if(otherCheck(dbRecord))
        {
            _db.Update2(dbRecord)
        }
   }
   else
   {
      // do some more logic checks
      _db.Insert(record)
   }
}

The tests would look something like this:

public void TestInsert()
{
    _mock.Setup(r => r.Find(record).Returns(null))

    class = new class(_mock.Object);
    class.methodToTest(fileRecord);

    _mock.Verify( r => r.Insert(record));
}

public void TestUpdate()
{
    var dbRecordThatPassesLogicCheck = new dbRecord(// initialize for test)
    _mock.Setup(db => db.Find(record).Returns(dbRecordThatPassesLogicCheck))

    class = new class(_mock.Object);
    class.methodToTest(fileRecord);

    _mock.Verify( r => r.Update(dbRecordThatPassesLogicCheck));
}

public void TestUpdate3()
{
    var dbRecordThatPassesLogicCheck2 = new dbRecord(// initialize for test)
    _mock.Setup(db => db.Find(record).Returns(dbRecordThatPassesLogicCheck2))

    class = new class(_mock.Object);
    class.methodToTest(fileRecord);

    _mock.Verify( db => db.Update3(dbRecordThatPassesLogicCheck2));
}

What Moq (or any other mocking framework) buys you here is the ability to simulate that the class calls Update if logicCheck is true, or calls Update3 if logicCheck2 passes but logicCheck1 does not. You don't mock out your private methods, you exercise them with preset input coming from the find method (whose repository you are mocking).

Your class should probably take a second interface dependency on the retrieval of the initial record get which will make the code more decoupled (now if file system read changes to web service read the class isn't directly affected) and allow you more testability.

Answer 4

We tend to use lots of small classes with injected dependencies. This provides usability and testability seams. We also tend to extract if .. else if .. else blocks into injectable Strategy objects - which makes their logic testable outside of the process that uses them.

This separates the concerns for reading the file

class RecordFileReader
{
    void UpdateAll(file)
    {
        foreach (var record in file)
        {
            _fileRecordImporter.Import(record)
        }
    }
}

from processing a single file record

class FileRecordImporter
{   
    void Import(record)
    {
        db_record = find if it is in db
        if (db_record != null)
        {
            _dbRecordUpdater.Update(db_record, record)
            return;
        }
        _dbRecordCreator.CreateFrom(record);
    }
}

and determining how to transfer information from the record to the db for updates

class DbRecordUpdater
{
    void Update(db_record, record)
    {
        var recordUpdater = _recordUpdaters.FirstOrDefault(x=>x.IsMatch(db_record, record));
        if (recordUpdater != null)
        {
            recordUpdater.Update(db_record, record)
        }
    }
}

and inserts

class DbRecordCreator
{
    void CreateFrom(record)
    {
        var recordCreator = _recordCreators.FirstOrDefault(x=>x.IsMatch(record));
        if (recordCreator != null)
        {
            var db_record = recordCreator.Create(record)
            ...
        }
    }
}