The answer marked above is the working solution for me. I could not add comments with my current points but i thought of sharing few more information for others who may find similar situation like me.
In my case, we are using Teamcity builds to run the solution and it was running (MS Build step) using Microsoft Visual Studio 2015. Luckily we also have the Microsoft Visual studio 2017 in the Visual studio section of the build step. I selected 2017, and it ran successfully.
For anyone who is using either TeamCity or Jenkins or similar, first thing is to ensure if your tool has the latest MS build/VS build version configured, like 2017 or 2019 (to be latest). If not, get that first and then point to that version.
There's another hack/option. Tools like JEnkins, Teamcity can also have a step to install the nugets and this post may help too to establish that.