This can happen when building a Xamarin project while using certain libraries. What can help is:

1. Update or install the latest version of Xamarin.Build.Download NuGet.
2. Close all Visual Studio windows.
3. Delete all the obj, bin, .vs folders, as well as the local packages folder in your solution root.
4. Delete the global NuGet cache from ~\.nuget
5. Load the Solution again, Rebuild, and hopefully the problem will go away!