Testing .NET Core with NUnit in Visual Studio 2017

The switch from the project.json format to the new csproj based format for .NET Core and .NET Standard projects also changed the API for test adapters. Because of this, NUnit’s dotnet-test-nunit adapter stopped working and we had to update our existing Visual Studio test adapter to support .NET Core. Last night I released the first alpha release of the test adapter to NuGet enabling testing of .NET Core and .NET Standard projects using Visual Studio 2017, Visual Studio Code, TFS Build, and, the dotnet test CLI command.

Creating a Test Project

The following instructions are for Visual Studio, but the principles apply to all development environments.

In the solution that you want to test, Add | New Project… and add a Visual C# | .NET Core | Class Library (.NET Core).

Add .NET Core Class Library

I am testing a .NET Standard library, but I am using a .NET Core class library for testing. Testing requires a platform to test on, so your test assembly must target a specific platform. I selected .NET Core, but I could also target .NET 4.5 or even multi-target the test assembly. Be aware that multi-targeted test projects are barely supported by Visual Studio and your results may vary. Hopefully we will see improvements in the tooling soon.

Adding NuGet References

First, you will want to add a project reference to the project you are testing, then add NuGet references to the test framework NUnit 3.6.1, to the test runner NUnit3TestAdapter 3.8.0-alpha1 and to the test SDK Microsoft.NET.Test.Sdk 15.0.0. Add the references either using the Manage NuGet Packages… user interface, or by editing the project file.

Add NuGet References to Test Project

Your csproj file should end up looking similar to this. Notice how much cleaner the new format is.

<project Sdk="Microsoft.NET.Sdk">

  <propertygroup>
    <targetframework>netcoreapp1.1</targetframework>
  </propertygroup>

  <itemgroup>
    <packagereference Include="Microsoft.NET.Test.Sdk" Version="15.0.0"></packagereference>
    <packagereference Include="NUnit" Version="3.6.1"></packagereference>
    <packagereference Include="NUnit3TestAdapter" Version="3.8.0-alpha1"></packagereference>
  </itemgroup>

  <itemgroup>
    <projectreference Include="..\NetStandardLibrary\NetStandardLibrary.csproj"></projectreference>
  </itemgroup>

  <itemgroup>
    <service Include="{82a7f48d-3b50-4b1e-b82e-3ada8210c358}"></service>
  </itemgroup>

</project>

From here, write your unit tests as you would for any project.

Running Tests in Visual Studio

Running tests in Visual Studio hasn’t changed. See my previous post on running NUnit tests in Visual Studio if you have questions.

Running Unit Tests in Visual Studio

A few things to note,

  • If you multi-target your tests, only one target will run in Visual Studio, I believe the first
  • Code Coverage is not supported for .NET Core yet, that will be available in a future update to Visual Studio
  • Live Unit Testing is also not available for .NET Core yet

Running Tests from the Command Line

Adding the NUnit 3 Test Adapter to your project will also allow you to use the dotnet test CLI. Here is an example run with the unit tests targeting both .NET 4.5 and .NET Core 1.1.

C:\src\Spikes\NetStandardLibrary
λ dotnet test .\NetStandardLibrary.Tests\NetStandardLibrary.Tests.csproj
Build started, please wait...
Build completed.

Test run for C:\src\Spikes\NetStandardLibrary\NetStandardLibrary.Tests\bin\Debug\net45\NetStandardLibrary.Tests.dll(.NETFramework,Version=v4.5)
Microsoft (R) Test Execution Command Line Tool Version 15.0.0.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
NUnit Adapter 3.8.0.0: Test execution started
Running all tests in C:\src\Spikes\NetStandardLibrary\NetStandardLibrary.Tests\bin\Debug\net45\NetStandardLibrary.Tests.dll
NUnit3TestExecutor converted 18 of 18 NUnit test cases
NUnit Adapter 3.8.0.0: Test execution complete

Total tests: 18. Passed: 18. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 0.9096 Seconds


Test run for C:\src\Spikes\NetStandardLibrary\NetStandardLibrary.Tests\bin\Debug\netcoreapp1.1\NetStandardLibrary.Tests.dll(.NETCoreApp,Version=v1.1)
Microsoft (R) Test Execution Command Line Tool Version 15.0.0.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
NUnit Adapter 3.8.0.0: Test execution started
Running all tests in C:\src\Spikes\NetStandardLibrary\NetStandardLibrary.Tests\bin\Debug\netcoreapp1.1\NetStandardLibrary.Tests.dll
NUnit3TestExecutor converted 18 of 18 NUnit test cases
NUnit Adapter 3.8.0.0: Test execution complete

Total tests: 18. Passed: 18. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 1.1661 Seconds

The tests run for both targets and that you get the output from the dotnet test command, not the normal coloured NUnit output that you would get from the NUnit Console. Hopefully we will be releasing an updated console runner in the near future.

  • sweet..works as intended on Microsoft.NET.Test.Sdk 15.3.0
    NUnit 3.8.1
    NUnit3TestAdapter 3.8.0

  • Okay. That makes it not as onerous. It would be great if we could merely identify a folder/directory that contained all of our test projects. NUnit allows us to filter based on namespaces. So we can set our builds up to look for tests in Test.Unit.* or Test.Integration.* namespaces …or both via Test.*. I’ll decorate all of my test classes while patiently waiting for `dotnet test` to be updated so that it passes the filter options to the NUnit adapter.

  • Shubhang Mani Tripathi

    You can use this decorator at the beginning of the test class as well. Individual tests in the class need not be decorated. But yes, to filter the tests, you’ll need to define categories somewhere

  • Does this mean that we have to go and decorate all of our unit tests with this TestCategory attribute just to filter it?

  • Robert Bratton

    Thanks for this information! I was frustrated with running unit tests in VS 2017 with Dot Net CORE until now.

  • Rob Prouse

    Sorry, not at the moment. You can’t even get them in the NUnit 3 format yet, only the VSTest format. We are a bit limited by the API that Microsoft provides, but are trying to figure out ways to work around them.

  • Benoit Pudal

    Hi, is there a way to get a result file in nunit2 format with dotnet test ?
    Thanks in advance for your help.

  • Jorge

    You can use `dotnet test path/to/csproj –filter “TestCategory~Unit”`

  • Shubhang Mani Tripathi

    Thanks a lot, this option worked! Can you please get this in the official documentation somewhere, so that others avoid getting stuck here?

    Edit: This was my command
    dotnet vstest path_to_dll –TestCaseFilter:”TestCategory=TestSuite.Unit”

  • Rob Prouse

    Sorry, I hadn’t tried it. The help for dotnet test states that extra command line options are passed through. It could be the format of your command, see dotnet vstest –help.

    You could also try running with dotnet vstest.

  • Shubhang Mani Tripathi

    Using dotnet test command with ‘/TestCaseFilter’ switch results in an ‘Unknown switch error’. What other options do I have for filtering test cases?

  • Rob Prouse

    The current version of dotnet test does not pass any options on to the adapter, it only passes them on to VSTest.Console.exe. That limits what NUnit can do, but you can use the /TestCaseFilter command line option to pass categories. Just be aware that you are using the VSTest format, not the NUnit –where format. See https://msdn.microsoft.com/en-CA/library/jj155796.aspx for more info.

  • Shubhang Mani Tripathi

    How can I specify the category of the tests to be executed? (something like –where cat == Unit)?

  • Pius Ngugi

    Works like a charm. Thanks very much

  • Christian Maslen

    Great stuff! Been looking forward to being able to use NUnit with Core and Standard projects.

  • Pingback: Compelling Sunday – 19 Posts on Programming and QA()

  • Günther Foidl

    +1 Thanks!

  • Pingback: NUnit 3 Tests for .NET Core RC2 and ASP.NET Core RC2 - Rob Prouse()