Docs.wiki.git

Repo URL: https://github.com/nunit/docs.wiki.git
Edited by:
Cover image: Cover image
Share this using: email, Google+, Twitter, Facebook.
Exports: EPUB | MOBI

There are two ways of installing the adapter within Visual Studio. See below for info on how to choose.

  • Use the Extension Manager
  • Add it as a Nuget package to your solution. (Requires VS 2012 Update 1 or later)
0.0.0.0.1 Installing With the Extension Manager

To install the NUnit Test Adapter using the Extension Manager, follow these steps:

  1. From within Visual Studio, select Tools | Extension Manager.
  2. In the left panel of the Extension Manager, select Online Extensions
  3. Locate (search for) the NUnit 3.0 Test Adapter in the center panel and highlight it.
  4. Click ‘Download’ and follow the instructions.

Use the Extension Manager to ensure that the NUnit 3.0 Test Adapter is enabled.

0.0.0.0.2 Installing the Nuget Package

To add it is a Nuget package, you must have an active solution, then follow these steps:

  1. From Tools menu, use Library Package Manager and select Manage NuGet packages for solution
  2. In the left panel, select Online
  3. Locate (search for) NUnit 3.0 Test Adapter in the center panel and highlight it
  4. Click ‘Install’
  5. In the “Select Projects” dialog, you need to select at least one project to add the adapter to, see notes below.
0.0.0.0.3 How to choose between Extension and NuGet package

The Extension will apply to Visual Studio itself, and will work for all projects you use. All users of your solution need to install the Extension. If you use TFS Build you must also install the extension to the build system there.

The NuGet Package will apply to the solution, and will work for any other user too, as it follows the solution, but requires the user to have VS 2012 Update 1 or above. It will also work for any TFS 2012 Update 1 or above server build, including TF Service and requires no further installation.

0.0.0.1 License - NUnit and NUnit3 Test Adapters for Visual Studio

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

0.0.0.2 NUnit3 Test Adapter for Visual Studio - Version 3.5 - October 22, 2016

0.0.0.2.1 Features
  • The adapter now uses version 3.5 of the NUnit engine
  • The NuGet package is now installed as a tool and no longer creates unnecessary references in the project
0.0.0.2.2 Resolved Issues
  • 97 Tests with dynamic/random parameters are never run closed:done is:bug pri:normal
  • 204 If a test writes to Console.Error, the test passes but the session fails
  • 220 Visual Studio Test Adapter - Writing to test output throws an error
  • 221 Change adapter package to use tools directory
  • 236 Update adapter to use Version 3.5 of the Engine
  • 238 Just warn upon failing to restore random seed
  • 239 Remove Error-level log messages from adapter where we don’t want run to fail
  • 243 Nuget package and copy local

0.0.0.3 NUnit3 Test Adapter for Visual Studio - Version 3.4.1 - August 5, 2016

0.0.0.3.1 Features
  • We now use Cake to build and package the adapter.
0.0.0.3.2 Resolved Issues
  • 198 Create Cake script for build
  • 202 NUnit3 Test Adapter not running all tests in Visual Studio 2015
  • 205 Adapter fails to find Mono.Cecil when targeting .NET 3.5

0.0.0.4 NUnit3 Test Adapter for Visual Studio - Version 3.4 - July 2, 2016

0.0.0.4.1 Features
  • The adapter now uses the NUnit 3.4.1 TestEngine.
  • TestRunParameters may now be provided in the .runsettings file.
  • Immediate text output from tests now displays in the Output window. This includes any output produced through Console.Error, TestContext.Error or TestContext.Progress.
0.0.0.4.2 Resolved Issues
  • 132 Print to console not shown with v3 of adapter
  • 138 Cannot run navigation tests under the console runner
  • 145 Implement TestRunParameters inside .runsettings for runtime parameters
  • 180 Upgrade to NUnit 3.4.1
  • 181 Can’t run requring a 32-bit process
  • 183 Use Mono.Cecil to retrieve source code locations
  • 190 Nunit30Settings.xml is used by other process leads to hidden tests
  • 192 Corrupt NUnit3Settings.xml causes crash

0.0.0.5 NUnit3 Test Adapter for Visual Studio - Version 3.2 - June 3, 2016

0.0.0.5.1 Features
  • The adapter now uses the NUnit 3.2.1 TestEngine.
0.0.0.5.2 Resolved Issues
  • 131 NUnit test adapter not running all tests - only on VS2015
  • 135 Upgrade TestEngine to 3.2.1
  • 162 Visual Studio 15 support
  • 163 No source location when inheriting test methods from base test fixture
  • 174 Clarify that NUnit v2 tests are not discovered by v3 adapter
  • 176 More Information link in vsix broken

0.0.0.6 NUnit3 Test Adapter for Visual Studio - Version 3.0 - April 2, 2016

0.0.0.6.1 Features
  • This is the final production release of the 3.0 adapter. It continues to use the 3.0.1 release of the NUnit TestEngine.

  • The adapter now uses a .runsettings file for all optional settings. Registry entries used in the CTP releases are no longer used.

0.0.0.6.2 Resolved Issues
  • 49 Need a way to specify test settings
  • 52 Having ApartmentAttribute on both classes and methods causes test runner to hang
  • 85 Failure to run tests under vstest.console from VS2015
  • 92 Provide option to run in parallel for paralellized tests
  • 120 The ability to set the LevelOfParallelism attribute through the VS adapter
  • 153 TFS Filter that matches no names runs all tests

0.0.0.7 NUnit3 Test Adapter for Visual Studio - Version 3.0 CTP 9 - April 2, 2016

0.0.0.7.1 Features
  • The adapter continues to use the 3.0.1 release of the NUnit TestEngine.
0.0.0.7.2 Resolved Issues
  • 2 CI Build
  • 34 Identifying Non-Primitive Parameterized Inputs in Adapter vs Console
  • 50 Nuget version install script doesn’t work with VS 2015
  • 66 Build the adapter in AppVeyor
  • 84 CopyLocal=False is an issue in a specific use case
  • 94 More Information Link in Adapter Broken
  • 96 Working directory is set to VS TestWindow extension directory
  • 102 Package VS2012 assemblies as a private NuGet Package
  • 104 Can’t overload async/await methods with NUnit Test Adapter
  • 106 Explicit tests appear as warnings in NUnit 3.0
  • 109 NUnit 2 tests are detected as errors
  • 112 Test adapter fails to load an assembly that references a class from NUnit.Framework but contains no tests
  • 117 Version 3.0.8.0 as nuget package only. No tests detected in Visual Studio 2015
  • 118 Corrupted ignore.addins file in installation

0.0.0.8 NUnit3 Test Adapter for Visual Studio - Version 3.0 CTP 8 - December 2, 2015

0.0.0.8.1 Features
  • The adapter now uses the 3.0.1 release of the NUnit TestEngine.
0.0.0.8.2 Resolved Issues
  • 81 Cannot run tests with ‘>’ in name
  • 86 Generic Test Fixtures are not getting triggered
  • 88 Upgrade adapter to use NUnit 3.0.1

0.0.0.9 NUnit3 Test Adapter for Visual Studio - Version 3.0 CTP 7 - November 16, 2015

0.0.0.9.1 Features
  • The adapter now uses the released NUnit 3.0 TestEngine.
0.0.0.9.2 Resolved Issues
  • 75 Update adapter to use final release of NUnit 3.0

0.0.0.10 NUnit3 Test Adapter for Visual Studio - Version 3.0 CTP 6 - November 10, 2015

0.0.0.10.1 Features
  • This release continues to use the NUnit RC 2 Engine.
0.0.0.10.2 Resolved Issues
  • 14 NUnit Adapter throws System.Reflection.TargetInvocationException, even if the solution build is OK
  • 56 Exception System.Reflection.TargetInvocationException after NUnit 3.0.0-beta-5 upgrade
  • 68 NUnit3TestExecutor.MakeTestFilter does not create valid xml
  • 69 Nunit 3.0.0-rc-2 : System.Reflection.TargetInvocationException
  • 70 NUnit3TestExecutor.MakeTestFilter creates element not handled by NUnit.Framework.Internal.TestFilter

0.0.0.11 NUnit3 Test Adapter for Visual Studio - Version 3.0 CTP 5 - November 9, 2015

0.0.0.11.1 Features
  • This release uses the NUnit RC 2 Engine.
0.0.0.11.2 Resolved Issues
  • 27 Async void methods do not show up as not runnable
  • 43 Remove Wrappers for Engine Classes
  • 45 Remove workaround for tests not sending events
  • 53 Replace core engine
  • 57 Confusing message when an NUnit V2 test is detected

0.0.0.12 NUnit3 Test Adapter for Visual Studio - Version 3.0 CTP 4 - July 20, 2015

0.0.0.12.1 Features
  • This release continues to use the NUnit 3.0 beta-2 engine but is nevertheless able to run tests that reference NUnit 3.0 beta-3 framework.

  • When a debugger is attached, only a single worker thread is used to run tests.

  • The adapter now compensates for the fact that NUnit does not send results for tests that were skipped due to an attribute on the fixture by generating those results itself.

0.0.0.12.2 Resolved Issues
  • 16 Adapter does not detect C++/CLI assemblies
  • 26 Ignored test case does not show up as ignored
  • 33 Inconsistent display behavior in Test Explorer
  • 36 Option to set number of worker threads

0.0.0.13 NUnit3 Test Adapter for Visual Studio - Version 3.0 CTP 3 - May 22, 2015

0.0.0.13.1 Features

This release was issued to correct a problem with locking of assemblies in ctp-2.

0.0.0.13.2 Resolved Issues
  • 29 Latest test adapter locking dlls

0.0.0.14 NUnit3 Test Adapter for Visual Studio - Version 3.0 CTP 2 - May 16, 2015

0.0.0.14.1 Features
  • The adapter now uses the new nunit.core.engine to load and run tests, eliminating adhoc code that worked directly with the framework. This will allow us to much greater flexibility in the future.

  • The adapter has been upgraded to use the beta-2 release of the NUnit core engine. Because the API has changed from beta-1, the adapter can only run tests built against the beta-2 release of NUnit.

0.0.0.14.2 Resolved Issues
  • 13 Adapter will not load as a NuGet package
  • 17 Can’t read app.config settings within test methods
  • 18 Separate NUnit3TestDemo from NUnitTestAdapter solution
  • 19 Use core engine
  • 20 Upgrade NUnit to beta-2

0.0.0.15 NUnit3 Test Adapter for Visual Studio - Version 3.0 CTP 1 - April 6, 2015

0.0.0.15.1 Features
  • Initial release of the test adapter using NUnit 3.0. Note that the adapter may <b>not</b> be used to run tests written against earlier versions of NUnit. The original adapter is still available for that purpose and both adapters may be installed if necessary.

0.0.0.16 NUnit Test Adapter for Visual Studio - Version 2.0 - April 1, 2015

0.0.0.16.1 Features
  • Tested for up to VS2015 Pre-release CTP 6
  • Updated to use NUnit 2.6.4
  • Adapter does not try to discover tests if the nunit.framework version is 3.0 or greater
0.0.0.16.2 Bug Fixes
  • 0.0.0.16.2 61 Confusing NUnit version message made clearer

  • 0.0.0.16.2 62 Adapter uses shadowcopy setting in discoverer but not in the executor

0.0.0.17 NUnit Test Adapter for Visual Studio (RTM) - Version 1.2 - September 17, 2014

0.0.0.17.1 Features
  • Tested for up to VS2013 Update 3
  • Bugs 39 and 40 was inability to run under VS2012. This is now fixed.
0.0.0.17.2 Bug Fixes
  • 0.0.0.17.2 24 Long-running tests ignored

  • 0.0.0.17.2 34 Adapter causes ArgumentException to be thrown by Microsoft logger proxy’s SendMessage method

  • 0.0.0.17.2 37 TestExecution throws Exception System.InvalidOperationException in TFS Build

  • 0.0.0.17.2 38 NUnit only accepts absolute paths to test assembly

  • 0.0.0.17.2 39 VSTest unable to find NUnit tests since 1.1.0.0

  • 0.0.0.17.2 40 NUnit version 1.1.0.0 is broken with test class which ran under 1.0.0.0

0.0.0.18 NUnit Test Adapter for Visual Studio (RTM) - Version 1.1 - April 26, 2014

0.0.0.18.1 Features
  • Support for NUnit 2.6.3
  • Tested for up to VS2013 Update 2 RC
  • Shadow copy now disabled by default, see issue #7 Unable to disable shadow copy.
  • Registry settings added for some customization options, see Tips and Tricks
  • All code moved to github
0.0.0.18.2 Bug Fixes
  • 0.0.0.18.2 13 Category attribute not working with TFS test case filter

  • 0.0.0.18.2 21 Xamarin.iOS Nunit project causes adapter to throw

0.0.0.19 NUnit Test Adapter for Visual Studio (RTM) - Version 1.0 - September 12, 2013

0.0.0.19.1 Features
  • This is the release version 1.0 of the test adapter.
0.0.0.19.2 Bug Fixes
  • 0.0.0.19.2 1208148 The test result output node is not shown for debug/trace statements

0.0.0.20 NUnit Test Adapter for Visual Studio (RC) - Version 0.97 - September 12, 2013

0.0.0.20.1 Features
  • This is the release candidate for version 1.0 of the test adapter.
0.0.0.20.2 Bug Fixes
  • 0.0.0.20.2 1208161 NUnit Test Adapter runs [Explicit] unit tests in TFS Build

  • 0.0.0.20.2 1210536 No Source Available for Async Tests

  • 0.0.0.20.2 1165188 Clicking “Run Selected Tests” doesn’t show Trace.WriteLine() output

0.0.0.21 NUnit Test Adapter for Visual Studio (Beta 6) - Version 0.96 - June 28, 2013

0.0.0.21.1 Features
  • Support for Visual Studio 2013 Preview
0.0.0.21.2 Bug Fixes
  • 0.0.0.21.2 1189268 Profile a test will crash with exception

0.0.0.22 NUnit Test Adapter for Visual Studio (Beta 5) - Version 0.95.2 - June 7, 2013

0.0.0.22.1 Bug Fixes
  • 0.0.0.22.1 1188000, adapter dont work with solutions with only .net 2.0/3.5 project

0.0.0.23 NUnit Test Adapter for Visual Studio (Beta 5) - Version 0.95.1 Hotfix- May 28, 2013

0.0.0.23.1 Bug Fixes
  • Hotfix for debug issue

0.0.0.24 NUnit Test Adapter for Visual Studio (Beta 5) - Version 0.95 - May 10, 2013

0.0.0.24.1 Features
  • 0.0.0.24.1 1174925 Add support for installing the adapter from Nuget

0.0.0.24.2 Bug Fixes
  • 0.0.0.24.2 1155617 Grouping by class name in VS 2012 doesn’t work

  • 0.0.0.24.2 1165359 Exception after building Coded UI test

  • 0.0.0.24.2 1116747 vstest.executionengine.x86.exe does not terminate

  • 0.0.0.24.2 1093178 Eliminate unnecessary files from VSIX

0.0.0.25 NUnit Test Adapter for Visual Studio (Beta 4) - Version 0.94 - December 22, 2012

0.0.0.25.1 Features
  • Works with Visual Studio 2012 Update 1 as well as the RTM.
  • Supports filtering and sorting tests by Traits under Update 1.
  • Supports use of standard filter expressions when running under TFS Update 1.
  • NUnit Categories specified on the fixture class are now recognized and honored.
0.0.0.25.2 Bug Fixes
  • 1074891 Can’t test multiple assemblies referencing different NUnit versions
  • 1075893 Test execution fails if solution contains native C++ project
  • 1076012 No source information found for async test methods
  • 1087629 TestFixture Category not being recognised as traits in VS2012 update 1
  • 1091020 Adapter doesnt support TFS Build traits/test case filtering

0.0.0.26 NUnit Test Adapter for Visual Studio (Beta 3-2) - Version 0.93.2 - November 2, 2012

0.0.0.26.1 Bug Fixes
  • 1074544 Failures in Test Discovery not reporting sufficient information

0.0.0.27 NUnit Test Adapter for Visual Studio (Beta 3-1) - Version 0.93.1 - October 26, 2012

0.0.0.27.1 Bug Fixes
  • 1072150 NUnit adapter 0.93 won’t run selected tests

0.0.0.28 NUnit Test Adapter for Visual Studio (Beta 3) - Version 0.93 - October 24, 2012

0.0.0.28.1 Features
  • Works with Visual Studio 2012 RTM. Some features require the November CTP update.
  • The adapter now uses NUnit 2.6.2. Among other things, this allows us to support async test methods. See the NUnit Release Notes for more info.
  • Source file and line number can now be found for test cases that have an alternate name set.
  • Console output from tests is now displayed in the Visual Studio Output window.
  • TestFixtureSetUp and TestFixtureTearDown errors are now displayed in the Output window.
  • The caret line (——^) is no longer displayed in the IDE since it depends on use of a fixed font.
  • Tests may now be grouped and filtered by Category (only under the November CTP update for VS2012).
0.0.0.28.2 Bug Fixes
  • 1021144 Text output from tests not displayed in Visual Studio IDE
  • 1033623 Not possible to include or exclude tests based on [Category] attribute Released
  • 1040779 Null reference exception on generic test fixtures
  • 1064620 Support async test methods
  • 1065209 Should call both RecordEnd and RecordResult at end of a test
  • 1065212 Upgrade NUnit to 2.6.2
  • 1065223 Error messages assume a fixed font, but don’t get one
  • 1065225 No display for TestFixtureSetUp/TearDown or SetUpFixture errors
  • 1065254 Cannot open test from Test Explorer for tests in a Parameterized Test Fixture
  • 1065306 Generic Fixtures aren’t discovered.
  • 1066393 Unable to display source for testcases with an alternate name set
  • 1066518 Executed fast test appears in Not Run category in Test Explorer

0.0.0.29 NUnit Test Adapter for Visual Studio (Beta 2) - Version 0.92 - May 3, 2012

0.0.0.29.1 Features
  • Works with Visual Studio 2012 Release Candidate
  • Uses NUnit 2.6
0.0.0.29.2 Bug Fixes
  • 992837 Unable to Debug using VS Test Adapter
  • 994146 Can’t run tests under .NET 2.0/3.5

0.0.0.30 NUnit Test Adapter for Visual Studio (Beta 1) - Version 0.91 - February 29, 2012

0.0.0.30.1 Features
  • Built against Visual Studio 11 Beta 1
  • Uses NUnit 2.6

0.0.0.31 NUnit Test Adapter for Visual Studio (Alpha) - Version 0.90 - February 21, 2012

0.0.0.31.1 Features
  • First release of the test adapter. Compatible with the Visual Studio 11 Developer Preview.
  • Uses NUnit 2.6.

Under Construction

0.0.0.32 Known Problems

  1. There is no test status in Visual Studio corresponding to NUnit’s Inconclusive result, so tests with this result are reported as Not Run. Click on the individual test to see the result.

  2. Theories are reported as individual cases, rather as a single unit.

  3. In NUnit, tests have names, which are not necessarily unique. Visual Studio wants the names to be unique. So if two tests have the same name, VS displays a warning message in the output window. The message may be ignored. Two separate results will be shown under the single test in the explorer pane.

  4. Startup performance is substantially improved but is still slower than using NUnit directly.

  5. A VSIX adapter of older version will be used regardless of version of NuGet adapter.

Workaround: Make sure you have upgraded VSIX adapter to latest version, or uninstalled it if you have the NuGet adapter in a solution. The adapter will display its version number in the Output window under Tests.

0.0.1 Issues with other tools

  • Versions of Resharper earlier than the 8.2 version has an issue with the NuGet adapter, which will prevent NUnit tests from running. Make sure you have updated Reshaper to at least version 8.2.

0.0.1.1 Further information

For more information see the blogs by <a href=“http://nunit.com/blogs/” title=“Charlie Poole”>Charlie Poole</a> and <a href=“http://geekswithblogs.net/terje/Default.aspx”>Terje Sandstrom</a>

The MSDN ALM blog post series on How to Manage Unit Tests in Visual Studio 2012 Update 1 is useful for later versions as well:

  • <a href=“http://blogs.msdn.com/b/visualstudioalm/archive/2012/11/09/how-to-manage-unit-tests-in-visual-studio-2012-update-1-part-1-using-traits-in-the-unit-test-explorer.aspx”>Part 1–Using Traits in the Unit Test Explorer</a>

  • <a href=“http://blogs.msdn.com/b/visualstudioalm/archive/2012/11/20/part-2-using-traits-with-different-test-frameworks-in-the-unit-test-explorer.aspx”>Part 2–Using Traits with different test frameworks in the Unit Test Explorer</a>

  • <a href=“http://blogs.msdn.com/b/visualstudioalm/archive/2013/06/11/part-3-unit-testing-with-traits-and-code-coverage-in-visual-studio-2012-using-the-tfs-build-and-the-new-nuget-adapter-approach.aspx”>Part 3: Unit testing with Traits and code coverage in Visual Studio 2012 using the TFS Build</a>

0.0.1.2 Reporting Problems

Bugs should be reported using the <a href=“https://github.com/nunit/nunit3-vs-adapter/issues”>issue tracker</a> on Github.

####NOTE: As of the 3.0 final release, these registry settings are no longer recognized. Instead, use settings in the .runsettings file. This page will be updated soon.

0.0.2 Registry Settings

Certain settings in the registry affect how the adapter runs. All these settings are added by using RegEdit under the key **HKCU.org*.

0.0.2.1 ShadowCopy

By default NUnit no longer uses shadowcopy. If this causes an issue for you shadowcopy can be enabled by setting the DWORD value UseShadowCopy to 1.

0.0.2.2 KeepEngineRunning

By default the NUnit adapter will “Kill” the Visual Studio Test Execution engine after each run. Visual Studio 2013 and later has a new setting under its top menu, Test | Test Settings | Keep Test Execution Engine Running. The adapter normally ignores this setting.

In some cases it can be useful to have the engine running, e.g. during debugging of the adapter itself. You can then set the adapter to follow the VS setting by setting the DWORD value UseVsKeepEngineRunning to 1.

0.0.2.3 Verbosity

Normally the adapter reports exceptions using a short format, consisting of the message only. You can change it to report a verbose format that includes the stack trace, by setting a the DWORD value Verbosity to 1.

Under Construction

In preparation for using the test adapter, make sure that the Unit Test Explorer is shown in your window. If you don’t see it, use the menu ( Test | Windows | Test Explorer ) to show it and position the window where you would like it within Visual Studio.

[[images/nunitTestAdapter.png]]

When you initially open a solution, no tests will be displayed. After compiling the assemblies in the solution, Visual Studio will interact with the NUnit Test Adapter to discover tests and a list of them will be shown in the Test Explorer.

Click on Run All in the Test Explorer to run all the tests. You may also select one or more tests in the list and run them by right-clicking and using the context menu. The context menu also contains entries for debugging tests and for navigating to the source code of a test.

Tests may be grouped by Duration and Outcome under VS2012 RTM and also by Traits and Project using Update 1 or later, and Class using Update 2 or later. NUnit translates any Categories and Properties used on your tests to Visual Studio as Traits.

Tests may be filtered in Visual Studio under Update 1 or later by Trait, Project, Error Message, File Path, Fully Qualified Name, Output and Outcome. Use the search edit box at the top of the list of tests to specify a filter.

Tests may be organized by play lists in Visual Studio under Update 2 or later. Playlists are more or less equal to the old test lists from VS 2010.

Parameterized tests will show up as separate test cases for each set of parameters.

For settings options, see the [[Tips and Tricks]]

Under Construction

The NUnit 3.0 Test Adapter allows you to run NUnit 3.0 tests inside Visual Studio. This is a new adapter, based partly on the code of the original NUnit Test Adapter, but modified to work with NUnit 3.0.

It is not possible to run NUnit 2.x tests using this adapter. Use the original adapter for that purpose. If you need to work with projects using NUnit 2.x and other projects using NUnit 3.0, you may install both versions of the adapter.

The current 3.0 Beta release is designed to work with Visual Studio 2012, 2013 and 2015. Some features are not available under VS2012 RTM.

Releases of Visual Studio prior to VS 2012 did not have the ability to directly run tests built with Open Source testing frameworks like NUnit.

0.0.3 User Docs

  • [[NUnit|NUnit-Documentation]]
  • [[NUnit Gui]]
  • [[Project Editor]]
  • Visual Studio
  • [[Test Adapter|Visual-Studio-Test-Adapter]]
    • [[Installation|Adapter Installation]]
    • [Usage]
    • [[Resources]]
    • [Known Problems]
    • [[Tips and Tricks]]
    • [[Release Notes|Adapter Release Notes]]
    • [[License|Adapter License]]
  • [[Test Generator|Visual-Studio-Test-Generator]]
  • [[User FAQ]]

0.0.4 Developer Docs

  • [[NUnit Vision]]
  • [[NUnit Roadmap]]
  • [[Team Practices]]
  • [[Specifications]]
  • [[Developer FAQ]]

NOTE: This page is completely out of date. We need to decide whether to continue maintaining it or remove it.

This page is a first draft of a roadmap taking us to an NUnit 3.0 release.

0.0.4.1 NUnit 3.0 Initial Alpha Release

  • All NUnit 2.6 features being removed should be removed before the first alpha
  • All NUnit 2.6 features that are NOT being removed should be in place
  • All outstanding bugs marked as “Review” or “Confirm” should be either closed or assigned a priority
  • All outstanding Critical and high priority bugs should be resolved
  • Framework, Engine and Console included
  • Framework and Engine Apis reasonably stable
  • Combined zip file for full installation
  • NuGet package for full and nunitlite frameworks
0.0.4.1.1 Platform Support
  • .NET 4.5, 4.0, 3.5 and 2.0 and corresponding Mono platforms
  • Silverlight 5.0
0.0.4.1.2 Specific Features
  • Implement ActionAttributes (#112)
  • Implement Changes to File, Directory and Path Assertions (#111)
  • Standardize commandline options for nunitlite runner (#13)

0.0.4.2 NUnit 3.0 Final Release

  • All outstanding Critical, High and Normal priority bugs should be resolved.
  • Framework, Engine, Console and Gui included
  • Framework, Driver and Engine Apis stable
  • Combined zip file and Msi installer for full installation
  • VSix and NuGet packages for 3.0-based test adapter
  • NuGet package for full and nunitlite frameworks
0.0.4.2.1 Platform Support
  • .NET 4.5, 4.0, 3.5 and 2.0 and corresponding Mono platforms
  • Silverlight 5.0, possibly 4.0
  • Windows Phone 8.0 and 8.1
  • Possibly, .NET CF 3.9 (1)
0.0.4.2.2 Specific Features
  • Parallel Test Execution within assemblies (#66)
  • Run test assemblies in parallel (Console#7)
  • NUnit-Console should support incremental output under TeamCity (Console#7)
  • Full async support (review outstanding issues to define this more clearly)
  • Included driver for NUnit 2.x tests

0.0.4.3 Notes

  1. Compact framework is not a high priority, but 3.9 is listed since we are removing the existing 2.0 and 3.5 support. If we can simply substitute 3.9 for 3.5 without significant changes, as advertised, then we will retain the code for it.

0.0.4.4 Outstanding Issues

  • Timing for additional platforms we want to support.
  • Other features to be added Stated simply…

The NUnit 3.0 Extended Testing Platform will be a world-class platform for individuals and teams using a test-centric approach to .NET software development.

On this page, we articulate the details of what that statement means to us. We don’t specify features here - see the Nunit Roadmap for that - but try to provide set of guidelines that will help us when we are considering what features to add.

Note that our vision of an Extended Testing Platform is substantially changed from the earlier vision of NUnit as a framework. Some of the changes are explained in more detail below. You can read the [[Original Vision]] if you would like to compare them directly.

0.0.5 A Bit More Detail

NUnit began as a relatively simple framework with a console-based runner. A Gui runner was added for version 2.0 along with a long list of features, but it remained essentially a test framework with a few runners. With NUnit 3.0, we aim to expand the role of NUnit so that it provides a platform for test-centric development.

Such an approach puts increased demands on NUnit as a tool. It must support a number of different kinds of tests, allow for alternative approaches to testing and be usable by team members in a variety of roles.

0.0.5.1 Kinds of Tests

NUnit began as a unit testing framework. In later versions, we decided to support use of NUnit for integration and acceptance testing, but only to the extent that this support didn’t interfere with our primary goal. With NUnit 3.0, we will fully support integration, acceptance, performance and other types of tests in addition to unit tests.

0.0.5.2 Approaches to Testing

All of us are pretty excited about Test-Driven Development. However, we recognize that many programmers are either not yet using TDD or are working on legacy projects for which TDD is not even possible. NUnit 3.0 will support various alternative approaches, including generation of tests for existing code bases.

0.0.5.3 Developer versus Programmer

NUnit has always been viewed primarily as a tool for programmers. With the 3.0 version, we want to recognize and fulfill the needs of programmers, testers, interaction designers, web designers, data base specialists, customers, managers and any other members of the whole team.

0.0.5.4 NUnit as a Platform

All of this is a pretty tall order. We expect to satisfy many of these needs directly, but it would be impossible to satisfy - or even predict - all of them. By treating NUnit as a platform, we will make it possible for individuals, teams or third parties to add support for activities and approaches that are not provided out of the box.

0.0.6 Future Directions

Taking the above into account along with the requests we’ve received from users and our own interests, here are some directions we expect the next few releases of NUnit to take

We’ll continue to expand the available choices regarding where tests are run. We now support use of multiple AppDomains as well as separate processes. We expect to expand this to cover test execution under a particular subsystem or on a separate machine.

NUnit will be extensible. We can’t do everything for everybody but we will make it reasonably easy to extend NUnit. In many cases, users will be able to implement a plugin that provides a special feature outside of our scope by simply creating a new attribute that embeds the required logic. In more complex cases, particularly in extending runners, we will rely on a plugiin-architecture.

We’ll continue to focus on .NET development. Testing non-.NET apps through tests written in a .NET language is an acceptable use of NUnit, but we won’t give high priority to features that are needed only for that purpose.

We’ll remain language neutral. We’d like to work with any supported .NET languages. We currently support C#, VB.NET, C++, J# and F#. We expect to add Iron Python and Iron Ruby and will support other languages where people ask for it, particularly if they are able to pitch in and help.

We’ll remain platform-neutral and will add more platforms. When we use the term .NET, we don’t only refer to MS.NET, but also to other ECMA-compliant implementations of the CLI. We currently support MS.NET and Mono and expect to add other platforms in the future. NUnit already allows running tests under different runtime platforms by use of a command-line option or by selecting an option in the Gui. We’ll expand this to support running tests under the compact framework as well as Silverlight. We’ll maintain separate download packages to support the different platforms, as we now do for Windows and Mono.

We will continue to be IDE-neutral. We’ll continue to ship releases that run outside any IDE in order to reach as many people as possible. IDE-specific features - such as those now present for Visual Studio - will remain user-selectable and off by default. Any major IDE integration package - such as plugins or addins - will be packaged separately.

We will become more usable by other programs. We’d like to make it easy for other software to run tests programatically using NUnit. To that end, we will provide standard interfaces that make it easy for people to write their own clients. We can’t guarantee that those interfaces will never change, but we’ll make an effort to keep them reasonably stable.

NUnit will be internationalized and we’ll work with those who want to provide localized versions.

We will improve our ability to work with other open source tools for .NET. This includes mock object frameworks, windows and web forms testers, Visual Studio addins, etc. We’ll work with others to provide integrated installs of NUnit in combination with other tools. Some tools - those we use in our own tests - will be packaged with NUnit in the future.

Note: This is a copy of the original NUnit Vision document created for NUnit 2.4. It has been superceded by a [[new vision statement|NUnit Vision]]. </note>

Stated simply…

0.0.6.0.1 NUnit is a unit testing framework for developers working with the .NET framework.

On this page, we articulate the details of what that statement means to us. We don’t specify features here - see the Nunit Roadmap for that - but try to provide set of guidelines that will help us when we are considering what features to add.

0.0.7 A Bit More Detail

NUnit began as a unit testing framework and that’s still its primary role. Many people use NUnit for integration testing, acceptance testing, etc. We support this where it doesn’t interfere with our primary goal. That is, if a feature enables acceptance testing but makes it harder to do unit testing, we won’t implement it.

When we talk about unit testing, we use the term as it is used in XP and other agile development methods. This is different from its use in traditional QA. In particular, unit testing is done by the programmer. Testers may also want to use NUnit, but they are not our primary audience.

All of us are pretty excited about Test-Driven Development. However, we recognize that many programmers are not yet using TDD, but would like to write unit tests to run under NUnit. We support this use of NUnit, and will add features that are needed by this group, but not if the feature gets in the way of doing test-driven development with NUnit.

0.0.8 Future Directions

Taking the above into account along with the requests we’ve received from users and our own interests, here are some directions we expect the next few releases of NUnit to take

We’ll expand the available choices regarding where tests are run. For example, we may use multiple AppDomains or separate processes. Some tests may need to run under a particular subsystem or on a separate machine.

NUnit will be extensible. We can’t do everything for everybody but we will make it reasonably easy to add plugins to NUnit. The use of plugins is our main strategy for dealing with feature requests falling outside the basic unit-testing orientation of NUnit. In many cases, users will be able to implement a plugin that provides a special feature outside of our scope.

We’ll continue to focus on .NET development. Testing non-.NET apps through tests written in a .NET language is an acceptable use of NUnit, but we won’t give high priority to features that are needed only for that purpose.

We’ll remain language neutral. We’d like to work with any supported .NET languages. We will make special efforts to support C#, VB.NET, C++ and J#. We’ll support other languages (e.g. Delphi, Python) where people ask for it, particularly if they are able to pitch in and help.

We’ll remain platform-neutral and will add more platforms. When we use the term .NET, we don’t only refer to MS.NET, but also to other ECMA-compliant implementations of the CLI. We currently support MS.NET and Mono and expect to add other platforms in the future. We’ll support running tests under the compact framework. We’ll maintain separate download packages to support the different platforms, as we now do for Windows and Mono.

We will continue to be IDE-neutral. We’ll continue to ship releases that run outside any IDE in order to reach as many people as possible. IDE-specific features - such as those now present for Visual Studio - will remain user-selectable and off by default. Any major IDE integration package - such as plugins or addins - will be packaged separately.

We will become more usable by other programs. We’d like to make it easy for other software to run tests programatically using NUnit. To that end, we will provide standard interfaces that make it easy for people to write their own clients. We can’t guarantee that those interfaces will never change, but we’ll make an effort to keep them reasonably stable.

NUnit will be internationalized and we’ll work with those who want to provide localized versions.

We will improve our ability to work with other open source tools for .NET. This includes mock object frameworks, windows and web forms testers, Visual Studio addins, etc. We’ll work with others to provide integrated installs of NUnit in combination with other tools. Some tools - those we use in our own tests - will be packaged with NUnit in the future.

0.0.8.0.1 General
  • [[What is the purpose of this FAQ]]?
  • [[Who maintains this FAQ]]?
  • [[How do I contribute a question]]?
0.0.8.0.2 Getting Started
  • [[What is NUnit]]?
  • [[What packages are available]]?
  • [[How do I download and install NUnit]]?

###NUnit-Gui 0.1 - April 8, 2016

0.0.8.1 General

This is the initial release of the NUnit GUI runner. This is pre-alpha code we are releasing primarily for the purpose of getting feedback on the user interface itself. It is not suitable for use in production.

0.0.8.2 Features

  • The left-hand panel displays the tests in a variety of ways.
  • As a traditional NUnit tree
  • As a list of fixtures
  • As a list of test cases

  • When displaying a list of fixtures or test cases, you may select various groupings. This is similar to how the Visual Studio test explorer window works.

  • The right-hand panel displays information about the individual test selected in the left-hand panel. It has two tabs.
  • Properties - shows the information about the test (upper pane) and it’s result (lower pane)
  • XML - shows the XML representation for the test or test result

0.0.8.3 Issues Resolved

  • 17 Gui should highlight not-runnable and ignored tests immediately upon load
  • 19 CI Server for nunit-gui
  • 32 Load last test on startup
  • 33 Command-line with assemblies and projects specified
  • 34 Command-line run option
  • 35 Command-line noload option
  • 36 Gui Layout
  • 37 File Open Menu Item
  • 38 File Close Menu Item
  • 41 File Reload menu item
  • 44 Select Runtime Menu Item
  • 45 Test List Display
  • 46 Status Bar Content
  • 47 View Status Bar Menu Item
  • 48 Support checkboxes in the gui tree display
  • 49 Recent Files Menu
  • 55 Running tests from the gui
  • 58 Test details
  • 59 Invalid file (non-assembly) is displayed as an assembly.
  • 62 Group headings are not shown with correct icon
  • 66 Remove unsupported options from GuiOptions class
  • 68 Implement XML view of test details
  • 69 Reloading nunit project file causes engine error
  • 72 ReloadProject versus ReloadTests
  • 84 Add context menu to XmlView
  • 85 Display test result info in XmlView

0.1 UNDER CONSTRUCTION

The docs for the new NUnit Gui will be created here ###User Docs

  • [[NUnit|NUnit-Documentation]]
  • NUnit Gui
  • [[Release Notes|Gui-Release-Notes]]
  • [[Project Editor]]
  • Visual Studio
  • [[Test Adapter|Visual-Studio-Test-Adapter]]
  • [[Test Generator|Visual-Studio-Test-Generator]]
  • [[User FAQ]]

0.1.1 Developer Docs

  • [[NUnit Vision]]
  • [[NUnit Roadmap]]
  • [[Team Practices]]
  • [[Specifications]]
  • [[Developer FAQ]]

Action Attributes are a feature of NUnit designed to better enable composability of test logic.

Often when writing unit tests we have logic that we want to run upon certain events in the test cycle (e.g. SetUp, TearDown, FixtureSetUp, FixtureTearDown, etc.). NUnit has had the ability to execute code upon these events by decorating fixture classes and methods with the appropriate NUnit- provided attributes.

Action Attributes allow the user to create custom attributes to encapsulate specific actions for use before or after any test is run.

0.1.2 The Problem of Composability

Suppose we have some tests in multiple fixtures that need the same in-memory test database to be created and destroyed on each test run. We could create a base fixture class and derive each fixture that depends on the test from that class. Alternatively, we could create a SetUpFixture class at the level of a common namespace shared by each fixture.

This works fine, until we need some other reusable functionality, say the ability to configure or reset a ServiceLocator. We could put that functionality in the base fixture class or setup fixture, but now we’re mixing two different responsibilities into the base class. In the case of a setup fixture, this only works if all classes requiring both features are located in a common namespace. In some cases we may not want the test database, but we do want ServiceLocator configuration; and sometimes we want the opposite. Still other times we’ll want both - so we’d have to make the base class configurable.

If we now discover a third piece of functionality we need to reuse, like configuring the Thread’s CurrentPrincipal in arbitrary ways, the complexity of the solution very quickly. We’ve violated the Single Responsibility Principle and are suffering for it. What we really want is the ability to separate the different pieces of resuable test logic and compose them together as our tests need them.

0.1.3 Resolving the Problem

Action Attributes get us out of our bind. Consider this example:

[TestFixture, ResetServiceLocator]
public class MyTests
{
    [Test, CreateTestDatabase]
    public void Test1() { /* ... */ }

    [Test, CreateTestDatabase, AsAdministratorPrincipal]
    public void Test2() { /* ... */ }

    [Test, CreateTestDatabase, AsNamedPrincipal("charlie.poole")]
    public void Test3() { /* ... */ }

    [Test, AsGuestPrincipal]
    public void Test4() { /* ... */ }
}

Here we have used user-defined attributes to identify five different actions that we want to compose together in different ways for different tests: * ResetServiceLocator * CreateTestDatabase * AsAdministratorPrincipal * AsNamedPrincipal * AsGuestPrincipal

We can reuse these actions in other test fixtures, simply by decorating them with the appropriate attributes.without having to inherit from a base class. We can even develop and distribute a library of common test actions.

0.1.4 Implementing an Action Attribute

Action attributes are defined by the programmer. They implement the ITestAction interface, which is defined as follows:

public interface ITestAction
{
    void BeforeTest(ITest test);

    void AfterTest(ITest test);

    ActionTargets Targets { get; }
}

For convenience, you may derive your own action attribute from NUnit’s TestActionAttribute, an abstract class with virtual implementations of each member of the interface. Alternatively, you may derive from System.Attribute and implement the interface directly.

0.1.4.1 Action Targets

The value returned from the Targets property determines when the BeforeTest and AfterTest methods will be called. The ActionTargets enum is defined as follows:

[Flags]
public enum ActionTargets
{
    Default = 0,

    Test = 1,

    Suite = 2
}

When an attribute that returns ActionTargets.Suite is applied to either a class or a parameterized method, NUnit will execute the attribute’s BeforeTest method prior to executing the test suite and then execute the AfterTest method after the test suite has finished executing. This is similar to how the OneTimeSetUp and OneTimeTearDown attributes work.

On the other hand, when an attribute that returns ActionTargets.Test is used in the same situations, NUnit will execute the attribute’s BeforeTest method prior to each contained test case and the AfterTest method after each test case. This is similar to how the SetUp and TearDown attributes work.

Action attributes that return ActionTargets.Default target the particular code item to which they are attached. When attached to a method, they behave as if ActionTargets.Test had been specified. When attached to a class or assembly, they behave as if ActionTargets.Suite was returned.

0.1.4.2 ITest Interface

The BeforeTest and AfterTest methods are provided with information about the test that is about to run (before) or has just run (after). The ITest interface is an internal NUnit inteface to the representation of a test, which may be either a test case or a suite. The before and after methods may use the interface to decide what actions to take or retrieve information about the test.

public interface ITest : IXmlNodeBuilder
{
    // Gets the id of the test
    int Id { get; }
    
    // Gets the name of the test
    string Name { get; }
    
    // Gets the fully qualified name of the test
    string FullName { get; }
    
    // Gets the Type of the test fixture, if applicable, or
    // null if no fixture type is associated with this test.
    Type FixtureType { get; }
    
    // Gets a MethodInfo for the method implementing this test.
    // Returns null if the test is not implemented as a method.
    MethodInfo Method { get; }
    
    // Gets the RunState of the test, indicating whether it can be run.
    RunState RunState { get; }
    
    // Count of the test cases ( 1 if this is a test case )
    int TestCaseCount { get; }
    
    // Gets the properties of the test
    IPropertyBag Properties { get; }
    
    // Gets the parent test, if any.
    ITest Parent { get; }
    
    // Returns true if this is a test suite
    bool IsSuite { get; }
    
    // Gets a bool indicating whether the current test
    // has any descendant tests.
    bool HasChildren { get; }
    
    // Gets this test's child tests
    System.Collections.Generic.IList<ITest> Tests { get; }
}

0.1.5 Examples

The examples that follow all use the following sample Action Attribute:

[AttributeUsage(AttributeTargets.Method | AttributeTargets.Class |
                AttributeTargets.Interface | AttributeTargets.Assembly,
                AllowMultiple = true)]
public class ConsoleActionAttribute : Attribute, ITestAction
{
    private string _Message;

    public ConsoleActionAttribute(string message) { _Message = message; }

    public void BeforeTest(ITest test)
    {
        WriteToConsole("Before", test);
    }

    public void AfterTest(ITest test)
    {
        WriteToConsole("After", test);
    }

    public ActionTargets Targets
    {
        get { return ActionTargets.Test | ActionTargets.Suite; }
    }

    private void WriteToConsole(string eventMessage, ITest details)
    {
        Console.WriteLine("{0} {1}: {2}, from {3}.{4}.",
            eventMessage,
            details.IsSuite ? "Suite" : "Case",
            _Message,
            details.FixtureType != null ? details.FixtureType.Name : "{no fixture}",
            details.Method != null ? details.Method.Name : "{no method}");
    }
}

Note that the above Action Attribute returns the union of ActionTargets.Test and ActionTargets.Suite. This is permitted, but will probably not be the normal case. It is done here so we can reuse the attribute in multiple examples. The attribute takes a single constructor argument, a message, that will be used to write output to the console. All of the Before and After methods write output via the WriteToConsole method.

0.1.6 Method Attached Actions

0.1.6.1 Example 1 (applied to simple test method):

[TestFixture]
public class ActionAttributeSampleTests
{
    [Test][ConsoleAction("Hello")]
    public void SimpleTest()
    {
        Console.WriteLine("Test ran.");
    }
}
0.1.6.1.1 Console Output:
  Before Case: Hello, from ActionAttributeSampleTests.SimpleTest.
  Test ran.
  After Case: Hello, from ActionAttributeSampleTests.SimpleTest.

0.1.6.2 Example 2 (applied action twice to test method):

[TestFixture]
public class ActionAttributeSampleTests
{
    [Test] [ConsoleAction("Hello")]
    [ConsoleAction("Greetings")]
    public void SimpleTest()
    {
        Console.WriteLine("Test run.");
    }
}
0.1.6.2.1 Console Output:
  Before Case: Greetings, from ActionAttributeSampleTests.SimpleTest.
  Before Case: Hello, from ActionAttributeSampleTests.SimpleTest.
  Test run.
  After Case: Hello, from ActionAttributeSampleTests.SimpleTest.
  After Case: Greetings, from ActionAttributeSampleTests.SimpleTest.
0.1.6.2.2 Remarks

You are permitted to apply the same attribute multiple times. Note that the order in which attributes are applied is indeterminate, although it will generally be stable for a single release of .NET.

0.1.6.3 Example 3 (applied to a test method with test cases):

[TestFixture]
public class ActionAttributeSampleTests
{
    [Test] [ConsoleAction("Hello")]
    [TestCase("02")]
    [TestCase("01")]
    public void SimpleTest(string number)
    {
        Console.WriteLine("Test run {0}.", number);
    }
}
0.1.6.3.1 Console Output:
  Before Suite: Hello, from ActionAttributeSampleTests.SimpleTest.
  Before Case: Hello, from ActionAttributeSampleTests.SimpleTest.
  Test run 01.
  After Case: Hello, from ActionAttributeSampleTests.SimpleTest.
  Before Case: Hello, from ActionAttributeSampleTests.SimpleTest.
  Test run 02.
  After Case: Hello, from ActionAttributeSampleTests.SimpleTest.
  After Suite: Hello, from ActionAttributeSampleTests.SimpleTest.
0.1.6.3.2 Remarks

When one or more [TestCase] attributes are applied to a method, NUnit treats the method as a test suite.
You’ll notice that BeforeTest was run once before the suite and AfterTest was run once after it. In addition, BeforeTest and AfterTest are run again for each individual test case. Note that the order in which test cases are executed is indeterminate.

0.1.7 Type Attached Actions

0.1.7.1 Example 1:

[TestFixture] [ConsoleAction("Hello")]
public class ActionAttributeSampleTests
{
    [Test]
    public void SimpleTestOne()
    {
        Console.WriteLine("Test One.");
    }
    
    [Test]
    public void SimpleTestTwo()
    {
        Console.WriteLine("Test Two.");
    }
}
0.1.7.1.1 Console Output:
  Before Suite: Hello, from ActionAttributeSampleTests.{no method}.
  Before Case: Hello, from ActionAttributeSampleTests.SimpleTestOne.
  Test ran.
  After Case: Hello, from ActionAttributeSampleTests.SimpleTestOne.
  Before Case: Hello, from ActionAttributeSampleTests.SimpleTestTwo.
  Test ran.
  After Case: Hello, from ActionAttributeSampleTests.SimpleTestTwo.
  After Suite: Hello, from ActionAttributeSampleTests.{no method}.
0.1.7.1.2 Remarks

In this case, the class is the test suite. BeforeTest and AfterTest are run once each for this class and then again for each test.

0.1.7.2 Example 2 (attached to interface):

[ConsoleAction("Hello")]
public interface IHaveAnAction
{
}

[TestFixture]
public class ActionAttributeSampleTests : IHaveAnAction
{
    [Test] 
    public void SimpleTest()
    {
        Console.WriteLine("Test run.");
    }
}
0.1.7.2.1 Console Output:
  Before Suite: Hello, from ActionAttributeSampleTests.{no method}.
  Before Case: Hello, from ActionAttributeSampleTests.SimpleTest.
  Test run.
  After Case: Hello, from ActionAttributeSampleTests.SimpleTest.
  After Suite: Hello, from ActionAttributeSampleTests.{no method}.
0.1.7.2.2 Remarks

Action attributes can be applied to an interface. If a class marked with [TestFixture] implements an interface that has an action attribute applied to the interface, the class inherits the action attribute from the interface. It behaves as if you applied the action attribute to the class itself.

0.1.7.3 Example 3 (action attribute is applied to interface and attribute uses interface to provide data to tests):

[AttributeUsage(AttributeTargets.Interface)]
public class InterfaceAwareActionAttribute : TestActionAttribute
{
    private readonly string _Message;

    public InterfaceAwareActionAttribute(string message) { _Message = message; }

    public override void BeforeTest(ITest details)
    {
        IHaveAnAction obj = details.Fixture as IHaveAnAction;
        if(obj != null)
            obj.Message = _Message;
    }

    public override ActionTargets Targets
    {
        get { return ActionTargets.Test; }
    }
}

[InterfaceAwareAction("Hello")]
public interface IHaveAnAction { string Message { get; set; } }

[TestFixture]
public class ActionAttributeSampleTests : IHaveAnAction
{
    [Test] 
    public void SimpleTest()
    {
        Console.WriteLine("{0}, World!", Message);
    }

    public string Message { get; set; }
}
0.1.7.3.1 Console Output:
  Hello, World!
0.1.7.3.2 Remarks</h5>

Here we see a new action attribute, InterfaceAwareAction. This attribute uses the Fixture property of the TestDetails passed into BeforeTest and casts it to an interface, IHaveAnAction. If the fixture implements the IHaveAnAction interface, the attribute sets the Message property to the string passed into the constructor of the attribute. Since the attribute is applied to the interface, any class that implements this interface gets it’s Message property set to the string provided to the constructor of the attribute. This is useful when the action attribute provides some data or service to the tests.

Note that this attribute inherits from TestActionAttribute. It uses the default (do-nothing) implementation of AfterTest and overrides both BeforeTest and Target.

0.1.8 Assembly Attached Action

0.1.8.1 Example 1:

[assembly: ConsoleAction("Hello")]

[TestFixture]
public class ActionAttributeSampleTests
{
    [Test] 
    public void SimpleTest()
    {
        Console.WriteLine("Test run.");
    }
}
0.1.8.1.1 Console Output:
  Before Suite: Hello, from {no fixture}.{no method}.
  Before Case: Hello, from ActionAttributeSampleTests.SimpleTest.
  Test run.
  After Case: Hello, from ActionAttributeSampleTests.SimpleTest.
  After Suite: Hello, from {no fixture}.{no method}.
0.1.8.1.2 Remarks

The ConsoleAction attribute in this example is applied to the entire assembly. NUnit treats an assembly as a test suite (in fact, a suite of suites). Since the ConsoleAction attribute implements both ITestSuiteAction and ITestCaseAction, NUnit will run BeforeTest once before any tests are run in the assembly, and AfterTest after all tests are run in the assembly. Additionally, BeforeTest and AfterTest will be run for every test case in the assembly. It is unlikely that action attributes are applied to assemblies often. However, it is useful to build action attributes that ensure state gets cleaned up before and after each tests to prevent individual tests from affecting the outcome of other test. For example, if you have any static or cached data or services, an action attribute can be used to clean them up for each test. ###Out Of Date - Needs Rewrite NUnit identifies tests and fixtures of custom attributes. Additional attributes are used to identify test data, modify tests and control how they execute. NUnit 3.0 is changing how attributes are used and recognized internally, while maintaining the same API for the test writer. This specification describes how the new version will recognize and use attributes, the interfaces implemented by attributes and the class hierarchy of NUnit’s custom attributes.

0.1.9 Rationale

In the NUnit 2.x releases, attributes are recognized by the text of their name rather than using the attribute Type. This means that there is no need or the runner to have an actual reference to the nunit.framework assembly, where attributes are defined. This approach was taken in order to support multiple versions of the framework. Each new release of NUnit was able to run tests built against older frameworks, so long as the names remained the same.

However, this approach has some drawbacks:

  • As new attributes were added to the framework, it became necessary to also add the name of each attribute to the nunit core, which contained the code to load and run tests. The growing number of attributes in the framework now presents and extremely large surface area for the core to be aware of.
  • Inheritance from attributes does not work as users would expect with NUnit failing to automatically recognize those attributes as equivalent to their base classes. In a few cases, NUnit contains special code to scan the base classes of any attributes found but it is not easy for users to discover which attributes allow this.
  • NUnit must examine all attributes of a given class or method, comparing their names with the names it understands. This is quite inefficient and can slow down loading of assemblies containing large numbers of classes or methods.

By locating the code to load and run tests in the framework assembly (see [[Framework Boundary Change]]), NUnit 3.0 eliminates the first problem above. Test runners now need only be aware of a much narrower API. The design section below describes how we are handling the second and third issues.

0.1.10 Design

With NUnit 3.0, attributes are now only used within the framework assembly. Since all references are within that assembly, we can now handle them as Types rather than strings. All existing code that uses attribute names will be modified to use Type references, improving efficiency and allowing use of standard object-oriented programming techniques. The following sections provide an outline of the planned design. As implementation progresses, the code itself should be examined for updated information and further details.

0.1.11 Active Attributes

NUnit has a relatively small number of test types (e.g. TestFixture, TestMethod, Theory) and a relatively larger number of attributes. The current (2.x) design uses attributes almost exclusively as passive containers for their arguments and properties. All active code around attributes is located in the code around loading and running tests.

In almost all cases, it is possible to reverse this logic and have the attributes play an active part in modifying the operation of tests. For example, the NUnit 2.6 test building logic understands that a test description may be found as a property of a DescriptionAttribute, TestAttribute or TestFixtureAttribute. In NUnit 3.0, each of those attributes will simply be called upon to update the test under construction with whatever information they can supply using the IApplyToTest interface.

0.1.12 Attribute Hierarchy

With just a few exceptions, NUnit’s current set of attributes inherit from System.Attribute directly. Since we are now switching to use of the standard .NET reflection mechanism via attribute Types, it is useful to categorize attributes according to how and when they are used.

For example, at a certain point in the construction of tests, NUnit must scan the attributes on a method to apply all those that provide modifications to the test being built. This is currently done by looking at <b>all</b> attributes of the method but it will be much more convenient to only retrieve TestModificationAttributes.

The top levels of the Attribute hierarchy are listed here to the extent that they have been thought out. This will change as we implement it and will not necessarily remain stable in future releases because the hierarchy is considered to be an internal implementation detail.

  • NUnitAttribute
    • TestIdentificationAttribute
    • TestModificationAttribute
    • PropertyAttribute
    • TestActionAttribute
    • DataAttribute
    • TestCaseDataAttribute
    • ParameterDataAttribute

0.1.12.1 Interface Usage

While class inheritance of attributes is useful for retrieving groups of them and for sharing implementation, public access to capabilities of an attribute is better provided through an interface. For example, most TestModificationAttributes will implement the IApplyToTest interface, allowing them to be called upon to modify the test under construction.

public interface IApplyToTest
{
    void ApplyToTest(ITest test);
}

Interfaces are defined in the NUnit.Framework.Api namespace.

0.1.12.2 Properties Versus Separate Attributes

In some cases, there is a design choice to be made between use of a separate attribute or a property of the primary attribute for holding a particular of information. For example, the description of a test may be given as a property of TestAttribute or using a separate DescriptionAttribute. Recent versions of NUnit have deprecated use of properties in favor of separate attributes. Separate attributes provide a more orthogonal design and have been preferred for that reason.

However, the introduction of parameterized tests has necessitated adding properties to the TestCaseAttribute and other attributes, representing functionality that was provided by a separate attribute on non-parameterized tests. For example, a test is ignored using the IgnoreAttribute while a TestCase is ignored by setting the Ignore property of TestCaseAttribute to true.

For NUnit 3.0, we will try to provide some consistency of use for cases like this. This may result in “undeprecating” certain existing properties and creating new ones.

0.1.12.3 Attribute Targets

NUnit will continue to rely on the AttributeUsageAttribute being properly defined on each of its attributes. This allows us to use one attribute type for both methods and classes while still counting on a given attribute only appearing in the proper context.

0.1.13 Unresolved Issues

One problem that is created by the new approach is that NUnit will no longer “just work” with tests compiled against framework versions prior to 3.0. This requires the NUnit test execution engine to use the old approach when dealing with tests compiled against a pre-3.0 framework version. This specification does not cover the execution engine, so the problem is not discussed further here.

NUnit 2.6 supports six types of addins, all of which are being removed from NUnit 3.0: * SuiteBuilders * TestCaseBuilders * TestDecorators * TestCaseProviders * DataPointProviders * EventListeners

0.1.14 General Approach

The addin design for NUnit 2.6 was intended to extend to the console and gui runners in addition to the framework. However, this was never implemented and all six of the existing addin types apply to the framework only.

In NUnit 3.0, the functions provided by these addins are being taken over by the use of custom attributes. NUnit 3.0 attributes are generally active. That is, they contain code to perform the function they call for rather than simply serving as markers to be interpreted by the runner.

0.1.14.1 Advantages

In general, all the same capabilities will be present in NUnit 3.0 and will be much more easily accessible to those who create extensions. Currently, creating an extension is complex and error prone. Use of active attributes generally involves one of two approaches:

  1. Derive the new attribute class from an existing NUnit base that provides the needed facilities. For example, a new data-providing attribute might derive from DataAttribute.

  2. Derive directly from NUnitAttribute, the base of the NUnit attribute hierarchy, and implement one or more interfaces that perform the desired function.

0.1.14.2 Limitations

All existing addins will need to be re-implemented as custom attributes. They will not work in NUnit 3.0.

Addins not based on custom attributes are no longer possible. In NUnit 2.6, for example, it was possible to write an addin that defined tests based on the name of a method, e.g.: methods beginning with “Test”. This sort of extension will no longer be possible in the NUnit 3.0 framework. However, this does not seem to be a big problem, since virtually all addins that we know about have been based on attributes.

0.1.15 Implementation

Because parts of NUnit are implemented as internal addins, it’s not possible to simply remove all addin support at once. Many things would stop working if we did this. Therefore, we will refactor code for each of the internal types to conform to the new design, only removing the overall addin framework when this is complete.

The remaining sections of this spec deal with how each of the addin types is being replaced. The order of the sections reflects the order in which we are implementing the changes. The implementation status of each of the types is shown in parentheses.

This spec refers to a number of interfaces that form part of the NUnit framework. Until a technical note covering these interfaces is published, please rely on the source code for documentation.

Some of the sections that follow have not had all their design work completed, so the degree of detail varies among them. More information will be added as work progresses.

0.1.15.1 TestDecorators

Status: REMOVED

TestDecorators in NUnit 2.6 could do one of three things:

  1. Modify the properties of the test or the execution context
  2. Drop the test by returning null.
  3. Replace the test, possibly wrapping the old test within the new one.

In NUnit 3.0, each of these is implemented differently.

To modify the properties of the test, create an attribute that implements IApplyToTest.

To modify the execution context, implement IApplyToContext. This is a new capability.

It is no longer possible - but not necessary either - to replace the test. In NUnit 3.0, the test object does not execute itself. Instead, a series of commands is created, which carry out the necessary operations to run the test. An attribute that implements ICommandDecorator is able to contribute to add additional commands to be executed when running the test.

It is no longer possible to simply eliminate the test. Once created, a test will always appear in the UI, for example. However, by use of a command decorator, it is possible to prevent the test from executing.

0.1.15.2 DataPointProviders

Status: Removed

NUnit 2.6 has two built-in providers of data for individual parameters of test methods: * ParameterDataProvider gets data from attributes that appear directly on the parameter. * DatapointProvider gets data from DataPoint and DataPointSource attributes, which appear on the data source rather than on the parameter.

In the current 3.0 code, ParameterDataProvider actually delegates its work to the attributes, which must implement IParameterDataSource. In NUnit 3.0, users will be able to create additional attributes implementing this interface and they will be used as data sources.

On the other hand, DataPointProvider contains the code for accessing the data specified by the DataPoint and DataPointSource attributes, which are simply used as markers. This is because the attributes do not have a reference to the member on which they are placed. It would be necessary to introduce a method to provide them with this reference in order to do the processing within the attribute and this would appear to introduce otherwise unneeded code with no particular benefit. Consequently, for the initial implementation of NUnit 3.0, it will not be possible to create custom attributes that work in similar fashion to DataPoint or DataPointSource. If the need arises, some interface may be introduced.

Since DataPointProviders are only called from within TestCaseProviders, implementation of these changes may need to be interleaved with changes related to TestCaseProviders. See the next section for details.

0.1.15.3 TestCaseProviders

Status: REMOVED

NUnit 2.6 currently has two built-in TestCaseProviders: * DataAttributeTestCaseProvider gets test case data from any DataAttribute, such as TestCaseAttribute. * CombinatorialTestCaseProvider creates test cases by combining parameter data from a DataPointProvider.

Currently, DataAttributeTestCaseProvider delegates all the work to the DataAttribute. This is the desired approach. Users implementing a new custom data attribute may inherit from DataAttribute or implement an interface.

CombinatorialTestCaseProvider works differently. It instantiates one of three available CombiningStrategytypes, based on attributes appearing on the method. The CombiningStrategy is used to generate test cases from the available parameter data.

For NUnit 3.0, the work of combining parameter data into cases will be moved into the CombiningStrategy attributes. An interface will be defined and users will be able to create new combining strategies by defining a custom attribute that implements the interface.

0.1.15.4 EventListeners

Status: REMOVED

EventListeners implement the ITestListener interface and are notified when important events in the life of a test occur. Almost all the functions of an EventListener can already be emulated in NUnit 2.6 by use of an ActionAttribute. ActionAttributes are not yet implemented in NUnit 3.0.

Consequently, the first step in removing EventListeners is to implement ActionAttributes. Some changes may be made in the process and a separate spec will be written to describe them.

The only function of EventListeners that cannot be duplicated by ActionAttributes at this time is the capture of text output from the test. A new approach will be designed for this purpose.

0.1.15.5 TestCaseBuilders

Status: REMOVED

NUnit 2.6 has one built-in TestCaseBuilder, NUnitTestCaseBuilder, which implements ITestCaseBuilder2. All TestCaseBuilders must implement either ITestCaseBuilder or ITestCaseBuilder2.

In NUnit 3.0, the interface and its implementation will be moved into the attributes that designate a test. For example, TestAttribute will actually build a test case. This may be done using ITestCaseBuilder2 or a new interface may be designed. Users wishing to create a new kind of test case will need to define a new attribute, which implements the interface.

A few issues need to be resolved:

  1. Some attributes, such as TestCaseAttribute or TestCaseSourceAttribute serve dual functions: they both mark a test case and provide data for the test case. Some combination of the code for generating test cases with that for creating tests is likely to be required.
  2. For backward compatibility, an extra TestAttribute accompanying one or more TestCaseAttributes should not generate an additional TestCase.

0.1.15.6 SuiteBuilders

Status: REMOVED

NUnit 2.6 has one built-in SuiteBuilder, which implements the required ISuiteBuilder interface.

Similarly to what is being done for test cases, the building of a test fixture will be moved into the TestFixture attribute. The supporting interface may need to be redefined. Users may create new types of test fixtures by defining a new attribute, which implements the interface.

Similar issues as with TestCaseBuilder must be resolved, in addition to a few others:

  1. TestFixtureAttribute both marks a test case and provides arguments used to construct the fixture.
  2. Extra TestFixtureAttributes in the hierarchy must be ignored.
  3. We want to add similar data-generation capabilities to TestFixture as exist for methods using TestCaseSource.
  4. We want to allow “TheoryFixtures”, which combine multiple related Theories into a single fixture.
  5. We want to support an AbstractTestFixtureAttribute, which marks a class as the base of a fixture hierarchy but not a fixture itself. AllItemsConstraint applies a constraint to each item in an IEnumerable, succeeding only if all of them succeed. An exception is thrown if the actual value passed does not implement IEnumerable.

<h4>Constructor</h4>

AllItemsConstraint(Constraint itemConstraint)

<h4>Syntax</h4>

Is.All...
Has.All...

<h4>Examples of Use</h4>

int[] iarray = new int[] { 1, 2, 3 };
string[] sarray = new string[] { "a", "b", "c" };
Assert.That( iarray, Is.All.Not.Null );
Assert.That( sarray, Is.All.InstanceOf<string>() );
Assert.That( iarray, Is.All.GreaterThan(0) );
Assert.That( iarray, Has.All.GreaterThan(0) );

AndConstraint combines two other constraints and succeeds only if they both succeed.

<h4>Constructor</h4>

AndConstraint(Constraint left, Constraint right)

<h4>Syntax</h4>

<Constraint>.And.<Constraint>

<h4>Examples of Use</h4>

Assert.That( 2.3, Is.GreaterThan( 2.0 ).And.LessThan( 3.0 ) );

0.1.15.7 See also…

  • [[OrConstraint]]

The ApartmentAttribute is used on a test method, class or assembly to specify that the tests should be run in a particular apartment, either the STA or the MTA.

When running tests in parallel, the test is simply scheduled to execute from a queue that uses the apartment specified. When the parallel feature is not in use, it causes creation of a new thread if the parent test is not already running in the correct apartment.

When this attribute is not specified, tests run in the MTA.

This attribute replaces the RequiresMTA and RequiresSTA attributes, which are now considered obsolete.

0.1.15.8 Assembly Level Examples


// All the tests in this assembly will use the MTA by default. Since
// this is the general default, the attribute is not actually needed.
[assembly:Apartment(ApartmentState.MTA)]

...

// All the tests in this assembly will use the STA by default
[assembly:Apartment(ApartmentState.STA)]

0.1.15.9 Test Fixture Examples


// TestFixture requiring use of the MTA. The attribute is not 
// needed unless the STA was specified at a higher level.
[TestFixture, Apartment(ApartmentState.MTA)]
public class FixtureRequiringMTA
{
  // All tests in the fixture will run in the MTA.
}

// TestFixture requiring use of the STA.
[TestFixture, Apartment(ApartmentState.STA)]
public class FixtureRequiringSTA
{
  // All tests in the fixture will run in the STA.
}

0.1.15.10 Test Method Examples

[TestFixture]
public class AnotherFixture
{
  [Test, Apartment(ApartmentState.MTA)]
  public void TestRequiringMTA()
  {
    // This test will run in the MTA.
  }
  
  [Test, Apartment(ApartmentState.STA)]
  public void TestRequiringSTA()
  {
    // This test will run in the STA.
  }
}

0.1.15.11 See also…

  • [[RequiresThread Attribute]]

0.1.16 DRAFT

This specification describes the overall layered architecture of the NUnit Platform. Details of individual components are covered only as they are relevant to the overall architecture.

0.1.17 Layers

The basic architecture of the NUnit Extended testing platform consists of three layers:

  • [[Test Runner Layer|Architectural-Overview#test-runner-layer]]
  • [[Test Engine Layer|Architectural-Overview#test-engine-layer]]
  • [[Framework Layer|Architectural-Overview#framework-layer]]

The layering can be seen in the diagram below.

This is the original architectural document, which is a bit out of date with regard to some details. A replacement is being created.

[[images/nunit-xtp-2008.png]]

0.1.17.1 Test Runner Layer

The Test Runner or UI layer contains various runners, some provided by the NUnit team, others by independent projects leveraging the NUnit platform. Some runners are stand-alone programs, while others are tasks or plugins running under the control of an IDE or other application. This diversity of runners is part of the reason we refer to NUnit 3.0 as a Testing Platform – we expect many different runners to come into existence and will facilitate their development by providing reusable controls for several key environments.

Programs in this layer are able to participate in the NUnit platform plugin architecture, providing extension points that allow them to be extended. Plugins at this level will usually add some functionality to the UI. Some of them may be standalone while others may require the presence of specific test engine plugins in order to operate.

The NUnit project will continue to provide both a console runner and a WinForms-based GUI with extended capabilities those in NUnit 2.6. In addition, two new GUI runners will be developed, one based on WPF, the other on GTK#.

We’ll work with the NAnt project to provide updates to the NAnt task for use with NUnit 3.0, with the goal of keeping that task current as new versions of NUnit are released. We will provide an NUnit plugin for the new Gallio platform.

In the area of IDE integration, we will deliver a Visual Studio addin or package for running NUnit tests. Since other folks are already providing open source plugins for SharpDevelop and MonoDevelop, we’ll work with them to ensure compatibility.

There is a long-standing need for a runner that executes tests in an actual or simulated web server environment. While good practice calls for extracting as much functionality as possible into separately testable assemblies, more complex applications often contain code which can only be tested in such an environment. For that reason, NUnit 3.0 will feature a web runner, which allows tests to be executed on a web server, reporting results back to the desktop.

Each of the runners will have the option of participating in the NUnit plugin architecture and using functionality provided by NUnit to whatever degree desired. For the runners developed directly by the NUnit team, this capability will be used to the max, allowing others to add GUI features that function by themselves or in conjunction with other plugins operating at the level of the test engine.

0.1.17.2 Test Engine Layer

The Test Engine Layer is the core of the NUnit platform. It provides a public API for use by applications that want to locate, load and run tests and display test results. Many aspects of the Test Engine are already present in NUnit 2.6, while others are new. I’ll focus on the new features here.

NUnit 2.6 already supports running tests in a separate process, allowing selection of the CLR version under which the test is to be run. NUnit 3.0 will extend this feature to allow test processes to run on other machines. It will also support distribution of test execution across a network of remote Test Agents. This facility is intended to be used in several distinct scenarios: * Simple load sharing when test execution time is excessive * Testing applications on multiple platforms * Testing applications, which are themselves distributed A great deal of distributed function is already present in NUnit 2.6 through pNUnit, the distributed testing runner contributed to our project by Codice Software. With NUnit 3.0, we plan to integrate this function more completely into NUnit, allowing them to be executed by any runner that uses the NUnit Test Engine.

NUnit will support performance and coverage analysis, test result reporting and maintenance of a history of test results. These features will function only as called upon by the user. That is, we will not carry on data collection activities with the potential of impacting performance when the user simply wants to run tests.

Through use of plugins, NUnit will be able to support a wide variety of test types beyond low-level, isolated programmer tests. What is available in this area will be dependent on the interests of users and their willingness to contribute their efforts to creating them. Examples of the kinds of things we envision, some of which we will provide directly, are: * Randomization of test execution order * Deterministic test ordering for integration tests * Parameterized (data-driven) tests * Transactional test behavior * Timed tests and parameterized timeout failures * Dynamic test generation based on data input * Repetitive test execution * Tests written as non-managed code * Test generation for legacy applications

0.1.17.3 Framework Layer

In NUnit 3.0, the NUnit framework itself – the assembly that is referenced by user tests – will be split along two dimensions. First, there will be separate framework assemblies for different version levels of the Common Language Runtime. By splitting the framework in this way, we will be able to take advantage of newer features, and allow users to take advantage of them, without compromising basic support for older runtimes. For a list of what we currently expect to support see the [[Framework Distributions]] page.

The second split we plan is between the core framework capabilities and the syntactic features that make it easy to access those features. A key example of this is the fluent interface introduced in NUnit 2.4 – the “Assert.That” syntax. One thing we learned through that experiment is that the same syntactic “sugar” does not work well for different language environments. Many of the 2.4/2.5/2.6 constructs are unusable or very difficult to use in other languages – C++ for example. By a combination of separate namespaces and separate assemblies, we will allow users to select the appropriate syntax for the work they are doing. Other people will be able to build on the syntax we provide or create entirely new syntactic overlays for their own purposes.

Through use of plugins in the Test Engine layer, NUnit will be able to recognize, load and run tests written using other frameworks. Our focus will be on facilitating the creation of plugins in support of external frameworks by the projects that produce those frameworks or by interested users. For frameworks where that sort of support is not possible – commercial frameworks, for example – we have the option of creating the plugins ourselves.

In some cases, individuals with an idea for a new framework may be able to create them more simply by writing an adapter on top of the NUnit framework itself. We will encourage and support this by giving priority to framework modifications that provide the necessary infrastructure for such projects.

NUnit will also provide or re-package some framework extensions for specific types of applications, including Windows Forms development, WPF development, web page access, XML testing, performance measurement and load testing. At this point, it is not yet possible to state which of these will involve existing third-party packages and which will be developed from scratch, because discussions are ongoing.

The NUnitLite framework will be supported running on devices and reporting results back to the Test Engine through a communications channel.

Various popular mock frameworks will be supported. NUnit’s own self-tests now use the NSubstitute framework, which is packaged with NUnit, but users may make use of whatever mocking framework they prefer.

0.1.17.4 See also…

You can also view the [[Original Architectural Overview Document | NUnit-3.0-Architecture-(2009)]] created for NUnit 3.0 in 2009.

NUnit isolates test assemblies from its own code and from one another by use of separate Processes and AppDomains.

By default, NUnit loads each test assembly into a separate <b>Process</b> under the control of the [NUnit Agent] program. This allows NUnit to ensure that each assembly is loaded in the environment for which it was built. Within the agent process, NUnit’s own code runs in the primary
<b>AppDomain</b> while the tests run in a separate <b>AppDomain</b>.

If desired, multiple test assemblies may be loaded into the same process and even the same domain by use of the <b>-process</b> and <b>-domain</b> command-line options. See [[Console Command Line]].

Assert.AreEqual tests whether the two arguments are equal.

Assert.AreEqual( double expected, double actual, double tolerance );
Assert.AreEqual( double expected, double actual, double tolerance, 
                 string message, params object[] parms );

Assert.AreEqual( object expected, object actual );
Assert.AreEqual( object expected, object actual, 
                 string message, params object[] parms );

0.1.17.5 Comparing Numerics of Different Types

The method overloads that compare two objects make special provision so that numeric values of different types compare as expected. This assert succeeds:

Assert.AreEqual( 5, 5.0 );

0.1.17.6 Comparing Floating Point Values

Values of type float and double are normally compared using an additional argument that indicates a tolerance within which they will be considered as equal. The value of GlobalSettings.DefaultFloatingPointTolerance is used if a third argument is not provided.

Special values are handled so that the following Asserts succeed:

Assert.AreEqual( double.PositiveInfinity, double.PositiveInfinity );
Assert.AreEqual( double.NegativeInfinity, double.NegativeInfinity );
Assert.AreEqual( double.NaN, double.NaN );

0.1.17.7 Comparing Arrays and Collections

NUnit is able to compare single-dimensioned arrays, multi-dimensioned arrays, nested arrays (arrays of arrays) and collections. Two arrays or collections are considered equal if they have the same dimensions and if each pair of corresponding elements is equal.

NUnit 3.0 adds the ability to compare generic collections and dictionaries.

0.1.17.8 See also…

Assert.AreNotEqual tests whether the two arguments are unequal.

Assert.AreNotEqual( object expected, object actual );
Assert.AreNotEqual( object expected, object actual, 
                    string message, params object[] parms );

See [[Assert.AreEqual]] for details of how NUnit performs equality comparisons.

0.1.17.9 See also…

Assert.AreNotSame tests that the two arguments do not reference the same object.

Assert.AreNotSame( object expected, object actual );
Assert.AreNotSame( object expected, object actual, 
                   string message, params object[] parms );

0.1.17.10 See also…

Assert.AreSame tests that the two arguments reference the same object.

Assert.AreSame( object expected, object actual );
Assert.AreSame( object expected, object actual,
                string message, params object[] parms );

0.1.17.11 See also…

  • SameAs Constraint Assert.Catch is similar to Assert.Throws but will pass for an exception that is derived from the one specified.
Exception Assert.Catch( TestDelegate code );
Exception Assert.Catch( TestDelegate code, 
                        string message, params object[] parms );

Exception Assert.Catch( Type expectedExceptionType, TestDelegate code );
Exception Assert.Catch( Type expectedExceptionType, TestDelegate code, 
                        string message, params object[] parms );

T Assert.Catch<T>( TestDelegate code );
T Assert.Catch<T>( TestDelegate code, 
                 string message, params object[] parms );

0.1.17.12 See also…>

  • [[Assert.CatchAsync]]
  • [[Assert.Throws]]
  • [[Assert.ThrowsAsync]]
  • [[ThrowsConstraint]]

Assert.CatchAsync is similar to [[Assert.ThrowsAsync]] but will pass for an exception that is derived from the one specified.

Exception Assert.CatchAsync( AsyncTestDelegate code );
Exception Assert.CatchAsync( AsyncTestDelegate code, 
                        string message, params object[] parms );

Exception Assert.CatchAsync( Type expectedExceptionType, AsyncTestDelegate code );
Exception Assert.CatchAsync( Type expectedExceptionType, AsyncTestDelegate code, 
                        string message, params object[] parms );

T Assert.CatchAsync<T>( AsyncTestDelegate code );
T Assert.CatchAsync<T>( AsyncTestDelegate code, 
                 string message, params object[] parms );

0.1.17.13 See also…

  • [[Assert.Catch]]
  • [[Assert.Throws]]
  • [[Assert.ThrowsAsync]]
  • [[ThrowsConstraint]]

Assert.Contains is used to test whether an object is contained in a collection.

Assert.Contains( object anObject, ICollection collection );
Assert.Contains( object anObject, ICollection collection,
                 string message, params object[] parms );

0.1.17.14 See also…

  • [[CollectionContainsConstraint]] Assert.DoesNotThrow verifies that the delegate provided as an argument does not throw an exception. See [[Assert.DoesNotThrowAsync]] for asynchronous code.
void Assert.DoesNotThrow( TestDelegate code );
void Assert.DoesNotThrow( TestDelegate code, 
                          string message, params object[] parms );

0.1.17.15 See also…

  • [[Assert.Throws]]
  • [[ThrowsConstraint]]

Assert.DoesNotThrowAsync verifies that the delegate provided as an argument does not throw an exception. See [[Assert.DoesNotThrow]] for synchronous code.

void Assert.DoesNotThrowAsync( AsyncTestDelegate code );
void Assert.DoesNotThrowAsync( AsyncTestDelegate code, 
                          string message, params object[] parms );

0.1.17.16 See also…

  • [[Assert.ThrowsAsync]]
  • [[ThrowsConstraint]]

The Assert.Fail method provides you with the ability to generate a failure based on tests that are not encapsulated by the other methods. It is also useful in developing your own project-specific assertions.

Assert.Fail();
Assert.Fail( string message, params object[] parms );s

Here’s an example of its use to create a private assertion that tests whether a string contains an expected value.

public void AssertStringContains( string expected, string actual )
{
    AssertStringContains( expected, actual, string.Empty );
}

public void AssertStringContains( string expected, string actual, 
                                  string message )
{
    if ( actual.IndexOf( expected ) < 0 )
        Assert.Fail( message );
}

0.1.17.17 See also…

  • [[Assert.Pass]]
  • [[Assert.Ignore]]
  • [[Assert.Inconclusive]]

Assert.False and Assert.IsFalse test that the specified condition is false. The two forms are provided for compatibility with past versions of NUnit and NUnitLite.

Assert.False( bool condition );
Assert.False( bool condition, string message, params object[] parms );

Assert.IsFalse( bool condition );
Assert.IsFalse( bool condition, string message, params object[] parms );

0.1.17.18 See also…

Assert.Greater tests whether one object is greater than than another. Contrary to the normal order of Asserts, these methods are designed to be read in the “natural” English-language or mathematical order. Thus Assert.Greater( x, y ) asserts that x is greater than y ( x > y ).

Assert.Greater( int arg1, int arg2 );
Assert.Greater( int arg1, int arg2, 
                string message, params object[] parms );

Assert.Greater( uint arg1, uint arg2 );
Assert.Greater( uint arg1, uint arg2, 
                string message, params object[] parms );

Assert.Greater( long arg1, long arg2 );
Assert.Greater( long arg1, long arg2, 
                string message, params object[] parms );

Assert.Greater( ulong arg1, ulong arg2 );
Assert.Greater( ulong arg1, ulong arg2, 
                string message, params object[] parms );

Assert.Greater( decimal arg1, decimal arg2 );
Assert.Greater( decimal arg1, decimal arg2, 
                string message, params object[] parms );

Assert.Greater( double arg1, double arg2 );
Assert.Greater( double arg1, double arg2, 
                string message, params object[] parms );

Assert.Greater( float arg1, float arg2 );
Assert.Greater( float arg1, float arg2, 
                string message, params object[] parms );

Assert.Greater( IComparable arg1, IComparable arg2 );
Assert.Greater( IComparable arg1, IComparable arg2, 
                string message, params object[] parms );

0.1.17.19 See also…

Assert.GreaterOrEqual tests whether one object is greater than or equal to another. Contrary to the normal order of Asserts, these methods are designed to be read in the “natural” English-language or mathematical order. Thus Assert.GreaterOrEqual( x, y ) asserts that x is greater than or equal to y ( x >= y ).

Assert.GreaterOrEqual( int arg1, int arg2 );
Assert.GreaterOrEqual( int arg1, int arg2, 
                       string message, params object[] parms );

Assert.GreaterOrEqual( uint arg1, uint arg2 );
Assert.GreaterOrEqual( uint arg1, uint arg2, 
                       string message, params object[] parms );

Assert.GreaterOrEqual( long arg1, long arg2 );
Assert.GreaterOrEqual( long arg1, long arg2, 
                       string message, params object[] parms );

Assert.GreaterOrEqual( ulong arg1, ulong arg2 );
Assert.GreaterOrEqual( ulong arg1, ulong arg2, 
                       string message, params object[] parms );

Assert.GreaterOrEqual( decimal arg1, decimal arg2 );
Assert.GreaterOrEqual( decimal arg1, decimal arg2, 
                       string message, params object[] parms );

Assert.GreaterOrEqual( double arg1, double arg2 );
Assert.GreaterOrEqual( double arg1, double arg2, 
                       string message, params object[] parms );

Assert.GreaterOrEqual( float arg1, float arg2 );
Assert.GreaterOrEqual( float arg1, float arg2, 
                       string message, params object[] parms );

Assert.GreaterOrEqual( IComparable arg1, IComparable arg2 );
Assert.GreaterOrEqual( IComparable arg1, IComparable arg2, 
                       string message, params object[] parms );

0.1.17.20 See also…

The Assert.Ignore method provides you with the ability to dynamically cause a test or suite to be ignored at runtime. It may be called in a test, setup or fixture setup method. We recommend that you use this only in isolated cases. The category facility is provided for more extensive inclusion or exclusion of tests or you may elect to simply divide tests run on different occasions into different assemblies.

Assert.Ignore();
Assert.Ignore( string message, params object[] parms );

0.1.17.21 See also…

  • [[Assert.Pass]]
  • [[Assert.Fail]]
  • [[Assert.Inconclusive]]

The Assert.Inconclusive method indicates that the test could not be completed with the data available. It should be used in situations where another run with different data might run to completion, with either a success or failure outcome.

Assert.Inconclusive();
Assert.Inconclusive( string message, params object[] parms );

0.1.17.22 See also…

  • [[Assert.Pass]]
  • [[Assert.Fail]]
  • [[Assert.Ignore]]

Assert.IsAssignableFrom succeeds if the object provided may be assigned a value of the expected type.

Assert.IsAssignableFrom( Type expected, object actual );
Assert.IsAssignableFrom( Type expected, object actual, 
                         string message, params object[] parms );
Assert.IsAssignableFrom<T>( object actual );
Assert.IsAssignableFrom<T>( object actual, 
                            string message, params object[] parms );

0.1.17.23 See also…

Assert.IsEmpty may be used to test either a string or a collection or IEnumerable. When used with a string, it succeeds if the string is the empty string. When used with a collection, it succeeds if the collection is empty.

Assert.IsEmpty( string aString );
Assert.IsEmpty( string aString, string message, params object[] args );

Assert.IsEmpty( IEnumerable collection );
Assert.IsEmpty( IEnumerable collection, string message,
                params object[] args );

Note: When used with an IEnumerable that is not also an ICollection, Assert.IsEmpty attempts to enumerate the contents. It should not be used in cases where this results in an unwanted side effect.

0.1.17.24 See also…

Assert.IsInstanceOf succeeds if the object provided as an actual value is an instance of the expected type.

Assert.IsInstanceOf( Type expected, object actual );
Assert.IsInstanceOf( Type expected, object actual, 
                         string message, params object[] parms );
Assert.IsInstanceOf<T>( object actual );
Assert.IsInstanceOf<T>( object actual, 
                        string message, params object[] parms );            

0.1.17.25 See also…

Assert.IsNaN tests that the specified double value is NaN.

Assert.IsNaN( double aDouble );
Assert.IsNaN( double aDouble, string message, params object[] parms );

0.1.17.26 See also…

Assert.IsNotAssignableFrom succeeds if the object provided may not be assigned a value of the expected type.

Assert.IsNotAssignableFrom( Type expected, object actual );
Assert.IsNotAssignableFrom( Type expected, object actual, 
                            string message, params object[] parms );
Assert.IsNotAssignableFrom<T>( object actual );
Assert.IsNotAssignableFrom<T>( object actual, 
                               string message, params object[] parms );

See also… * Type Constraints

Assert.IsNotEmpty may be used to test either a string or a collection or IEnumerable. When used with a string, it succeeds if the string is not the empty string. When used with a collection, it succeeds if the collection is not empty.

Assert.IsNotEmpty( string aString );
Assert.IsNotEmpty( string aString, string message, params object[] args );

Assert.IsNotEmpty( IEnumerable collection );
Assert.IsNotEmpty( IEnumerable collection, string message,
                   params object[] args );

Note: When used with an IEnumerable that is not also an ICollection, Assert.IsEmpty attempts to enumerate the contents. It should not be used in cases where this results in an unwanted side effect.

0.1.17.27 See also…

  • Condition Constraints Assert.IsNotInstanceOf succeeds if the object provided as an actual value is not an instance of the expected type.
Assert.IsNotInstanceOf( Type expected, object actual );
Assert.IsNotInstanceOf( Type expected, object actual, 
                         string message, params object[] parms );
Assert.IsNotInstanceOf<T>( object actual );
Assert.IsNotInstanceOf<T>( object actual, 
                        string message, params object[] parms );            

0.1.17.28 See also…

Assert.Less tests whether one object is less than than another. Contrary to the normal order of Asserts, these methods are designed to be read in the “natural” English-language or mathematical order. Thus Assert.Less( x, y ) asserts that x is less than y ( x < y ).

Assert.Less( int arg1, int arg2 );
Assert.Less( int arg1, int arg2, string message, params object[] parms );
                
Assert.Less( uint arg1, uint arg2 );
Assert.Less( uint arg1, uint arg2, string message, params object[] parms );
                
Assert.Less( long arg1, long arg2 );
Assert.Less( long arg1, long arg2, string message, params object[] parms );

Assert.Less( ulong arg1, ulong arg2 );
Assert.Less( ulong arg1, ulong arg2, string message, params object[] parms );

Assert.Less( decimal arg1, decimal arg2 );
Assert.Less( decimal arg1, decimal arg2, 
             string message, params object[] parms );
                
Assert.Less( double arg1, double arg2 );
Assert.Less( double arg1, double arg2, 
             string message, params object[] parms );
                
Assert.Less( float arg1, float arg2 );
Assert.Less( float arg1, float arg2, 
             string message, params object[] parms );
                
Assert.Less( IComparable arg1, IComparable arg2 );
Assert.Less( IComparable arg1, IComparable arg2, 
             string message, params object[] parms );

0.1.17.29 See also…

Assert.LessOrEqual tests whether one object is less than or equal to another. Contrary to the normal order of Asserts, these methods are designed to be read in the “natural” English-language or mathematical order. Thus Assert.LessOrEqual( x, y ) asserts that x is less than or equal to y ( x <= y ).

Assert.LessOrEqual( int arg1, int arg2 );
Assert.LessOrEqual( int arg1, int arg2, 
                    string message, params object[] parms );
                
Assert.LessOrEqual( uint arg1, uint arg2 );
Assert.LessOrEqual( uint arg1, uint arg2, 
                    string message, params object[] parms );
                
Assert.LessOrEqual( long arg1, long arg2 );
Assert.LessOrEqual( long arg1, long arg2, 
                    string message, params object[] parms );

Assert.LessOrEqual( ulong arg1, ulong arg2 );
Assert.LessOrEqual( ulong arg1, ulong arg2, 
                    string message, params object[] parms );

Assert.LessOrEqual( decimal arg1, decimal arg2 );
Assert.LessOrEqual( decimal arg1, decimal arg2, 
                    string message, params object[] parms );
                
Assert.LessOrEqual( double arg1, double arg2 );
Assert.LessOrEqual( double arg1, double arg2, 
                    string message, params object[] parms );
                
Assert.LessOrEqual( float arg1, float arg2 );
Assert.LessOrEqual( float arg1, float arg2, 
                    string message, params object[] parms );
                
Assert.LessOrEqual( IComparable arg1, IComparable arg2 );
Assert.LessOrEqual( IComparable arg1, IComparable arg2, 
                    string message, params object[] parms );

0.1.17.30 See also…

Assert.NotNull and Assert.IsNotNull test that the specified object is non-null. The two forms are provided for compatibility with past versions of NUnit and NUnitLite.

Assert.NotNull( object anObject );
Assert.NotNull( object anObject, string message, params object[] parms );

Assert.IsNotNull( object anObject );
Assert.IsNotNull( object anObject, string message, params object[] parms );

0.1.17.31 See also…

Assert.Null and Assert.IsNull test that the specified object is null. The two forms are provided for compatibility with past versions of NUnit and NUnitLite.

Assert.Null( object anObject );
Assert.Null( object anObject, string message, params object[] parms );

Assert.IsNull( object anObject );
Assert.IsNull( object anObject, string message, params object[] parms );

0.1.17.32 See also…

The Assert.Pass method allows you to immediately end the test, recording it as successful. Since it causes an exception to be thrown, it is more efficient to simply allow the test to return. However, Assert.Pass allows you to record a message in the test result and may also make the test easier to read in some situations. Additionally, like the other methods on this page, it can be invoked from a nested method call with the result of immediately terminating test execution.

Assert.Pass();
Assert.Pass( string message, params object[] parms );

0.1.17.33 See also…

  • [[Assert.Pass]]
  • [[Assert.Ignore]]
  • [[Assert.Inconclusive]]

The Assert.Throws method is pretty much in a class by itself. Rather than comparing values, it attempts to invoke a code snippet, represented as a delegate, in order to verify that it throws a particular exception.

It’s also in a class by itself in that it returns an Exception, rather than void, if the Assert is successful. See the example below for a few ways to use this.

Assert.Throws may be used with a constraint argument, which is applied to the actual exception thrown, or with the Type of exception expected. The Type format is available in both both a non-generic and generic form.

If the code under test is async, you must use [[Assert.ThrowsAsync]].

Exception Assert.Throws( Type expectedExceptionType, TestDelegate code );
Exception Assert.Throws( Type expectedExceptionType, TestDelegate code, 
                         string message, params object[] parms);

Exception Assert.Throws( IResolveConstraint constraint, TestDelegate code );
Exception Assert.Throws( IResolveConstraint constraint, TestDelegate code, 
                         string message, params object[] parms);

Assert.Throws<T>( TestDelegate code );
Assert.Throws<T>( TestDelegate code, 
                  string message, params object[] parms);

In the above code TestDelegate is a delegate of the form void TestDelegate(), which is used to execute the code in question. This may be an anonymous delegate or, when compiling under C# 3.0 or greater, a lambda expression.

The following example shows different ways of writing the same test.

[TestFixture]
public class AssertThrowsTests
{
  [Test]
  public void Tests()
  {  
    // Using a method as a delegate
    Assert.Throws<ArgumentException>( MethodThatThrows );

    // Using an anonymous delegate
    Assert.Throws<ArgumentException>(
      delegate { throw new ArgumentException(); } );

    // Using a Lambda expression
    Assert.Throws<ArgumentException>(
      () => { throw new ArgumentException(); } );
  }
  
  void MethodThatThrows()
  {
    throw new ArgumentException();
  }
}

This example shows use of the return value to perform additional verification of the exception.

[TestFixture]
public class UsingReturnValue
{
  [Test]
  public void TestException()
  {
    MyException ex = Assert.Throws<MyException>(
      delegate { throw new MyException( "message", 42 ); } );

    Assert.That( ex.Message, Is.EqualTo( "message" ) );
    Assert.That( ex.MyParam, Is.EqualTo( 42 ) ); 
  }
}

This example does the same thing using the overload that includes a constraint.

[TestFixture]
public class UsingConstraint
{
  [Test]
  public void TestException()
  {
    Assert.Throws( Is.TypeOf<MyException>()
                 .And.Message.EqualTo( "message" )
                 .And.Property( "MyParam" ).EqualTo( 42 ),
      delegate { throw new MyException( "message", 42 ); } );
  }
}

Use the form that matches your style of coding.

0.1.18 Exact Versus Derived Types

When used with a Type argument, Assert.Throws requires that exact type to be thrown. If you want to test for any derived Type, use one of the forms that allows specifying a constraint. Alternatively, you may use [[Assert.Catch]], which differs from Assert.Throws in allowing derived types. See the following code for examples:

// Require an ApplicationException - derived types fail!
Assert.Throws( typeof(ApplicationException), code );
Assert.Throws<ApplicationException>()( code );

// Allow both ApplicationException and any derived type
Assert.Throws( Is.InstanceOf( typeof(ApplicationException), code );
Assert.Throws( Is.InstanceOf<ApplicationException>;(), code );

// Allow both ApplicationException and any derived type
Assert.Catch<ApplicationException>( code );

// Allow any kind of exception
Assert.Catch( code );

0.1.18.1 See also…

  • [[Assert.Catch]]
  • [[Assert.CatchAsync]]
  • [[Assert.ThrowsAsync]]
  • [[ThrowsConstraint]]

The Assert.ThrowsAsync is the async equivalent to [[Assert.Throws]] for asynchronous code. See [[Assert.Throws]] for more information.

Exception Assert.ThrowsAsync( Type expectedExceptionType, AsyncTestDelegate code );
Exception Assert.ThrowsAsync( Type expectedExceptionType, AsyncTestDelegate code, 
                         string message, params object[] parms);

Exception Assert.ThrowsAsync( IResolveConstraint constraint, AsyncTestDelegate code );
Exception Assert.ThrowsAsync( IResolveConstraint constraint, AsyncTestDelegate code, 
                         string message, params object[] parms);

TActual Assert.ThrowsAsync<TActual>( AsyncTestDelegate code );
TActual Assert.ThrowsAsync<TActual>( AsyncTestDelegate code, 
                  string message, params object[] parms);

In the above code AsyncTestDelegate is a delegate of the form Task AsyncTestDelegate(), which is used to execute the code in question. This will likely be a lambda expression.

The following example shows the most common way of writing tests.

[TestFixture]
public class AssertThrowsTests
{
  [Test]
  public void Tests()
  {  
    // Using a method as a delegate
    Assert.ThrowsAsync<ArgumentException>(async () => await MethodThatThrows());
  }
  
  async Task MethodThatThrows()
  {
    await Task.Delay(100);
    throw new ArgumentException();
  }
}

This example shows use of the return value to perform additional verification of the exception. Note that you do not need to await the result.

[TestFixture]
public class UsingReturnValue
{
  [Test]
  public async Task TestException()
  {
    MyException ex = Assert.ThrowsAsync<MyException>(async () => await MethodThatThrows());

    Assert.That( ex.Message, Is.EqualTo( "message" ) );
    Assert.That( ex.MyParam, Is.EqualTo( 42 ) ); 
  }
}

0.1.18.2 See also…

  • [[Assert.Catch]]
  • [[Assert.CatchAsync]]
  • [[Assert.Throws]]
  • [[ThrowsConstraint]]

Assert.True and Assert.IsTrue test that the specified condition is true. The two forms are provided for compatibility with past versions of NUnit and NUnitLite.

Assert.True( bool condition );
Assert.True( bool condition, string message, params object[] parms );

Assert.IsTrue( bool condition );
Assert.IsTrue( bool condition, string message, params object[] parms );

You may also use Assert.That with a boolean argument to achieve the same result.

0.1.18.3 See also…

Some users prefer a shorter form of assertion than is given by Assert.That. If you derive your test fixture class from <b>AssertionHelper</b>, the Expect() method may be used instead…

Expect( bool condition );
Expect( bool condition, string message, params object[] parms );

Expect( ActualValueDelegate del, IResolveConstraint constraint )
Expect( ActualValueDelegate del, IResolveConstraint constraint,
             string message, params object[] parms )
             
Expect<TActual>( TActual actual, IResolveConstraint constraint )
Expect<TActual>( TActual actual, IResolveConstraint constraint,
             string message, params object[] parms )
             
Expect( TestDelegate del, IResolveConstraint constraint );

In addition, AssertionHelper allows the derived class to make direct use of many of the syntactic elements that would normally require you to specify the Is, Has or Does class in order to use them. For example, you can write…

Expect(actual, EqualTo("Hello"));

Use Intellisense to determine which syntactic elements are available in the current release.

Assertions are central to unit testing in any of the xUnit frameworks, and NUnit is no exception. NUnit provides a rich set of assertions as static methods of the Assert class.

If an assertion fails, the method call does not return and an error is reported. If a test contains multiple assertions, any that follow the one that failed will not be executed. For this reason, it’s usually best to try for one assertion per test.

Each method may be called without a message, with a simple text message or with a message and arguments. In the last case the message is formatted using the provided text and arguments.

0.1.19 Two Models

In NUnit 3.0, assertions are written primarily using the Assert.That method, which takes [[constraint objects|Constraints]] as an argument. We call this the [[Constraint Model]] of assertions.

In earlier versions of NUnit, a separate method of the Assert class was used for each different assertion. This [[Classic Model]] is still supported but since no new features have been added to it for some time, the constraint-based model must be used in order to have full access to NUnit’s capabilities.

For example, the following code must use the constraint model. There is no real classic equivalent.

int[] array = new int[] { 1, 2, 3 };
Assert.That( array, Has.Exactly(1).EqualTo(3) );
Assert.That( array, Has.Exactly(2).GreaterThan(1) );
Assert.That( array, Has.Exactly(3).LessThan(100) );

Where equivalent’s do exist, the two approaches will always give the same result, because the methods of the classic approach have all been implemented internally using constraints. For example…

Assert.AreEqual(4, 2+2);
Assert.That(2+2, Is.EqualTo(4));

AssignableFromConstraint tests that one type is assignable from another

<h4>Constructor</h4>

AssignableFromConstraint( Type )

<h4>Syntax</h4>

Is.AssignableFrom( Type )
Is.AssignableFrom<T>()

<h4>Examples of Use</h4>

Assert.That( "Hello", Is.AssignableFrom(typeof(string)));
Assert.That( 5, Is.Not.AssignableFrom(typeof(string)));

AssignableToConstraint tests that one type is assignable to another

<h4>Constructor</h4>

AssignableToConstraint( Type )

<h4>Syntax</h4>

Is.AssignableTo( Type )
Is.AssignableTo<T>()

<h4>Examples of Use</h4>

Assert.That( "Hello", Is.AssignableTo(typeof(object)));
Assert.That( 5, Is.Not.AssignableTo(typeof(string)));

Under Construction

0.1.20 DRAFT IN PROCESS

This technical note describes the inheritance hierarchy used for attributes in NUnit. It applies to the built-in attributes and may also be used as a guide for where to place custom attributes in the hierarchy. However, use of these attributes as base classes is entirely optional for custom attributes, as all functionality is accessed through the implemented interfaces.

Essentially, we use attribute inheritance to define the “what” of an attribute… that is, what it primarily is and what it primarily is used for. The actual functionality of an attribute is defined by the use of interfaces.

For information on interfaces used by NUnit attributes, see [[Attribute Interfaces]].

0.1.21 Abstract Attribute Classes

The base of the hierarchy is formed by a number of abstract classes. They generally incorporate no code.

0.1.21.1 NUnitAttribute

All NUnit attributes inherit directly or indirectly from NUnitAttribute. Its only purpose is to allow easy selection of all NUnit attributes on a member. If a custom attribute does not fit under any of the other attribute classes, you may derive it directly from NUnitAttribute.

0.1.21.2 TestFixtureBuilderAttribute

TestFixtureBuilderAttribute is the base class for any attribute that knows how to build a test fixture of some kind from a provided class. In this context, a test fixture means any test that is based on a user class.

NUnit includes the following attributes derived from TestFixtureBuilderAttribute: * TestFixtureAttribute * SetUpFixtureAttribute.

Derived classes that build fixtures should implement the IFixtureBuilder interface. TestFixtureBuilderAttribute does not implement this interface itself since future versions of NUnit may introduce additional interfaces that build fixtures. Therefore, the choice of interface is left to the derived class.

0.1.21.3 TestCaseBuilderAttribute

TestCaseBuilderAttribute is the base class for any attribute that knows how to build a test case from a given method. Test cases may be simple (without arguments) or parameterized (taking arguments) and are always based on a MethodInfo.

NUnit includes the following attributes derived from TestCaseBuilderAttribute:*CombiningStrategyAttribute*CombinatorialAttribute*PairwiseAttribute*SequentialAttribute*TestAttribute*TestCaseAttribute*TestCaseSourceAttribute*TheoryAttribute`

As with TestFixtureBuilderAttribute, the choice of a specific interface to implement is left to the derived class. Custom classes that build test cases should implement one of the interfaces designed for the purpose: ISimpleTestCaseBuilder or ITestCaseBuilder. Further interfaces may be defined in the future. In addition, any builder that is intended to cause a non-attributed class to be used as an NUnit TestFixture should implement IImplyFixture.

0.1.21.4 IncludeExcludeAttribute

IncludeExcludeAttribute is the base class for any attributes used to decide whether to include a test in the current run or exclude it based on the string properties Include, Exclude and Reason. The abstract base simply makes these properties available to the derived class, which is responsible for taking action on them.

NUnit currently defines two attributes that derive from IncludeExcludeAttribute: * CultureAttribute * PlatformAttribute

These two attributes implement IApplyToTest and set the Runstate of the test based on interpreting the arguments and the current environment. Custom classes derived from IncludeExcludeAttribute should do the same thing.

0.1.22 OUT OF DATE - UPDATE IN PROCESS

NUnit 3.0 implements a great deal of its functionality in its attributes. This functionality is accessed through a number of standard interfaces, which are implemented by the attributes. This technical note describes the interfaces used on NUnit attributes and recognized by NUnit when loading or running tests.

For ease of understanding, the interfaces are grouped according to the stage in the life-cycle of a test at which they are used. The two primary stages in the life of a test are Load-Time and Execution-Time.

0.1.23 Load-Time Interfaces

Loading tests means loading the assembly into memory and examining its content to discover the classes and fixtures that represent tests. The internal structures that represent tests are built at this time. If requested by the application, information about the tests may be returned for display, as is done in the NUnit GUI runner.

0.1.23.1 IFixtureBuilder

This interface is used by attributes that know how to build a fixture from a user class. IFixtureBuilder is defined as follows:

public interface IFixtureBuilder
{
    TestSuite BuildFrom(Type type);
}

Custom fixture builders should examine the provided Type and return an appropriate TestFixture based on it. If the fixture is intended to be an NUnit TestFixture, then the helper class NUnitTestFixtureBuilder may be used to create it.

The following NUnit attributes currently implement this interface: * TestFixtureAttribute * SetUpFixtureAttribute

It would make more sense for this interface method to return TestFixture rather than TestSuite. We use TestSuite because it is the common base for both TestFixture and SetupFixture. In a future version, we will try to adjust the hierarchy so that all suites based on a class are derived from TestFixture.

0.1.23.2 ITestBuilder

This interface is used by attributes that know how to build one or more parameterized TestMethod instances from a MethodInfo. ITestMethodBuilder is defined as follows:

public interface ITestBuilder
{
    IEnumerable<TestMethod> BuildFrom(MethodInfo method, Test suite);
}

Custom attributes implementing this interface should examine the MethodInfo and return as many TestMethod instances as it is able to construct, using the parameters available to it. Some attributes will only return a single test, just as TestCaseAttribute does. Others, working like TheoryAttribute may return multiple tests. If no data is available to create tests, an empty collection should be returned.

If the returned tests are standard NUnit TestMethods, the helper class NUnitTestCaseBuilder may be used to create them.

The following NUnit attributes currently implement ITestBuilder: * CombiningStrategyAttribute, with the following derived classes: * CombinatorialAttribute * PairwiseAttribute * SequentialAttribute * TestCaseAttribute * TestCaseSourceAttribute * TheoryAttribute

0.1.23.3 ISimpleTestBuilder

This interface is used by attributes that know how to build a single, non-parameterized test from a MethodInfo. ISimpleTestBuilder is defined as follows:

public interface ISimpleTestBuilder
{
    TestMethod BuildFrom(MethodInfo method, Test suite);
}

Custom attributes implementing ISimpleTestFixture should examine the MethodInfo provided and return a single TestMethod instance, as appropriate to that method. The BuildFrom method should never return null, even if the specified method is not valid for a test. In that case, it should return a TestMethod with a RunState of NonRunnable, in order to provide feedback to the user who placed the attribute on the method.

NUnit treats attributes implementing this interface specially. They are ignored if any other attributes are present that implement ITestBuilder. This allows, for example, use of [Test] on a method that also has [Combinatorial] specified, without any error arising. Such usage has existed in NUnit for some time and this special handling of the interface allows us to preserve it.

In the current build, only TestAttribute implements this interface.

0.1.23.4 IImplyFixture

The IImplyFixture interface is an empty interface, used solely as a marker:

public interface IImplyFixture
{
}

If a class contains any method with an attribute that implements this interface, that class is treated as an NUnit TestFixture without any TestFixture attribute being specified. The following NUnit attributes currently implement this interface: * TestAttribute * TestCaseAttribute * TestCaseSourceAttribute * TheoryAttribute

0.1.23.5 IApplyToTest

The IApplyToTest interface is used to make modifications to a test immediately after it is constructed. It is defined as follows:

public interface IApplyToTest
{
    void ApplyToTest(Test test);
}

The Test Type is quite general and the argument may represent a suite or an individual test case. If the distinction is important, then you must code the attribute to examine the argument and react accordingly.

The interface may appear on the same attribute that is used to construct the test or on a separate attribute. In either case, it will only be called after the test is built.

The order in which ApplyToTest is called on multiple attributes is indeterminate. If two attributes make completely independent changes to a test, then the order is not relevant. But if they both change the same property, or related properties, then it may necessary to make tests in the attribute code to ensure that the correct value ‘wins’.

The most common example of this is for attributes that change the RunState of a test. If one attribute is trying to set it to RunState.Ignore, while the other wants it to be RunState.NotRunnable, we would normally expect the ‘worst’ value to win and for the test to be non-runnable. We can achieve that by code like the following:

// In the attribute setting NotRunable
test.RunState = RunState.NotRunnable;
...

// In the attribute setting Ignore
if (test.RunState != RunState.NotRunnable)
    test.RunState = RunState.Ignore;

The following NUnit attributes implement IApplyToTest: * CategoryAttribute * CombiningStrategyAttribute * CultureAttribute * ExplicitAttribute * IgnoreAttribute * PlatformAttribute * PropertyAttribute (and, through it, a large number of derived attributes) * RequiresThreadAttribute * TestAttribute * TestFixtureAttribute

0.1.24 Execution-Time Interfaces

At execution-time, some or all of the tests that were previously loaded are actually run. Their results are returned and made available to the application.

0.1.24.1 IApplyToContext

NUnit tests run within a context, known as the TestExecutionContext. The context for a test case is nested within the context for its containing suite and so on, up to the assembly level. Attributes that implement IApplyToContext are called immediately after the context is created and before the test is run in order to make changes to the context. Once the test execution has completed, the context is discarded so that - effectively - any changes are reverted to their original values.

The IApplyToContext interface is defined as follows:

public interface IApplyToContext
{
    void ApplyToContext(TestExecutionContext context);
}

An example of the use of the context may be helpful. One item in the TestExecutionContext is the default timeout value for test cases. When any test is marked with [Timeout(nnn)] the context value is replaced by the supplied argument. The new timeout applies for any test case it appears on and any test case that is contained in a suite that it appears on. When the test or suite completes, the new value is discarded and the value contained in the original context is once against used.

Custom attributes that implement IApplyToContext should modify the TestExecutionContext in accordance with the arguments supplied to them. They are not called after the test is run and have no cleanup to perform.

The NUnit attributes that implement IApplyToContext are as follows: * ParallelizableAttribute * SetCultureAttribute * SetUICultureAttribute * TimeoutAttribute

0.1.24.2 ICommandDecoratorSource

In NUnit 2.x, tests were self-executing objects. In NUnit 3.0, execution is done using command objects, which are constructed for each test. Execution of a single test will generally require multiple nested commands. Some attributes are able to contribute to the chain of commands. For example, MaxTimeAttribute adds a command, which examines the elapsed time to complete a test and fails it if a specified maximum was exceeded.

Attributes add to the command chain by creating a command decorator and implementing the ICommandDecoratorSource interface. The interface is defined as follows:

public interface ICommandDecoratorSource
{
    IEnumerable<ICommandDecorator> GetDecorators(MethodInfo method);
}

Custom attributes that require participation in the command chain may create and return a decorator. Note that the decorators returned may vary according to the state of the attribute and that returning an empty collection is valid.

0.1.24.3 ICommandDecorator

This interface is implemented by command decorators, which are normally separate objects from the attributes. The ICommandDecorator interface is defined as follows:

public interface ICommandDecorator
{
    CommandStage Stage { get; }
    int Priority { get; }
    TestCommand Decorate(TestCommand command);
}

There are various stages of command execution. In the current version of NUnit, the following stages are used:

public enum CommandStage
{
    Default,
    BelowSetUpTearDown, // Command runs after setup and cleans up before teardown
    SetUpTearDown,
    AboveSetUpTearDown  // Command runs before setup and cleans up after teardown
}

This is a preliminary definition, subject to change.

The int value of priority is used to set the order of execution within a single stage. It usually defaults to zero.

The Decorate method should return an appropriate command in which the original command has been nested. For examples, see the implementation of MaxTimeAttribute and ExpectedExceptionAttribute.

AttributeConstraint tests for the existence of an attribute on a Type and then applies a constraint to that attribute.

0.1.24.4 Constructor

AttributeConstraint(Type type, IConstraint baseConstraint)

0.1.24.5 Syntax

Has.Attribute( typeof(TestFixtureAttribute) )...
Has.Attribute<TestFixtureAttribute>()...

0.1.24.6 Examples of Use

Assert.That( someObject, Has.Attribute( typeof(TestFixtureAttribute) )
    .Property("Description").EqualTo("My description") );
Assert.That( someObject, Has.Attribute<TestFixtureAttribute>()
    .Property("Description").EqualTo("My description") );

0.1.24.7 See also…

  • [[AttributeExistsConstraint]]

AttributeExistsConstraint tests for the existence of an attribute on a Type.

0.1.24.8 Constructor

AttributeExistsConstraint(Type type)

0.1.24.9 Syntax

Has.Attribute( typeof(TestFixtureAttribute) )
Has.Attribute<TestFixtureAttribute>()

0.1.24.10 Examples of Use

Assert.That( someObject, Has.Attribute( typeof(TestFixtureAttribute) ));
Assert.That( someObject, Has.Attribute<TestFixtureAttribute>() );

NUnit uses custom attributes to identify tests. All NUnit attributes are contained in the NUnit.Framework namespace. Each source file that contains tests must include a using statement for that namespace and the project must reference the framework assembly, nunit.framework.dll.

This table lists all the attributes supported by NUnit.

| Attribute | Usage | |———————————–|————-| | [[Apartment Attribute]] | Indicates that the test should run in a particular apartment. | | [[Author Attribute]] | Provides the name of the test author. | | [[Category Attribute]] | Specifies one or more categories for the test. | | [[Combinatorial Attribute]] | Generates test cases for all possible combinations of the values provided. | | [[Culture Attribute]] | Specifies cultures for which a test or fixture should be run. | | [[Datapoint Attribute]] | Provides data for [[Theories|Theory-Attribute]]. | | [[DatapointSource Attribute]] | Provides data for [[Theories|Theory-Attribute]]. | | [[Description Attribute]] | Applies descriptive text to a Test, TestFixture or Assembly. | | [[Explicit Attribute]] | Indicates that a test should be skipped unless explicitly run. | | [[Ignore Attribute]] | Indicates that a test shouldn’t be run for some reason. | | [[LevelOfParallelism Attribute]] | Specifies the level of parallelism at assembly level. | | [[Maxtime Attribute]] | Specifies the maximum time in milliseconds for a test case to succeed. | | [[OneTimeSetUp Attribute]] | Identifies methods to be called once prior to any child tests. | | [[OneTimeTearDown Attribute]] | Identifies methods to be called once after all child tests. | | [[Order Attribute]] | Specifies the order in which decorated test should be run (against others). | | [[Pairwise Attribute]] | Generate test cases for all possible pairs of the values provided. | | [[Parallelizable Attribute]] | Indicates whether test and/or its descendants can be run in parallel. | | [[Platform Attribute]] | Specifies platforms for which a test or fixture should be run. | | [[Property Attribute]] | Allows setting named properties on any test case or fixture. | | [[Random Attribute]] | Specifies generation of random values as arguments to a parameterized test. | | [[Range Attribute]] | Specifies a range of values as arguments to a parameterized test. | | [[Repeat Attribute]] | Specifies that the decorated method should be executed multiple times. | | [[RequiresThread Attribute]] | Indicates that a test method, class or assembly should be run on a separate thread. | | [[Retry Attribute]] | Causes a test to be rerun if it fails, up to a maximum number of times. | | [[Sequential Attribute]] | Generates test cases using values in the order provided, without additional combinations. | | [[SetCulture Attribute]] | Sets the current Culture for the duration of a test. | | [[SetUICulture Attribute]] | Sets the current UI Culture for the duration of a test. | | [[SetUp Attribute]] | Indicates a method of a TestFixture called just before each test method. | | [[SetUpFixture Attribute]] | Marks a class with one-time setup or teardown methods for all the test fixtures in a namespace. | | [[SingleThreaded Attribute]] | Marks a fixture that requires all it’s tests to run on the same thread. | | [[TearDown Attribute]] | Indicates a method of a TestFixture called just after each test method. | | [[Test Attribute]] | Marks a method of a TestFixture that represents a test. | | [[TestCase Attribute]] | Marks a method with parameters as a test and provides inline arguments. | | [[TestCaseSource Attribute]] | Marks a method with parameters as a test and provides a source of arguments. | | [[TestFixture Attribute]] | Marks a class as a test fixture and may provide inline constructor arguments. | | [[TestFixtureSetup Attribute]] | Deprecated synonym for [[OneTimeSetUp Attribute]]. | | [[TestFixtureSource Attribute]] | Marks a class as a test fixture and provides a source for constructor arguments. | | [[TestFixtureTeardown Attribute]] | Deprecated synonym for [[OneTimeTearDown Attribute]]. | | [[TestOf Attribute]] | Indicates the name or Type of the class being tested. | | [[Theory Attribute]] | Marks a test method as a Theory, a special kind of test in NUnit. | | [[Timeout Attribute]] | Provides a timeout value in milliseconds for test cases. | | [[Values Attribute]] | Provides a set of inline values for a parameter of a test method. | | [[ValueSource Attribute]] | Provides a source of values for a parameter of a test method. | <p>The Author Attribute adds information about the author of the tests. It can be applied to test fixtures and to tests.

<p>The constructor takes the name of the test author and optionally the author’s email address. Author can also be specified on a TestFixture or Test attribute.

[TestFixture]
[Author("Jane Doe", "jane.doe@example.com")]
public class MyTests
{
 [Test]
 public void Test1() { /* ... */ }

 [Test]
 [Author("Joe Developer")]
 public void Test2() { /* ... */ }
}

[TestFixture(Author = "Jane Doe")]
public class MyOtherTests
{
 [Test]
 public void Test1() { /* ... */ }

 [Test(Author = "Joe Developer")]
 public void Test2() { /* ... */ }
}

<p><strong>Note:</strong> you can currently only have one Author attribute per fixture or test. BinarySerializableConstraint tests whether an object is serializable in binary format.

<h4>Constructor</h4>

BinarySerializableConstraint()

<h4>Syntax</h4>

Is.BinarySerializable

<h4>Examples of Use</h4>

Assert.That( someObject, Is.BinarySerializable ));

0.1.24.11 See also…

  • [[XmlSerializableConstraint]]

0.2 NUnit 3.4

Breaking changes introduced in NUnit 3.4

  • The --teamcity comsole command-line option now requires the TeamCityEventListener extension to be intalled. This only affects users who install the extensions individually or copy them to another directory. If you install using the Windows installer or the NUnit.Console NuGet package the TeamCity extension is included.

  • String arguments in the names of test cases are no longer truncated to 40 characters.

  • The .NET 2.0 build of the nunit framework uses a private implementation of System.Linq. NUnit installs the NUnit.System.Linq assembly alongside the nunit.framework assembly. If you copy the framework to another location, you must ensure that both are copied. The extra assembly is not used in other builds because System.Linq is already supported in those environments.

0.3 NUnit 3.0

A relatively large number of features present in NUnit 2.6, were either removed in NUnit 3.0 or had their behavior modified in a way that will break existing code.

A key change is that the NUnit Test Engine will not recognize a test assembly that does not reference the NUnit framework directly. Normally, test assemblies use NUnit Asserts, attributes and other Types and methods. However, certain third-party frameworks are designed to completely isolate the user from the details of NUnit. They mediate between the test assembly and the NUnit framework in order to run tests. In such a case, NUnit will indicate that the assembly either contains no tests or the proper driver could not be found. To resolve this situation, simply add one NUnit attribute or other reference. An assembly-level ParallelizableAttribute is useful for this purpose.

Other breaking changes are grouped in the following tables.

0.3.0.0.0.1 Attributes
Name Notes
ExpectedExceptionAttribute No longer supported. Use Assert.Throws or Assert.That.
IgnoreAttribute The reason is now mandatory
RequiredAddinAttribute No longer supported.
RequiresMTAAttribute Deprecated. Use ApartmentAttribute
RequiresSTAAttribute Deprecated. Use ApartmentAttribute
SuiteAttribute No longer supported.
System.MTAThreadAttribute No longer treated as RequiresMTAAttribute
System.STAThreadAttribute No longer treated as RequiresSTAAttribute
TearDown and OneTimeTearDown There is a change to the logic by which teardown methods are called. See [[SetUp and TearDown Changes]] for details.
TestCaseAttribute Named parameter Result= is no longer supported. Use ExpectedResult=. Named parameter Ignore= now takes a string, giving the reason for ignoring the test.
TestCaseSourceAttribute The attribute forms using a string argument to refer to the data source must now use only static fields, properties or methods.
TestFixtureAttribute Named parameter Ignore= now takes a string, giving the reason for ignoring the test.
TestFixtureSetUpAttribute Deprecated. Use OneTimeSetUpAttribute.
TestFixtureTearDownAttribute Deprecated. Use OneTimeTearDownAttribute.
ValueSourceAttribute The source name of the data source must now use only static fields, properties or methods.
0.3.0.0.0.2 Assertions and Constraints
Feature Notes
Assert.IsNullOrEmpty No longer supported. Use Assert.That(..., Is.Null.Or.Empty)
Assert.IsNotNullOrEmpty No longer supported. Use Assert.That(..., Is.Not.Null.And.Not.Empty)
Is.InstanceOfType No longer supported. Use Is.InstanceOf
Is.StringStarting Deprecated. Use Does.StartWith
Is.StringContaining Deprecated. Use Does.Contain
Is.StringEnding Deprecated. Use Does.EndWith
Is.StringMatching Deprecated. Use Does.Match
NullOrEmptyStringConstraint No longer supported. See Assert.IsNullOrEmpty above
SubDirectoryContainsConstraint No longer supported. Various alternatives are available.
Text.All No longer supported. Use Does.All
Text.Contains No longer supported. Use Does.Contain or Contains.Substring
Text.DoesNotContain No longer supported. Use Does.Not.Contain
Text.StartsWith No longer supported. Use Does.StartWith
Text.DoesNotStartWith No longer supported. Use Does.Not.StartWith
Text.EndsWith No longer supported. Use Does.EndWith
Text.DoesNotEndWith No longer supported. Use Does.Not.EndWith
Text.Matches No longer supported. Use Does.Match
Text.DoesNotMatch No longer supported. Use Does.Not.Match
0.3.0.0.0.3 Other Framework Features
Feature Notes
Addins No longer supported. See [[Addin Replacement in the Framework]].
CurrentDirectory No longer set to the directory containing the test assembly. Use TestContext.CurrentContext.TestDirectory to locate that directory.
NUnitLite NUnitLite executable tests must now reference nunit.framework in addition to nunitlite.
SetUpFixture Now uses OneTimeSetUpAttribute and OneTimeTearDownAttribute to designate higher-level setup and teardown methods. SetUpAttribute and TearDownAttribute are no longer allowed.
TestCaseData The Throws Named Property is no longer available. Use Assert.Throws or Assert.That in your test case.
TestContext The fields available in the [[TestContext]] have changed, although the same information remains available as for NUnit V2.
0.3.0.0.0.4 Console Features

The console runner is now called nunit3-console. The following breaking changes apply to the options that the new runner supports.

Option Function Notes
–fixture=STR Test fixture or namespace to be loaded No longer supported. Use –test instead.
–run=STR List of tests to run No longer supported. Replaced by –test.
–runlist=PATH File containing list of tests to run No longer supported. Replaced by –testlist.
–include=LIST List of categories to include No longer supported. Use –where instead.
–exclude=LIST List of categories to exclude No longer supported. Use –where instead.
–process=PROCESS ProcessModel for test assemblies Default value is now Separate for a single assembly, Multiple for more than one. Multiple assemblies run in parallel by default.
–domain=DOMAIN DomainUsage for test assemblies Default value is now Separate for a single assembly, Multiple for more than one.
–apartment=APT Apartment in which to run tests No longer supported. Use ApartmentAttribute.
–xmlconsole Display XML to the console No longer supported.
–basepath Set ApplicationBase for execution No longer supported.
–privatebinpath Specify probing path for execution No longer supported.
–cleanup Remove left-over cache files No longer supported.
–noshadow Disable shadow copy No longer supported. The console runner now disables shadow copy by default. use –shadowcopy on the command-line to turn it on.
–nothread Disable use of a separate thread for tests No longer supported.
–nodots Do not display dots as a progress indicator No longer supported.

<p>The Category attribute provides an alternative to suites for dealing with groups of tests. Either individual test cases or fixtures may be identified as belonging to a particular category. Both the gui and console test runners allow specifying a list of categories to be included in or excluded from the run. When categories are used, only the tests in the selected categories will be run. Those tests in categories that are not selected are not reported at all.</p> <p>This feature is accessible by use of the /include and /exclude arguments to the console runner and through a separate “Categories” tab in the gui. The gui provides a visual indication of which categories are selected at any time.</p>

Warning: While the C# syntax allows you to place a Category attribute on a SetUpFixture class, the attribute is ignored by NUnit and has no effect in current releases.

<h4>Test Fixture Syntax</h4>

namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture]
  [Category("LongRunning")]
  public class LongRunningTests
  {
    // ...
  }
}

<h4>Test Syntax</h4>

namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture]
  public class SuccessTests
  {
    [Test]
    [Category("Long")]
    public void VeryLongTest()
    { /* ... */ }
}

<h3>Custom Category Attributes</h3>

Custom attributes that derive from <b>CategoryAttribute</b> will be recognized by NUnit. The default protected constructor of CategoryAttribute sets the category name to the name of your class.

Here’s an example that creates a category of Critical tests. It works just like any other category, but has a simpler syntax. A test reporting system might make use of the attribute to provide special reports.

[AttributeUsage(AttributeTargets.Method, AllowMultiple=false)]
public class CriticalAttribute : CategoryAttribute { }

...

[Test, Critical]
public void MyTest()
{ /*...*/ }

The classic Assert model uses a separate method to express each individual assertion of which it is capable.

Here’s a simple assert using the classic model:

      StringAssert.AreEqualIgnoringCase( "Hello", myString );

The Assert class provides the most common assertions in NUnit: * [[Assert.True]] * [[Assert.False]] * [[Assert.Null]] * [[Assert.NotNull]] * [[Assert.IsNaN]] * [[Assert.IsEmpty]] * [[Assert.IsNotEmpty]] * [[Assert.AreEqual]] * [[Assert.AreNotEqual]] * [[Assert.AreSame]] * [[Assert.AreNotSame]] * [[Assert.Contains]] * [[Assert.Greater]] * [[Assert.GreaterOrEqual]] * [[Assert.Less]] * [[Assert.LessOrEqual]] * [[Assert.IsInstanceOf]] * [[Assert.IsNotInstanceOf]] * [[Assert.IsAssignableFrom]] * [[Assert.IsNotAssignableFrom]] * [[Assert.Throws]] * [[Assert.ThrowsAsync]] * [[Assert.DoesNotThrow]] * [[Assert.DoesNotThrowAsync]] * [[Assert.Catch]] * [[Assert.CatchAsync]] * [[Assert.Pass]] * [[Assert.Fail]] * [[Assert.Ignore]] * [[Assert.Inconclusive]]

Additional assertions are provided by the following classes: * [[String Assert]] * [[Collection Assert]] * [[File Assert]] * [[Directory Assert]]

0.3.0.1 See also…

  • [[Constraint Model]]

The CollectionAssert class provides a number of methods that are useful when examining collections and their contents or for comparing two collections. These methods may be used with any object implementing IEnumerable.

The <b>AreEqual</b> overloads succeed if the corresponding elements of the two collections are equal. <b>AreEquivalent</b> tests whether the collection contents are equal, but without regard to order. In both cases, elements are compared using NUnit’s default equality comparison.

CollectionAssert.AllItemsAreInstancesOfType( IEnumerable collection, Type expectedType );
CollectionAssert.AllItemsAreInstancesOfType( 
    IEnumerable collection, Type expectedType, string message, params object[] args );

CollectionAssert.AllItemsAreNotNull( IEnumerable collection );
CollectionAssert.AllItemsAreNotNull( 
    IEnumerable collection, string message, params object[] args );

CollectionAssert.AllItemsAreUnique( IEnumerable collection ); 
CollectionAssert.AllItemsAreUnique( 
    IEnumerable collection, string message, params object[] args );

CollectionAssert.AreEqual( IEnumerable expected, IEnumerable actual );
CollectionAssert.AreEqual( 
    IEnumerable expected, IEnumerable actual, string message, params object[] args );

CollectionAssert.AreEquivalent( IEnumerable expected, IEnumerable actual);
CollectionAssert.AreEquivalent( 
    IEnumerable expected, IEnumerable actual, string message, params object[] args );

CollectionAssert.AreNotEqual( IEnumerable expected, IEnumerable actual );
CollectionAssert.AreNotEqual( 
    IEnumerable expected, IEnumerable actual, string message, params object[] args );

CollectionAssert.AreNotEquivalent( IEnumerable expected, IEnumerable actual );
CollectionAssert.AreNotEquivalent( 
    IEnumerable expected, IEnumerable actual, string message, params object[] args );

CollectionAssert.Contains( IEnumerable expected, object actual );
CollectionAssert.Contains( 
    IEnumerable expected, object actual, string message, params object[] args );

CollectionAssert.DoesNotContain( IEnumerable expected, object actual );
CollectionAssert.DoesNotContain( 
    IEnumerable expected, object actual, string message, params object[] args );

CollectionAssert.IsSubsetOf( IEnumerable subset, IEnumerable superset );
CollectionAssert.IsSubsetOf( 
    IEnumerable subset, IEnumerable superset, string message, params object[] args );

CollectionAssert.IsNotSubsetOf( IEnumerable subset, IEnumerable superset);
CollectionAssert.IsNotSubsetOf( 
    IEnumerable subset, IEnumerable superset, string message, params object[] args );

CollectionAssert.IsEmpty( IEnumerable collection );
CollectionAssert.IsEmpty( 
    IEnumerable collection, string message, params object[] args );

CollectionAssert.IsNotEmpty( IEnumerable collection );
CollectionAssert.IsNotEmpty( 
    IEnumerable collection, string message, params object[] args );

CollectionAssert.IsOrdered( IEnumerable collection );
CollectionAssert.IsOrdered( 
    IEnumerable collection, string message, params object[] args );
          
CollectionAssert.IsOrdered( IEnumerable collection, IComparer comparer );
CollectionAssert.IsOrdered( IEnumerable collection,
    IComparer comparer, string message, params object[] args );

0.3.0.2 See also…

  • Collection Constraints CollectionContainsConstraint tests that an IEnumerable contains an object. If the actual value passed does not implement IEnumerable, an exception is thrown.

<h4>Constructor</h4>

CollectionContainsConstraint( object )

<h4>Syntax</h4>

Has.Member( object )
Contains.Item( object )
Does.Contain( object )

<h4>Modifiers</h4>

...Using(IComparer comparer)
...Using<T>(IComparer<T> comparer)
...Using<T>(Comparison<T> comparer)

<h4>Examples of Use</h4>

int[] iarray = new int[] { 1, 2, 3 };
string[] sarray = new string[] { "a", "b", "c" };
Assert.That( iarray, Has.Member(3) );
Assert.That( sarray, Has.Member("b") );
Assert.That( sarray, Contains.Item("c") );
Assert.That( sarray, Has.No.Member("x") );    
Assert.That( iarray, Does.Contain(3) );

<h4>Note</h4>

Has.Member(), Contains.Item() and Does.Contain() work the same as Has.Some.EqualTo(). The last statement generates a [[SomeItemsConstraint]] based on an [[EqualConstraint]] and offers additional options such as ignoring case or specifying a tolerance. The syntax on this page may be viewed as a shortcut for specifying simpler cases.

CollectionEquivalentConstraint tests that two IEnumerables are equivalent - that they contain the same items, in any order. If the actual value passed does not implement IEnumerable an exception is thrown.

<h4>Constructor</h4>

CollectionEquivalentConstraint( IEnumerable other )

<h4>Syntax</h4>

Is.EquivalentTo( IEnumerable other )

<h4>Examples of Use</h4>

int[] iarray = new int[] { 1, 2, 3 };
string[] sarray = new string[] { "a", "b", "c" };
Assert.That( new string[] { "c", "a", "b" }, Is.EquivalentTo( sarray ) );
Assert.That( new int[] { 1, 2, 2 }, Is.Not.EquivalentTo( iarray ) );

<h4>Notes</h4>

  1. To compare items in order, use Is.EqualTo().

CollectionOrderedConstraint tests that an IEnumerable is ordered. If the actual value passed does not implement IEnumerable, an exception is thrown.

The constraint supports both simple and property-based ordering (Ordered.By).

0.3.1 Simple Ordering

Simple ordering is based on the values of the items themselves. It is implied when the By modifier is not used.

int[] iarray = new int[] { 1, 2, 3 };
Assert.That(iarray, Is.Ordered);

string[] sarray = new string[] { "c", "b", "a" };
Assert.That(sarray, Is.Ordered.Descending);

The following modifiers are supported:

...Ascending
...Descending
...Using(IComparer comparer)
...Using<T>(IComparer<T> comparer)
...Using<T>(Comparison<T> comparer)

By default, the order is expected to be ascending.

0.3.2 Property-Based Ordering

Property-based ordering uses one or more properties that are common to every item in the enumeration. It is used when one or more instances of the By modifier appears in the ordering expression.

string[] sarray = new string[] ( "a", "aa", "aaa" );
Assert.That(sarray, Is.Ordered.By("Length"));

string[] sarray2 = new string[] ( "aaa", "aa", "a" );
Assert.That(sarray2, Is.Ordered.Descending.By("Length"));

The following Modifiers are supported:

...Then
...Ascending
...Descending
...By(string propertyName)
...Using(IComparer comparer)
...Using<T>(IComparer<T> comparer)
...Using<T>(Comparison<T> comparer)

0.3.3 Ordering on Multiple Properties

An ordering expression may use multiple By modifiers, each referring to a different proproperty. The following examples assume a collection of items with proerties named A and B.

Assert.That(collection, Is.Ordered.By("A").Then.By("B"));
Assert.That(collection, Is.Ordered.By("A").Then.By("B").Descending);
Assert.That(collection, Is.Ordered.Ascending.By("A").Then.Descending.By("B"));
Assert.That(collection, Is.Ordered.Ascending.By("A").By("B").Descending);
Assert.That(collection, Is.Ordered.Ascending.By("A").Descending.By("B")); // Illegal!

0.3.3.1 Notes:

  1. The Then modifier divides the expression into ordering steps. Each step may optionally contain one Ascending or Descending modifier and one Using modifier.
  2. If Then is not used, each new By modifier marks the beginning of a step. The last example statement is illegal because the first group contains both Ascending and Descending. Use of Then is recommended for clarity.

CollectionSubsetConstraint tests that one IEnumerable is a subset of another. If the actual value passed does not implement IEnumerable, an exception is thrown.

<h4>Constructor</h4>

CollectionSubsetConstraint( IEnumerable )

<h4>Syntax</h4>

Is.SubsetOf( IEnumerable )

<h4>Example of Use</h4>

int[] iarray = new int[] { 1, 3 };
Assert.That( iarray, Is.SubsetOf( new int[] { 1, 2, 3 }) );

CollectionSupersetConstraint tests that one IEnumerable is a superset of another. If the actual value passed does not implement IEnumerable, an exception is thrown.

<h4>Constructor</h4>

CollectionSupersetConstraint( IEnumerable )

<h4>Syntax</h4>

Is.SupersetOf( IEnumerable )

<h4>Example of Use</h4>

int[] iarray = new int[] { 1, 2, 3 };
Assert.That( iarray, Is.SupersetOf( new int[] { 1, 3 }) );

<p>The <b>CombinatorialAttribute</b> is used on a test to specify that NUnit should generate test cases for all possible combinations of the individual data items provided for the parameters of a test. Since this is the default, use of this attribute is optional.

<h4>Example</h4>

<p>The following test will be executed six times:

[Test, Combinatorial]
public void MyTest(
    [Values(1,2,3)] int x,
    [Values("A","B")] string s)
{
    ...
}

MyTest is called six times, as follows:

    MyTest(1, "A")
    MyTest(1, "B")
    MyTest(2, "A")
    MyTest(2, "B")
    MyTest(3, "A")
    MyTest(3, "B")

<h4>Limitations</h4>

<p>When used on a generic method the programmer must ensure that all possible combinations of arguments are valid. When multiple parameters use the same generic type (e.g.: T) this may not be possible and the attribute may generate invalid test cases.

<h4>See also…</h4> * [[Sequential Attribute]] * [[Pairwise Attribute]]

<p>Normally, a configuration file used to provide settings or to control the environment in which tests are run, should be given the name as the assembly file with the suffix “.config” added. For example, the configuration file used to run nunit.tests.dll must be named nunit.tests.dll.config and located in the same directory as the dll.

<p>For multiple assemblies specified by a project, such as an NUnit or Visual Studio project, the same rule is used unless command-line options are used to load the project assemblies into a common AppDomain. In that case, the name of the project file is used, with “.config” added. For example, the command

    nunit-console MyProject.nunit /process:Separate /domain:Single

would make use of the config file MyProject.nunit.config.

<p>The console interface runner is invoked by a command in the form</p>

<pre> NUNIT3-CONSOLE [inputfiles] [options]</pre>

<p>where <b>inputfiles</b> is one or more assemblies or test projects of a type that NUnit can process and <b>options</b> is zero or more options. Input files and options may be mixed in any order.</p>

<h3>Input Files</h3>

<p>The console program must always have an assembly or project specified. Assemblies are specified by file name or path, which may be absolute or relative. Relative paths are interpreted based on the current directory.</p>

<p>In addition to assemblies, you may specify any project type that is understood by NUnit. Out of the box, this includes various Visual Studio project types as well as NUnit (.nunit) projects.</p>

<p>If the NUnit V2 framework driver is installed, test assemblies may be run based on any version of the NUnit framework beginning with 2.0. Without the V2 driver, only version 3.0 and higher tests may be run.

<h3>Options</h3>

<table class=“nunit” style=“margin-left:0; max-width:700px”>

<tr><th>–test=NAMES</th> <td>Comma-separated list of NAMES of tests to run or explore. This option may be repeated. Note that this option is retained for backward compatibility. The –where option can now be used instead.</td></tr>

<tr><th>–testlist=FILE</th> <td>The name (or path) of a FILE containing a list of tests to run or explore, one per line.</td></tr>

<tr><th>–where=EXPRESSION</th> <td>An expression indicating which tests to run. It may specify test names, classes, methods, catgories or properties comparing them to actual values with the operators ==, !=, =~ and !~. See [[Test Selection Language]] for a full description of the syntax.</td></tr>

<tr><th>–params|p=PARAMETER</th> <td>A test PARAMETER specified in the form NAME=VALUE. Multiple parameters may be specified, separated by semicolons or by repeating the –params option multiple times.</td></tr>

<tr><th>–config=NAME</th> <td>NAME of a project configuration to load (e.g.: Debug).</td></tr>

<tr><th>–process=PROCESS</th> <td>PROCESS isolation for test assemblies. Values: Single, Separate, Multiple. If not specified, defaults to Separate for a single assembly or Multiple for more than one. By default, processes are run in parallel.</td></tr>

<tr><th>–inprocess</th> <td>This option is a synonym for –process=Single</td></tr>

<tr><th>–agents=NUMBER</th> <td>NUMBER of agents that may be allowed to run simultaneously assuming you are not running inprocess. If not specified, all agent processes run tests at the same time, whatever the number of assemblies. This setting is used to control running your assemblies in parallel.</td></tr>

<tr><th>–domain=DOMAIN</th> <td>DOMAIN isolation for test assemblies. Values: None, Single, Multiple. If not specified, defaults to Separate for a single assembly or Multiple for more than one.</td></tr>

<tr><th>–framework=FRAMEWORK</th> <td>FRAMEWORK type/version to use for tests. Examples: mono, net-4.5, v4.0, 2.0, mono-4.0</td></tr>

<tr><th>–x86</th> <td>Run tests in a 32-bit process on 64-bit systems.</td></tr>

<tr><th>–dispose-runners</th> <td>Dispose each test runner after it has finished running its tests</td></tr>

<tr><th>–timeout=MILLISECONDS</th> <td>Set timeout for each test case in MILLISECONDS.</td></tr>

<tr><th>–seed=SEED</th> <td>Set the random SEED used to generate test cases.</td></tr>

<tr><th>–workers=NUMBER</th> <td>Specify the NUMBER of worker threads to be used in running tests. This setting is used to control running your tests in parallel and is used in conjunction with the [[Parallelizable Attribute]]. If not specified, workers defaults to the number of processors on the machine, or 2, whichever is greater.</td></tr>

<tr><th>–stoponerror</th> <td>Stop run immediately upon any test failure or error.</td></tr>

<tr><th>–debug</th> <td>Causes NUnit to break into the debugger immediately before it executes your tests. This is particularly useful when the tests are running in a separate process to which you would otherwise have to attach.</td></tr>

<tr><th>–debug-agent</th> <td>Available only in debug builds of NUnit, this option is for use by developers in debugging the nunit-agent itself. It breaks in the agent code immediately upon entry of the process.</td></tr>

<tr><th>–wait</th> <td>Wait for input before closing console window.</td></tr>

<tr><th>–work=PATH</th> <td>PATH of the directory to use for output files.</td></tr>

<tr><th>–output, –out=PATH</th> <td>File PATH to contain text output from the tests.</td></tr>

<tr><th>–err=PATH</th> <td>File PATH to contain error output from the tests.</td></tr>

<tr><th>–result=SPEC</th> <td>An output SPEC for saving the test results. This option may be repeated.</td></tr>

<tr><th>–explore[=SPEC]</th> <td>Display or save test info rather than running tests. Optionally provide an output SPEC for saving the test info. This option may be repeated.</td></tr>

<tr><th>–noresult</th> <td>Don’t save any test results.</td></tr>

<tr><th>–labels=VALUE</th> <td>Specify whether to write test case names to the output. Values: Off, On, All</td></tr>

<tr><th>–trace=LEVEL</th> <td>Set internal trace LEVEL. Values: Off, Error, Warning, Info, Verbose (Debug)</td></tr>

<tr><th>–shadowcopy</th> <td>Tells .NET to copy loaded assemblies to the shadowcopy directory.</td></tr>

<tr><th>–teamcity</th> <td>Turns on use of TeamCity service messages.</td></tr>

<tr><th>–loaduserprofile</th> <td>Causes the user profile to be loaded in any separate test processes.</td></tr>

<tr><th>–list-extensions</th> <td>Lists all extension points and the extensions installed on each of them.</td></tr>

<tr><th>–noheader, –noh</th> <td>Suppress display of program information at start of run.</td></tr>

<tr><th>–nocolor, –noc</th> <td>Displays console output without color.</td></tr>

<tr><th>–verbose, -v</th> <td>Display additional information as the test runs.</td></tr>

<tr><th>–help, -h</th> <td>Display this message and exit.</td></tr>

</table>

<h3>Description</h3>

<p>By default, this command runs the tests contained in the assemblies and projects specified. If the <b>–explore</b> option is used, no tests are executed but a description of the tests is saved in the specified or default format.</p>

<p>Several options that specify processing of XML output take an output specification as a value. A SPEC may take one of the following forms:</p>

<ul> <li>–OPTION:filename <li>–OPTION:filename;format=formatname <li>–OPTION:filename;transform=xsltfile </ul>

<p>The –result option may use any of the following formats: <ul> <li>nunit3 - the native XML format for NUnit 3.0 <li>nunit2 - legacy XML format used by earlier releases of NUnit</p> </ul>

<p>The –explore option may use any of the following formats: <ul> <li>nunit3 - the native XML format for NUnit 3.0 <li>cases - a text file listing the full names of all test cases.</p> </ul>

<p>If –explore is used without any specification following, a list of test cases is output to the console.</p>

<p>If neither –result nor –explore is used, NUnit saves the results to TestResult.xml in nunit3 format.</p>

<p>Any transforms provided must handle input in the native nunit3 format.</p>

The nunit3-console.exe program is a text-based runner for listing and running our tests from the command-line. It is able to run all NUnit 3.0 or higher tests atively and can run NUnit 2.x tests if the v2 driver is installed.

This runner is useful for automation of tests and integration into other systems. It automatically saves its results in XML format, allowing you to produce reports or otherwise process the results. The following is a screenshot of the console program output.

Screen shot of nunit-console
Screen shot of nunit-console

In this example, nunit3-console has just run selected tests in the mock-nunit-assembly.exe assembly that is part of the NUnit distribution. This assembly contains a number of tests, some of which are either ignored or marked explicit. The summary line shows the result of the test run.

The constraint-based Assert model uses a single method of the Assert class for all assertions. The logic necessary to carry out each assertion is embedded in the constraint object passed as the second parameter to that method.

Here’s a very simple assert using the constraint model:

      Assert.That( myString, Is.EqualTo("Hello") );

The second argument in this assertion uses one of NUnit’s <b>syntax helpers</b> to create an <b>EqualConstraint</b>. The same assertion could also be made in this form:

      Assert.That( myString, new EqualConstraint("Hello") );

Using this model, all assertions are made using one of the forms of the Assert.That() method, which has a number of overloads…

Assert.That( bool condition );
Assert.That( bool condition, string message, params object[] parms );

Assert.That( Func<bool> condition );
Assert.That( Func<bool> condition, string message, params object[] parms );              
             
Assert.That<TActual>( 
    ActualValueDelegate<TActual> del, IResolveConstraint constraint )
Assert.That<TActual>( 
    ActualValueDelegate<TActual> del, IResolveConstraint constraint,
    string message, object[] parms )
             
Assert.That<TActual>( TActual actual, IResolveConstraint constraint )
Assert.That<TActual>( TActual actual, IResolveConstraint constraint,
    string message, params object[] parms )
             
Assert.That( TestDelegate del, IResolveConstraint constraint );

The overloads that take a bool work exactly like Assert.IsTrue.

For overloads taking a constraint, the argument must be a object implementing the <b>IResolveConstraint</b> interface, which supports performing a test on an actual value and generating appropriate messages. This interface is described in more detail under [[Custom Constraints]].

NUnit provides a number of constraint classes similar to the <b>EqualConstraint</b> used in the example above. Generally, these classes may be used directly or through a syntax helper. The valid forms are described on the pages related to each constraint.

0.3.4 See also…

  • [[Classic Model]]

This page lists all the constraints available in NUnit. * Alphabetical List of Constraints * Constraints by Category

0.3.4.1 Alphabetical List of Constraints

Constraint Name | Purpose ————————————-|——– [[AllItemsConstraint]] | [[AndConstraint]] | [[AssignableFromConstraint]] | [[AssignableToConstraint]] | [[AttributeConstraint]] | [[AttributeExistsConstraint]] | [[BinarySerializableConstraint]] | [[CollectionContainsConstraint]] | [[CollectionEquivalentConstraint]] | [[CollectionOrderedConstraint]] | [[CollectionSubsetConstraint]] | [[CollectionSupersetConstraint]] | [[DelayedConstraint]] | [[EmptyCollectionConstraint]] | [[EmptyConstraint]] | [[EmptyDirectoryConstraint]] | [[EmptyStringConstraint]] | [[EndsWithConstraint]] | [[EqualConstraint]] | [[ExactCountConstraint]] | [[ExactTypeConstraint]] | [[FalseConstraint]] | [[FileOrDirectoryExistsConstraint]] | [[GreaterThanConstraint]] | [[GreaterThanOrEqualConstraint]] | [[InstanceOfTypeConstraint]] | [[LessThanConstraint]] | [[LessThanOrEqualConstraint]] | [[NaNConstraint]] | [[NoItemConstraint]] | [[NotConstraint]] | [[NullConstraint]] | [[OrConstraint]] | [[PropertyConstraint]] | [[PropertyExistsConstraint]] | [[RangeConstraint]] | [[RegexConstraint]] | [[ReusableConstraint]] | [[SameAsConstraint]] | [[SamePathConstraint]] | [[SamePathOrUnderConstraint]] | [[SomeItemsConstraint]] | [[StartsWithConstraint]] | [[SubPathConstraint]] | [[SubstringConstraint]] | [[ThrowsConstraint]] | [[ThrowsNothingConstraint]] | [[TrueConstraint]] | [[UniqueItemsConstraint]] | [[XmlSerializableConstraint]] |

0.3.4.2 Constraints by Category

0.3.4.2.1 Collection Constraints

Constraint Name | Purpose ————————————-|——– [[AllItemsConstraint]] | [[CollectionContainsConstraint]] | [[CollectionEquivalentConstraint]] | [[CollectionOrderedConstraint]] | [[CollectionSubsetConstraint]] | [[CollectionSupersetConstraint]] | [[EmptyCollectionConstraint]] | [[ExactCountConstraint]] | [[NoItemConstraint]] | [[SomeItemsConstraint]] | [[UniqueItemsConstraint]] |

0.3.4.2.2 Comparison Constraints

Constraint Name | Purpose ————————————-|——– [[GreaterThanConstraint]] | [[GreaterThanOrEqualConstraint]] | [[LessThanConstraint]] | [[LessThanOrEqualConstraint]] | [[RangeConstraint]] |

0.3.4.2.3 Compound Constraints

Constraint Name | Purpose ————————————-|——– [[AndConstraint]] | [[NotConstraint]] | [[OrConstraint]] |

0.3.4.2.4 Condition Constraints

Constraint Name | Purpose ————————————-|——– [[EmptyConstraint]] | [[FalseConstraint]] | [[NaNConstraint]] | [[NullConstraint]] | [[TrueConstraint]] |

0.3.4.2.5 File and Directory Constraints

Constraint Name | Purpose ————————————-|——– [[EmptyDirectoryConstraint]] | [[FileOrDirectoryExistsConstraint]] | [[SamePathConstraint]] | [[SamePathOrUnderConstraint]] | [[SubPathConstraint]] |

0.3.4.2.6 String Constraints

Constraint Name | Purpose ————————————-|——– [[EmptyStringConstraint]] | [[EndsWithConstraint]] | [[RegexConstraint]] | [[StartsWithConstraint]] | [[SubstringConstraint]] |

0.3.4.2.7 Type Constraints

Constraint Name | Purpose ————————————-|——– [[AssignableFromConstraint]] | [[AssignableToConstraint]] | [[ExactTypeConstraint]] | [[InstanceOfTypeConstraint]] |

See also: * [[Constraint Model]]

Counting tests sounds simple but there are a few issues. This page documents how the framework counts tests in various categories.

  1. In general, when counting tests, we are talking about test cases. In the case of non-parameterized test methods, that’s the same as the number of methods. Where there are parameters, we count the number of individual cases.

  2. When all the tests in an assembly are run, the total number of tests is the number of tests present in the assembly. When selection is made - either via the command line or through a Gui - we count only tests that are actually selected. Non-selected tests don’t appear at all in the XML result file and are not taken into account by the counts.

  3. Tests are categorized in one of four statuses: Passed, Failed, Inconclusive and Skipped.

3.1. Passed tests currently only have one ResultState, Success.

3.2. Failed tests are caused by any of the following: * Failure of an assertion (ResultState.Failure) * An unexpected exception (ResultState.Error) * An invalid test (ResultState.NotRunnable) * User cancellation (ResultState.Cancelled)

3.3. Inconclusive tests currently only have one ResultState, Inconclusive

3.4. Skipped tests are caused by * Ignoring the test (ResultState.Ignored) * The test was marked explicit (ResultState.Skipped) * Note: this may change to ResultState.Explicit in the future * A Platform, Culture or UICulture attribute test failed (ResultState.Skipped)

  1. Users may define new ResultStates in any of the four statuses.

  2. Failures, errors, skipping or ignoring in the SetUp for a test counts the same as if the action happened in the test method itself.

  3. When fixtures are ignored or otherwise skipped using an attribute on the class, all tests within the fixture are given the corresponding result.

  4. Failures, errors, skipping or ignoring in the OneTimeSetUp for a fixture causes all the test cases in the fixture to get the corresponding result.

  5. An invalid fixture causes all of its test cases to count as invalid.

The Culture attribute is used to specify cultures for which a test or fixture should be run. It does not affect the culture setting, but merely uses it to determine whether to run the test. If you wish to change the culture when running a test, use the SetCulture attribute instead.

If the specified culture requirements for a test are not met it is skipped. In the gui, the tree node for the test remains gray and the status bar color is not affected.

One use of the Culture attribute is to provide alternative tests under different cultures. You may specify either specific cultures, like “en-GB” or neutral cultures like “de”.

0.3.4.3 Test Fixture Syntax

namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture]
  [Culture("fr-FR")]
  public class FrenchCultureTests
  {
    // ...
  }
}

0.3.4.4 Test Syntax

namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture]
  public class SuccessTests
  {
    [Test]
    [Culture(Exclude="en,de")]
    public void SomeTest()
    { /* ... */ }
}

0.3.4.5 See also…

  • [[SetCulture Attribute]]

0.4 DRAFT - Out of Date

Experienced developers using NUnit generally end up with a library of custom setup, teardown, test and support methods. Custom Asserts provide one way to re-package such methods, allowing the normal NUnit syntax to be used to invoke them and providing error messages that match the style and layout of standard NUnit messages.

The standard NUnit Asserts create an object known as an <b>asserter</b> and pass it to the <b>DoAssert</b> method, which includes code similar to this…

<pre class=“prettyprint”> if ( !asserter.Test() ) throw new AssertionException( asserter.Message ); </pre></p>

<p><b>Asserters</b> encapsulate the comparison to be performed as well as the objects being compared. They implement the <b>IAsserter</b> interface, defined as follows:

<pre class=“prettyprint”> public interface IAsserter { // Test the condition for the assertion. bool Test();

    // Return the message giving the failure reason.
    string Message { get; }
}

</pre></p>

<p>When implementing an <b>asserter</b>, you will have to decide on an approach for creating the message. For complex tests, it may be necessary to create and cache the message - or info used to create it - while the test is being performed. For example, when NUnit compares arrays, it notes the point of failure for use in the message. Otherwise, it would have to pass the entire array a second time.</p>

<p>Generally, the constructor for the <b>asserter</b> will include any required parameters, the actual value and an optional user message. You may invoke the asserter directly, using <b>Assert.DoAssert</b>, but it is generally more convenient and readable to create an class similar to NUnit’s <b>Assert</b> class, which contains static methods that wrap the object creation. For an example of how to do this, see the <b>StringAssert</b> class in the NUnit source.</p>

<h2>Page Under Development</h2>

NUnit 2.4 introduced addins, which allowed a knowledgeable user to extend the framework by adding new test types, handling events, etc. This approach to extending the framework continued in use through NUnit 2.6.4, the final release of NUnit V2.

In NUnit 3.0, the functions provided by these addins are being taken over by the use of [[Custom Attributes]]. This approach is much simpler than the original approach and we think it will promote a great deal of user innovation in extending NUnit. NUnit 3.0 attributes are generally active. That is, they contain code to perform the function they call for rather than simply serving as markers to be interpreted by the runner.

0.5 Page Under Development

You can implement your own custom constraints by creating a class that inherits from the Constraint abstract class, which supports performing a test on an actual value and generating appropriate messages.

0.5.1 Constraint Abstract Class

Implementations must override the one abstract method ApplyTo<TActual> which evaluates the previously stored expected value (if any) against the method’s parameter, the actual value. There are also several virtual methods that may be overridden to change some default behaviors.

The relevant portions of the Constraint class are represented below.

namespace NUnit.Framework.Constraints
{
    public abstract class Constraint
    {
        protected Constraint(params object[] args) {}
        public abstract ConstraintResult ApplyTo<TActual>(TActual actual);
        ...
        public virtual ConstraintResult ApplyTo<TActual>(ActualValueDelegate<TActual> del) {}
        public virtual ConstraintResult ApplyTo<TActual>(ref TActual actual) {}
        protected virtual object GetTestObject<TActual>(ActualValueDelegate<TActual> del) {}
        public virtual string Description { get; protected set; }
        protected virtual string GetStringRepresentation() {}
    }
}

0.5.1.1 Constraint Constructor

The Constraint constructor accepts zero or more arguments and saves them to be used in the printed description later. Constraints like NullConstraint or UniqueItemsConstraint take no arguments and simply state some condition about the actual value supplied. Constraints with a single argument usually treat it as the expected value resulting from some operation. Multiple arguments can be provided where the semantics of the constraint call for it.

0.5.1.2 ApplyTo Implementation

The ApplyTo<TActual>(TActual actual) method must be overridden and provides for the core implementation of the custom constraint. Whatever logic defines pass or fail of the constraint and actual/expected values goes into the ApplyTo<TActual>(TActual actual) method.

For example, a very naive implementation of a reference equality constraint might look like this:

        public override ConstraintResult ApplyTo<TActual>(TActual actual)
        {
            return new ConstraintResult(this, actual, ReferenceEquals(actual, Arguments[0]));
        }

The key here is there needs to be some evaluation of the constraint logic, and the return value must be a ConstraintResult or subclass thereof. Custom subclasses of ConstraintResult may be used to further customize the message provided upon failure, as described below.

0.5.1.3 ApplyTo Overloads

Constraints may be called with a delegate to return the actual value instead of the actual value itself. This serves to delay evaluation of the value. The default implementation of ApplyTo<TActual>(ActualValueDelegate<TActual> del) waits for the delegate to complete if it’s an async operation, other immediately calls the delegate if synchronous, and then calls the abstract ApplyTo<TActual>(TActual actual) method with the value.

Another overload also exists, ApplyTo<TActual>(ref TActual actual). The default implementation dereferences the value and then calls the abstract ApplyTo<TActual>(TActual actual) method with the value. This public virtual method is available by use from calling code but currently is not used from any framework calls within NUnit itself.

0.5.1.4 GetTestObject Optional Override

The default implementation of ApplyTo<TActual>(ActualValueDelegate<TActual> del) does not simply execute the delegate but actually calls out to another virtual method, GetTestObject<TActual>(ActualValueDelegate<TActual> del). This method can be overridden to keep the default behavior of ApplyTo<TActual>(ActualValueDelegate<TActual> del) while still customizing how the actual value delegate is invoked.

0.5.2 Description Property

This virtual property is used to provide a description of the constraint for messages. Simple constant values can be set in the custom constraint’s constructor. If more complex logic is needed, override the property and provide a custom implementation of get.

Here are a few simple examples from built-in constraints.

public class FalseConstraint : Constraint
{
    public FalseConstraint()
    {
        this.Description = "False";
    }
}

public class NullConstraint : Constraint
{
    public NullConstraint()
    {
        this.Description = "null";
    }
}

Here are a few complex examples from built-in constraints.

public class AndConstraint : BinaryConstraint
{
    public override string Description
    {
        get { return Left.Description + " and " + Right.Description; }
    }
}

public abstract class PrefixConstraint : Constraint
{
    public override string Description
    {
        get
        {
            return string.Format(
                baseConstraint is EqualConstraint ? "{0} equal to {1}" : "{0} {1}", 
                descriptionPrefix, 
                baseConstraint.Description);
        }
    }
}

0.5.2.1 GetStringRepresentation Method

NUnit calls the GetStringRepresentation method to return a string representation of the constraint, including the expected value(s). The default implementation returns the lowercase display name of the constraint followed by all expected values, separated by a space.

For example, a custom constraint ReferenceEqualsConstraint with an instance of a custom MyObject class as expected value would result in a default string representation of <referenceequals MyObject>.

You can override the initial display name only by setting DisplayName in your constructor. This public property cannot be overridden, but the Constraint base class sets it in the base constructor to be the name of the class, minus the “Constraint” suffix and minus any generic suffixes.

0.5.3 Custom Constraint Usage Syntax

Having written a custom constraint class, you can use it directly through its constructor:

Assert.That(myObject, new CustomConstraint());

You may also use it in expressions through NUnit’s Matches syntax element:

Assert.That(myObject, Is.Not.Null.And.Matches(new CustomConstraint());

The direct construction approach is not very convenient or easy to read. For its built-in constraints, NUnit includes classes that implement a special constraint syntax, allowing you to write things like…

Assert.That(actual, Is.All.InRange(1,100) );

Custom constraints can support this syntax by providing a static helper class and extension method on ConstraintExpression, such as this.

public static class CustomConstraintExtensions
{
    public static ContentsEqualConstraint ContentsEqual(this ConstraintExpression expression, object expected)
    {
        var constraint = new ContentsEqualConstraint(expected);
        expression.Append(constraint);
        return constraint;
    }
}

To fully utilize your custom constraint the same way built-in constraints are used, you’ll need to implement three additional classes (which can cover all your constraints, not for each custom constraint).

  1. Provide a static class patterned after NUnit’s Is class, with properties or methods that construct your custom constructor. If you like, you can even call it Is and extend NUnit’s Is, provided you place it in your own namespace and avoid any conflicts. This allows you to write things like:

C# Assert.That(actual, Is.Custom(x,y) );

with this sample implementation:

C# public class Is : NUnit.Framework.Is { public static CustomConstraint Custom(object expected) { return new CustomConstraint(expected); } }

  1. Provide an extension method for NUnit’s ConstraintExpression, allowing you to write things like:

C# Assert.That(actual, Is.Not.Custom(x,y) );

with this sample implementation:

```C#
public static class CustomConstraintExtensions
{
    public static CustomConstraint Custom(this ConstraintExpression expression, object expected)
    {
        var constraint = new CustomConstraint(expected);
        expression.Append(constraint);
        return constraint;
    }
}    
```
  1. Optionally, provide your own base class that inherits from AssertionHelper* that additionally provides helpers for your custom constraints.

C# Expect(actual, Custom(expected));

```C#
public class CustomHelper : AssertionHelper
{
    public CustomConstraint Custom(object expected)
    {
        return new CustomConstraint(expected);
    }
}
```

* Note that AssertionHelper is currently being evaluated for possible removal from the NUnit framework.

The <b>Datapoint</b> attribute is used to provide data for <b>Theories</b> and is ignored for ordinary tests - including tests with parameters.

When a Theory is loaded, NUnit creates arguments for each of its parameters by using any fields of the same type as the parameter annotated with the <b>DatapointAttribute</b>. Fields must be members of the class containing the Theory and their Type must exactly match the argument for which data is being supplied.

0.5.3.1 Automatically Supplied Datapoints

It is normally not necessary to specify datapoints for <b>boolean</b> or <b>enum</b> arguments. NUnit automatically supplies values of <b>true</b> and <b>false</b> for <b>boolean</b> arguments and will supply all defined values of any enumeration.

If for some reason you don’t wish to use all possible values, you can override this behavior by supplying your own datapoints. If you supply any datapoints for an argument, automatic datapoint generation is suppressed.

<h4>Example</h4>

For an example of use, see [[Theory Attribute]]

<h4>See also…</h4>

  • [[Theory Attribute]]
  • [[Parameterized Tests]]

The <b>DatapointSource</b> attribute is used to provide data for <b>Theories</b> and is ignored for ordinary tests - including tests with parameters.

Collections of datapoints may be provided by use of the <b>DatapointSourceAttribute</b>. This attribute may be placed on methods or properties in addition to fields. The returned value must be either an array of the required type or an <b>IEnumerable<T></b> returning an enumeration of the required type. The data Type must exactly match the argument for which data is being supplied.

In earlier versions of NUnit, the obsolete <b>DatapointsAttribute</b> was used in place of <b>DatapointSourceAttribute</b>.

0.5.3.2 Automatically Supplied Datapoints

It is normally not necessary to specify datapoints for <b>boolean</b> or <b>enum</b> arguments. NUnit automatically supplies values of <b>true</b> and <b>false</b> for <b>boolean</b> arguments and will supply all defined values of any enumeration.

If for some reason you don’t wish to use all possible values, you can override this behavior by supplying your own datapoints. If you supply any datapoints for an argument, automatic datapoint generation is suppressed.

0.5.3.3 Example

For an example of use, see [[Theory Attribute]]

<h4>See also…</h4>

  • [[Theory Attribute]]
  • [[Parameterized Tests]]

<p><b>DelayedConstraint</b> delays the application of another constraint until a certain amount of time has passed. In it’s simplest form, it replaces use of a Sleep in the code but it also supports polling, which may allow use of a longer maximum time while still keeping the tests as fast as possible.

<p>The <b>After</b> modifier is permitted on any constraint, and the delay applies to the entire expression up to the point where <b>After</b> appears.

<p>Use of a <b>DelayedConstraint</b> with a value argument makes no sense, since the value will be extracted at the point of call. It’s intended use is with delegates and references. If a delegate is used with polling, it may be called multiple times so only methods without side effects should be used in this way.

<table class=“constraints”> <tr><th>Syntax Helper</th><th>Constructor</th><th>Operation</th></tr> <tr><td>After(int)</td><td>DelayedConstraint(Constraint, int)</td></td><td>tests that a constraint is satisfied after a delay.</tr> <tr><td>After(int, int)</td><td>DelayedConstraint(Constraint, int, int)</td></td><td>tests that a constraint is satisfied after a delay using polling.</tr> </table>

<p>The Description attribute is used to apply descriptive text to a Test, TestFixture or Assembly. The text appears in the XML output file and is shown in the Test Properties dialog.</p>

<h4>Example:</h4>

[assembly: Description("Assembly description here")]

namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture, Description("Fixture description here")]
  public class SomeTests
  {
    [Test, Description("Test description here")] 
    public void OneTest()
    { /* ... */ }
  }
}

<p><b>Note:</b> The Test and TestFixture attributes continue to support an optional Description property. The Description attribute should be used for new applications. If both are used, the Description attribute takes precedence.</p>

The DirectoryAssert class provides methods for comparing two directories or verifying the existence of a directory. Directories may be provided as DirectoryInfos or as strings giving the path to each directory.

DirectoryAssert.AreEqual(DirectoryInfo expected, DirectoryInfo actual);
DirectoryAssert.AreEqual(DirectoryInfo expected, DirectoryInfo actual, 
    string message, params object[] args);

DirectoryAssert.AreNotEqual(DirectoryInfo expected, DirectoryInfo actual);
DirectoryAssert.AreNotEqual(DirectoryInfo expected, DirectoryInfo actual, 
    string message, params object[] args);

DirectoryAssert.Exists(DirectoryInfo actual);
DirectoryAssert.Exists(DirectoryInfo actual, 
    string message, params object[] args);

DirectoryAssert.Exists(string actual);
DirectoryAssert.Exists(string actual, 
    string message, params object[] args);

DirectoryAssert.DoesNotExist(DirectoryInfo actual);
DirectoryAssert.DoesNotExist(DirectoryInfo actual, 
    string message, params object[] args);

DirectoryAssert.DoesNotExist(string actual);
DirectoryAssert.DoesNotExist(string actual, 
    string message, params object[] args);

0.5.3.4 See also…

The EmptyCollectionConstraint tests if a Collection or other IEnumerable is empty. An ArgumentException is thrown if the actual value is not an IEnumerable or is null.

<h4>Constructor</h4>

EmptyCollectionConstraint()

<h4>Syntax</h4>

Is.Empty

<h4>Examples of Use</h4>

Assert.That(new int[] { }, Is.Empty);
Assert.That(new int[] { 1, 2, 3 }, Is.Not.Empty);

Note: Is.Empty actually creates an EmptyConstraint. Subsequently applying it to an IEnumerable or ICollection causes an EmptyCollectionConstraint to be created. EmptyConstraint tests that an object is an empty string, directory or collection.

<h4>Constructor</h4>

EmptyConstraint()

<h4>Syntax</h4>

Is.Empty

<h4>Examples of Use</h4>

Assert.That( aString, Is.Empty );
Assert.That( dirInfo, Is.Empty );
Assert.That( collection, Is.Empty );

0.5.3.5 Notes:

  • EmptyConstraint creates and uses either an [[EmptyStringConstraint]], [[EmptyDirectoryConstraint]] or [[EmptyCollectionConstraint]] depending on the argument tested.
  • A DirectoryInfo argument is required in order to test for an empty directory. To test whether a string represents a directory path, you must first construct a DirectoryInfo.

The EmptyDirectoryConstraint tests if a Directory is empty.

<h4>Constructor</h4>

EmptyDirectoryConstraint()

<h4>Syntax</h4>

Is.Empty

<h4>Examples of Use</h4>

Assert.That(new DirectoryInfo(actual), Is.Empty);
Assert.That(new DirectoryInfo(actual), Is.Not.Empty);

Note: Is.Empty actually creates an EmptyConstraint. Subsequently applying it to a DirectoryInfo causes an EmptyDirectoryConstraint to be created. The EmptyStringConstraint tests if a string is empty.

<h4>Constructor</h4>

EmptyStringConstraint()

<h4>Syntax</h4>

Is.Empty

<h4>Examples of Use</h4>

Assert.That(string.Empty, Is.Empty);
Assert.That("A String", Is.Not.Empty);

Note: Is.Empty actually creates an EmptyConstraint. Subsequently applying it to a string causes an EmptyStringConstraint to be created. EndsWithConstraint tests for an ending string.

<h4>Constructor</h4>

EndsWithConstraint(string expected)

<h4>Syntax</h4>

Does.EndWith(string expected)
EndsWith(string expected)

<h4>Modifiers</h4>

...IgnoreCase

<h4>Examples of Use</h4>

string phrase = "Make your tests fail before passing!"

Assert.That( phrase, Does.EndWith( "!" ) );
Assert.That( phrase, Does.EndWith( "PASSING!" ).IgnoreCase );
Expect( phrase, EndsWith( "!" ) );

<h4>Notes</h4> 1. <b>EndsWith</b> may appear only in the body of a constraint expression or when the inherited syntax is used.

The NUnit Test Engine uses a plugin architecture to allow new functionality to be added by third parties. We originally planned to use Mono.Addins for this purpose and did so in the first betas. Because Mono.Addins no longer supports .NET 2.0, we were using a modified version that we created ourselves and which we would have to maintain in the future. Since Mono.Addins has many more features than we expect to use we decided to return to a custom plugin architecture.

The NUnit 3.0 Engine Extensibility model is essentially based on the NUnit V2 addin design with a number of improvements, primarily inspired by Mono.Addins. On this page, we describe that model as a guide for folks working on NUnit or otherwise needing to understand it. See [[Writing Engine Extensions]] for user-focused informaton about how to create an extension.

0.6 Extension Points

NUnit 3.0 Extensibility centers around ExtensionPoints. An ExtensionPoint is a place in the application where add-ins can register themselves in order to provide added functionality. Each extension point is identified by a string, called the Path, is associated with a particular Type to be used by extensions and may also have an optional Description.

How an extension is used depends on the particular extension point. For example, some extension points may only allow one extension to be registered while others may support multiple extensions. Where multiple extensions are supported, they may be used as alternatives or they may all operate, depending on the precise purpose of the extension point. In other words, read the docs and understand the exact semantics of an extension point before trying to create an extension for it.

In our initial implementation, all extension points are contained in the engine. In a future release, we will allow addins to define extension points and host extensions as well.

ExtensionPoints are identified by use of either the TypeExtensionPointAttribute or the ExtensionPointAttribute.

0.6.0.1 TypeExtensionPointAttribute

This is the most common way we identify ExtensionPoints in NUnit. The TypeExtensionPointAttribute is applied to an interface or class, exposed to the user in the nunit.engine.api assembly. This indicates that extensions must implement the specified interface or derive from the class.

For example, here is the code used to define the extension point for driver factories - classes that know how to create an appropriate driver for a test assembly.

    /// <summary>
    /// Interface implemented by a Type that knows how to create a driver for a test assembly.
    /// </summary>
    [TypeExtensionPoint(
        Description = "Supplies a driver to run tests that use a specific test framework.")]
    public interface IDriverFactory
    {
        /// <summary>
        /// Gets a flag indicating whether a given assembly name and version
        /// represent a test framework supported by this factory.
        /// </summary>
        bool IsSupportedTestFramework(string assemblyName, Version version);

        /// <summary>
        /// Gets a driver for a given test assembly and a framework
        /// which the assembly is already known to reference.
        /// </summary>
        /// <param name="domain">The domain in which the assembly will be loaded</param>
        /// <param name="assemblyName">The Name of the test framework reference</param>
        /// <param name="version">The version of the test framework reference</param>
        /// <returns></returns>
        IFrameworkDriver GetDriver(AppDomain domain, string assemblyName, Version version);
    }

In this case, we used the default constructor. An alternate constructor allows specifying the Path but is usually not needed. In this case, NUnit will assign a default Path of “/NUnit/Engine/TypeExtensions/IDriverFactory”. Extensions that implement IDriverFactory will be automatically associated with this extension point.

Description is the only named property for this attribute.

0.6.0.2 ExtensionPointAttribute

Extensions may also defined by use of the ExtensionPointAttribute at the assembly level. The Path and the Type must be specified in the attribute constructor. Each attribute identifies one extension point supported by that assembly, specifying an identifying string (the Path) and the required Type of any extension objects to be registered with it.

The following example shows an alternative way we might have identified the same driver factory extension point shown above. This is not actual NUnit code, but only a hypothetical example.

[assembly: ExtensionPoint(
               "/NUnit/Engine/TypeExtensions/IDriverFactory",
               typeof(IDriverFactory),
               Description="Supplies a driver to run tests that use a specific test framework.")]

This example defines exactly the same extension point as in the TypeExtensionPointAttribute example, albeit in a more roundabout way.

Again, Description is the only named property for this attribute.

0.6.0.3 Supported Extension Points

The following extension types are supported by the engine:

  • [[Project Loaders]]
  • [[Result Writers]]
  • [[Framework Drivers]]
  • [[Event Listeners]] (NUnit 3.4 and later only)

0.7 Extensions

An Extension is a single object of the required Type, which is registered with an ExtensionPoint. Extensions are identified by the ExtensionAttribute and additional information may be provided by use of the ExtensionPropertyAttribute, both of which are applied to the class that implements the extension.

All Extensions must have a default constructor, which is used by NUnit to create the object when it is needed.

0.7.0.1 ExtensionAttribute

The ExtensionAttribute has only a default constructor, as well as two named properties, Path and Description. If the path is not provided, NUnit will try to find the appropriate extension point based on what Types are inherited or implemented by the class on which the attribute is placed.

Assuming the extension point definition used above, any of the following would identify the classes as driver factories.

    [Extension(Path = "/NUnit/Engine/TypeExtensions/IDriverFactory")]
    public class DriverFactory1 : IDriverFactory
    {
        ...
    }

    [Extension]
    public class DriverFactory2 : IDriverFactory
    {
        ...
    }

Generally, the Path will be omitted and the default value used. It may be needed in some cases, where classes implement multiple interfaces or inherit other classes that do so. Usually, this is not necessary if you follow the Single Responsibility principle.

0.7.0.2 ExtensionPropertyAttribute

Using only the ExtensionAttribute, NUnit would have to create instances of every extension in order to query it (for example) about its capabilities. Since extensions are generally in separate assemblies, this means that many potentially unneeded assemblies would be loaded.

The ExtensionPropertyAttribute avoids this problem by allowing the extension to specify information about what it is able to do. NUnit scans the attributes using Mono.Cecil without actually loading the assembly, so that resources are not taken up by unused assemblies.

To illustrate this, we will use the example of the engine’s project loader ExtensionPoint. You can read about this extension point in detail at [[Project-Loaders]] but the essential thing for this example is that the extension point is passed a file path and must determine whether that file is in a format that it can interpret and load. We do that by loading the extension and calling its CanLoadFrom(string path) method.

If we only knew what file extensions were used by the particular format, we could avoid loading the extension unnecessarily. That’s where ExtensionPropertyAttribute comes in. The following is an example taken from NUnit’s own extension for loading NUnit projects.

    [Extension]
    [ExtensionProperty("FileExtension", ".nunit")]
    public class NUnitProjectLoader : IProjectLoader
    {
        ...
    }

By use of the ExtensionPropertyAttribute the assembly containing this extension will never be loaded unless the user asks NUnit to run tests in a file of type .nunit. If this attribute were not present, then the engine would have to load the assembly, construct the object and call its CanLoadFrom method.

Of course, this means that the extension author must know a great deal about how each extension point works. That’s why we provide a page for each supported extension points with details of how to use it.

0.7.0.3 Locating Addins

Assemblies containing Addins and Extensions are stored in one or more locations indicated in files of type .addins. Each line of the file contains the path of an addin assembly or a directory containing assemblies. Wildcards may be used for assembly entries and relative paths are interpreted based on the location of the .addins file. The default nunit.engine.addins is located in the engine directory and lists addins we build with NUnit, which are contained in the addins directory.

The following is an example of a possible .addins file, with comments indicating what each line does:

# This line is a comment and is ignored. The next (blank) line is ignored as well.

*.dll                   # include all dlls in the same directory
addins/*.dll            # include all dlls in the addins directory too
special/myassembly.dll  # include a specific dll in a special directory
/some/other/directory/  # process another directory, which may contain its own addins file
                        # note that an absolute path is allowed, but is probably not a good idea
                        # in most cases 

Any assemblies specified in a .addins file will be scanned fully, looking for addins and extensions. Any directories specified will be browsed, first looking for any .addins files. If one or more files are found, the content of the files will direct all further browsing. If no such file is found, then all .dll files in the directory will be scanned, just as if a .addins file contained “*.dll."

Assemblies are examined using Cecil, without actually loading them. Info is saved for actual instantiation of extensions on a just-in-time basis.

An EqualConstraint is used to test whether an actual value is equal to the expected value supplied in its constructor, optionally within a specified tolerance.

<h4>Constructor</h4>

EqualConstraint( object expected )

<h4>Syntax</h4>

Is.EqualTo( object expected )
Is.Zero // Equivalent to Is.EqualTo(0)

<h4>Modifiers</h4>

...IgnoreCase
...AsCollection
...NoClip
...WithSameOffset
...Within(object tolerance)
      .Ulps
      .Percent
      .Days
      .Hours
      .Minutes
      .Seconds
      .Milliseconds
      .Ticks
...Using(IEqualityComparer comparer)
...Using(IEqualityComparer<T> comparer)
...Using(IComparer comparer)
...Using<T>(IComparer<T> comparer)
...Using<T>(Comparison<T> comparer)

<h4>Comparing Numerics</h4> Numerics are compared based on their values. Different types may be compared successfully if their values are equal.

Using the <b>Within</b> modifier, numerics may be tested for equality within a fixed or percent tolerance.

Assert.That(2 + 2, Is.EqualTo(4.0));
Assert.That(2 + 2 == 4);
Assert.That(2 + 2, Is.Not.EqualTo(5));
Assert.That(2 + 2 != 5);
Assert.That( 5.0, Is.EqualTo( 5 );
Assert.That( 5.5, Is.EqualTo( 5 ).Within(0.075);
Assert.That( 5.5, Is.EqualTo( 5 ).Within(1.5).Percent;

<h4>Comparing Floating Point Values</h4> Values of type float and double are normally compared using a tolerance specified by the <b>Within</b> modifier. The special values PositiveInfinity, NegativeInfinity and NaN compare as equal to themselves.

Floating-point values may be compared using a tolerance in “Units in the Last Place” or ULPs. For certain types of numerical work, this is safer than a fixed tolerance because it automatically compensates for the added inaccuracy of larger numbers.

Assert.That( 2.1 + 1.2, Is.EqualTo( 3.3 ).Within( .0005 );
Assert.That( double.PositiveInfinity, Is.EqualTo( double.PositiveInfinity ) );
Assert.That( double.NegativeInfinity, Is.EqualTo( double.NegativeInfinity ) );
Assert.That( double.NaN, Is.EqualTo( double.NaN ) );
Assert.That( 20000000000000004.0, Is.EqualTo(20000000000000000.0).Within(1).Ulps);

<h4>Comparing Strings</h4>

String comparisons normally respect case. The <b>IgnoreCase</b> modifier causes the comparison to be case-insensitive. It may also be used when comparing arrays or collections of strings.

Assert.That( "Hello!", Is.Not.EqualTo( "HELLO!" ) );
Assert.That( "Hello!", Is.EqualTo( "HELLO!" ).IgnoreCase );

string[] expected = new string[] { "Hello", World" };
string[] actual = new string[] { "HELLO", "world" };
Assert.That( actual, Is.EqualTo( expected ).IgnoreCase;

0.7.0.4 Comparing DateTimes and TimeSpans

<b>DateTimes</b> and <b>TimeSpans</b> may be compared either with or without a tolerance. A tolerance is specified using <b>Within</b> with either a <b>TimeSpan</b> as an argument or with a numeric value followed by a one of the time conversion modifiers: <b>Days</b>, <b>Hours</b>, <b>Minutes</b>, <b>Seconds</b>, <b>Milliseconds</b> or <b>Ticks</b>.

When comparing <b>DateTimeOffsets</b> you can use the optional <b>WithSameOffset</b> modifier to check the offset along with the date and time.

DateTime now = DateTime.Now;
DateTime later = now + TimeSpan.FromHours(1.0);

Assert.That( now, Is.EqualTo(now) );
Assert.That( later. Is.EqualTo(now).Within(TimeSpan.FromHours(3.0);
Assert.That( later, Is.EqualTo(now).Within(3).Hours;

<h4>Comparing Arrays and Collections</h4>

Since version 2.2, NUnit has been able to compare two single-dimensioned arrays. Beginning with version 2.4, multi-dimensioned arrays, nested arrays (arrays of arrays) and collections may be compared. With version 2.5, any IEnumerable is supported. Two arrays, collections or IEnumerables are considered equal if they have the the same dimensions and if each of the corresponding elements is equal.

If you want to treat two arrays of different shapes as simple collections for purposes of comparison, use the <b>AsCollection</b> modifier, which causes the comparison to be made element by element, without regard for the rank or dimensions of the array. Note that jagged arrays (arrays of arrays) do not have a single underlying collection. The modifier would be applied to each array separately, which has no effect in most cases.

int[] i3 = new int[] { 1, 2, 3 };
double[] d3 = new double[] { 1.0, 2.0, 3.0 };
int[] iunequal = new int[] { 1, 3, 2 };
Assert.That(i3, Is.EqualTo(d3));
Assert.That(i3, Is.Not.EqualTo(iunequal));

int array2x2 = new int[,] { { 1, 2 } { 3, 4 } };
int array4 = new int[] { 1, 2, 3, 4 };      
Assert.That( array2x2, Is.Not.EqualTo( array4 ) );
Assert.That( array2x2, Is.EqualTo( array4 ).AsCollection );

<h4>Comparing Dictionaries</h4>

Two dictionaries are considered equal if

<ol> <li>The list of keys is the same - without regard to ordering. <li>The values associated with each key are equal. </ol>

You can use this capability to compare any two objects implementing <b>IDictionary</b>. Generic and non-generic dictionaries (Hashtables) may be successfully compared.

<h4>Comparing DirectoryInfo</h4>

Two DirectoryInfo objects are considered equal if both have the same path, creation time and last access time.

Assert.That(new DirectoryInfo(actual), Is.EqualTo(expected));

<h4>User-Specified Comparers</h4>

If the default NUnit or .NET behavior for testing equality doesn’t meet your needs, you can supply a comparer of your own through the <b>Using</b> modifier. When used with <b>EqualConstraint</b>, you may supply an <b>IEqualityComparer</b>, <b>IEqualityComparer<T></b>, <b>IComparer</b>, <b>IComparer<T&gt</b>; or <b>Comparison<T></b> as the argument to <b>Using</b>.

Assert.That( myObj1, Is.EqualTo( myObj2 ).Using( myComparer ) );

Prior to NUnit 2.6, only one comparer could be used. If multiple comparers were specified, all but one was ignored. Beginning with NUnit 2.6, multiple generic comparers for different types may be specified. NUnit will use the appropriate comparer for any two types being compared. As a result, it is now possible to provide a comparer for an array, a collection type or a dictionary. The user-provided comparer will be used directly, bypassing the default NUnit logic for array, collection or dictionary equality.

class ListOfIntComparer : IEqualityComparer<List<int>>
{
    ...
}

var list1 = new List<int>();
var list2 = new List<int>();
var myComparer = new ListOfIntComparer();
...
Assert.That( list1, Is.EqualTo(list2).Using( myComparer ) );

<h4>Notes</h4> <ol> <li><p>When checking the equality of user-defined classes, NUnit first examines each class to determine whether it implements <b>IEquatable<T></b>. If either object implements the interface for the type of the other object, then that implementation is used in making the comparison. If neither class implements the appropriate interface, NUnit makes use of the <b>Equals</b> override on the expected object. If you neglect to either implement <b>IEquatable<T></b> or to override <b>Equals</b>, you can expect failures comparing non-identical objects. In particular, overriding <b>operator==</b> without overriding <b>Equals</b> or implementing the interface has no effect. <li><p>The <b>Within</b> modifier was originally designed for use with floating point values only. Beginning with NUnit 2.4, comparisons of <b>DateTime</b> values may use a <b>TimeSpan</b> as a tolerance. Beginning with NUnit 2.4.2, non-float numeric comparisons may also specify a tolerance. <li><p>Beginning with NUnit 2.4.4, float and double comparisons for which no tolerance is specified use a default, use the value of <b>GlobalSettings.DefaultFloatingPointTolerance</b>. If this is not set, a tolerance of 0.0d is used. <li><p>Prior to NUnit 2.2.3, comparison of two NaN values would always fail, as specified by IEEE floating point standards. The new behavior, was introduced after some discussion becuase it seems more useful in tests. To avoid confusion, consider using <b>Is.NaN</b> where appropriate. <li><p>When an equality test between two strings fails, the relevant portion of of both strings is displayed in the error message, clipping the strings to fit the length of the line as needed. Beginning with 2.4.4, this behavior may be modified by use of the <b>NoClip</b> modifier on the constraint. In addition, the maximum line length may be modified for all tests by setting the value of <b>TextMessageWriter.MaximumLineLength</b> in the appropriate level of setup. <li><p>When used with arrays, collections or dictionaries, EqualConstraint operates recursively. Any modifiers are saved and used as they apply to individual items. <li><p>A user-specified comparer will not be called by <b>EqualConstraint</b> if either or both arguments are null. If both are null, the Constraint succeeds. If only one is null, it fails. <li><p>NUnit has special semantics for comparing <b>Streams</b> and <b>DirectoryInfos</b>. For a <b>Stream</b>, the contents are compared. For a <b>DirectoryInfo</b>, the first-level directory contents are compared. </ol>

Event Listeners are extensions that respond to specific events occuring during the running of a test. They implement the ITestEventListener interface. NUnit itself makes extensive use of this interface when running tests. By creating an extension, user code may also respond to test events.

The definition of an Event Listener extension will look something like this:

[Extension(EngineVersion="3.4")]
public class MyEventListener : ITestEventListener
{
    ...
}

Note: The EngineVersion property is used to document the fact that the event listener extension point was only added to the engine with version 3.4. It’s function here is purely documentary because the EngineVersion property itself was also added in version 3.4. Event listeners should not be installed with earlier versions.

The ITestEventListener interface is defined as follows:

/// <summary>
/// The ITestListener interface is used to receive notices of significant
/// events while a test is running. It's single method accepts an Xml string, 
/// which may represent any event generated by the test framework, the driver
/// or any of the runners internal to the engine. Use of Xml means that
/// any driver and framework may add additional events and the engine will
/// simply pass them on through this interface.
/// </summary>
[TypeExtensionPoint(
    Description = "Allows an extension to process progress reports and other events from the test.")]
public interface ITestEventListener
{
    /// <summary>
    /// Handle a progress report or other event.
    /// </summary>
    /// <param name="report">An XML progress report.</param>
    void OnTestEvent(string report);
}

The argument to OnTestEvent is an XML-formatted string, with a different top-level element for each potential event.

  • Start of run - <start-run...>
  • End of run - <test-run...>
  • Start of a test suite - <start-suite...>
  • End of a test suite - <test-suite...>
  • Start of a test case - <start-test...>
  • End of a test case - <test-case...>

The XML report signalling the end of a test case contains all available information, including the result. The start events only provide basic identifying information. See [[XML Formats]] for a full description of each report. ExactCountConstraint applies a constraint to each item in an IEnumerable, succeeding if the specified number of items succeed. An exception is thrown if the actual value passed does not implement IEnumerable.

<h4>Constructor</h4>

ExactCountConstraint(int expectedCount, Constraint itemConstraint)

<h4>Syntax</h4>

Has.Exactly(int expectedCount)...

<h4>Examples of Use</h4>

int[] array = new int[] { 1, 2, 3 };
Assert.That( array, Has.Exactly(1).EqualTo(3) );
Assert.That( array, Has.Exactly(2).GreaterThan(1) );
Assert.That( array, Has.Exactly(3).LessThan(100) );

ExactTypeConstraint tests that an object is an exact Type.

<h4>Constructor</h4>

ExactTypeConstraint( Type )

<h4>Syntax</h4>

Is.TypeOf( Type )
Is.TypeOf<T>()

<h4>Examples of Use</h4>

Assert.That("Hello", Is.TypeOf(typeof(string)));
Assert.That("Hello", Is.Not.TypeOf(typeof(int)));

Assert.That("World", Is.TypeOf<string>();

The Explicit attribute causes a test or test fixture to be skipped unless it is explicitly selected for running. The test or fixture will be run if it is selected by name or if it is included by use of a filter. Note that a not filter, which excludes certain tests, is not treated as an explicit selection and never causes an explicit test to be run.

An optional string argument may be used to give the reason for marking the test Explicit.

If a test or fixture with the Explicit attribute is encountered in the course of running tests, it is skipped unless it has been specifically selected by one of the above means. The test does not affect the overall result of the test run. Explicit tests are displayed in the gui as skipped.

Warning: While the C# syntax allows you to place an Explicit attribute on a SetUpFixture class, the attribute is ignored by NUnit and has no effect in current releases.

<h4>Test Fixture Syntax</h4> #####C# #####

namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture, Explicit]
  public class ExplicitTests
  {
    // ...
  }
}
0.7.0.4.1 Visual Basic
Imports System
Imports Nunit.Framework

Namespace Nunit.Tests

  <TestFixture(), Explicit()>
  Public Class ExplicitTests
    ' ...
  End Class
End Namespace
0.7.0.4.2 C++
using namespace System;
using namespace NUnit::Framework;

namespace NUnitTests
{
  [TestFixture]
  [Explicit]
  public __gc class ExplicitTests
  {
    // ...
  };
}

#include "cppsample.h"

namespace NUnitTests {
  // ...
}

<h4>Test Syntax</h4>

0.7.0.4.3 C#
namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture]
  public class SuccessTests
  {
    [Test, Explicit]
    public void ExplicitTest()
    { /* ... */ }
}
0.7.0.4.4 Visual Basic
Imports System
Imports Nunit.Framework

Namespace Nunit.Tests

  <TestFixture()> Public Class SuccessTests
    <Test(), Explicit()> Public Sub ExplicitTest()
      ' ...
    End Sub
  End Class
End Namespace
0.7.0.4.5 C++
#using <Nunit.Framework.dll>
using namespace System;
using namespace NUnit::Framework;

namespace NUnitTests
{
  [TestFixture]
  public __gc class SuccessTests
  {
    [Test][Explicit] void ExplicitTest();
  };
}

#include "cppsample.h"

namespace NUnitTests {
  // ...
}

NUnit is intended to be extensible. We can’t do everything for everybody but we want to make it reasonably easy to extend NUnit. In many cases, users will be able to implement a special feature outside of our scope by simply creating a new attribute that embeds the required logic. In other cases, particularly in extending the engine, we rely on a plugin-architecture.

0.7.0.5 Types of Extensibility

  • [[Framework Extensibility]]
  • [[Engine Extensibility]] FalseConstraint tests that a value is false.

<h4>Constructor</h4>

FalseConstraint()

<h4>Syntax</h4>

Is.False

<h4>Example of Use</h4>

Assert.That( condition, Is.False );

The FileAssert class provides methods for comparing or verifying the existence of files, which may be provided as Streams, as FileInfos or as strings giving the path to each file.

FileAssert.AreEqual( Stream expected, Stream actual );
FileAssert.AreEqual( 
    Stream expected, Stream actual, string message, params object[] args );

FileAssert.AreEqual( FileInfo expected, FileInfo actual );
FileAssert.AreEqual( 
    FileInfo expected, FileInfo actual, string message, params object[] args );

FileAssert.AreEqual( string expected, string actual );
FileAssert.AreEqual( 
    string expected, string actual, string message, params object[] args );

FileAssert.AreNotEqual( Stream expected, Stream actual );
FileAssert.AreNotEqual( 
    Stream expected, Stream actual, string message, params object[] args );

FileAssert.AreNotEqual( FileInfo expected, FileInfo actual );
FileAssert.AreNotEqual( 
    FileInfo expected, FileInfo actual, string message, params object[] args );

FileAssert.AreNotEqual( string expected, string actual );
FileAssert.AreNotEqual( 
    string expected, string actual, string message, params object[] args );

FileAssert.Exists(FileInfo actual);
FileAssert.Exists(
    FileInfo actual, string message, params object[] args);

FileAssert.Exists(string actual);
FileAssert.Exists(
    string actual, string message, params object[] args);

FileAssert.DoesNotExist(FileInfo actual);
FileAssert.DoesNotExist(
    FileInfo actual, string message, params object[] args);

FileAssert.DoesNotExist(string actual);
FileAssert.DoesNotExist(
    string actual, string message, params object[] args);

0.7.0.6 See also…

FileOrDirectoryExistsConstraint tests that a File or Directory exists.

<h4>Constructor</h4>

FileOrDirectoryExistsConstraint()

<h4>Syntax</h4>

Does.Exist
Does.Not.Exist

<h4>Modifiers</h4>

IgnoreDirectories
IgnoreFiles

<h4>Examples of Use</h4>

Assert.That(fileStr, Does.Exist);
Assert.That(dirStr, Does.Exist);
Assert.That(fileStr, Does.Not.Exist);
Assert.That(dirStr, Does.Not.Exist);

Assert.That(new FileInfo(fileStr), Does.Exist);
Assert.That(new DirectoryInfo(dirStr), Does.Exist);

0.7.1 DRAFT

NUnit 3.0 is intentionally changing certain design decisions made in earlier versions. We document those changes here so that developers may find them all in one place.

For details, see the individual specifications referenced under each design change subheading.

0.7.2 Key Design Changes

0.7.2.1 Multiple Framework Distributions

NUnit 3.0 introduces separate framework builds for each supported runtime version, including .NET 2.0, 3.5, 4.0 and 4.5. We will continue to use the same framework assemblies for both the Microsoft and Mono implementations.

See [[Framework Distributions]]

0.7.2.2 Merge NUnitLite

NUnitLite is a light-weight version of NUnit, requiring minimal resources and running on platforms not supported by full NUnit. NUnitLite is approximately equivalent to the framework portion of NUnit, with the addition of a rudimenatry test runner.

For the NUnit 3.0 release, the NUnitLite project code has been merged with the code of NUnit itself, using conditional compilation to support a reduced feature set.

See [[merge_nunitlite_with_nunit|Merge NUnitLite]]

0.7.2.3 Framework Boundary

Historically, most xUnit frameworks incorporate the logic for loading and running tests in the same module that includes assertions, attributes and other types referenced by the tests. NUnit started that way but the loading and test execution logic was extracted into a separate assembly in later versions.

This approach had some benefit - most notably NUnit’s ability to run tests built against older versions of the framework - but has proven less useful as more features were added. Essentially, the surface area that NUnit presents to a client program wanting to run tests grows each time a new feature is added. This has made it very difficult for third parties to keep up with NUnit’s feature growth.

For NUnit 3.0, the boundary has been moved. Each version of the framework incorporates a test runner that knows how to load and execute the tests supported by that version.

See [Framework Boundary]

0.7.2.4 Framework Api

The internal (core) interfaces used by earlier versions of NUnit are not suitable for external use because they tend to change as features are added. The 3.0 release incorporates a new interface, supporting the same functionality as the existing interfaces, but with fewer dependencies on custom types. It is usable by both NUnit and third-party runners and will provide functions of test discovery, loading and execution.

See [Framework Api]

0.7.2.5 Active Attributes

In the NUnit 2.x series, Attributes are fundamentally passive objects. They are used as markers for certain kinds of functionality, but that functionality is implemented in the nunit.core assembly. Reflection is used to identify attributes, in order to avoid a reference from the core to a specific version of the framework.

In NUnit 3.0, since the test loader is part of the framework assembly, we’ll be able to identify attributes without use of reflection, which should improve load performance. Attributes will actually contain the code that carries out the necessary functionality.

See [[DRAFT:Active Attributes]]

0.7.2.6 Test Loading

Earlier releases of NUnit load tests in a hierarchy based on the namespace and may optionally load them as a flat list of fixtures. The test hierarchy is built as the tests are loaded and reflected in the gui display.

With NUnit 3.0, the test loader will only load fixtures and will not create a hierarchy. It will be the responsibility of the Gui to construct whatever display hierarchy the user chooses as a view of the tests.

This will simplify the loading of tests and is compatible with NUnitLite, which already loads tests this way.

See [Test Loading]

0.7.2.7 Test Execution

Currently, the sequencing of tests is the responsibility of the tests themselves. That is, each test suite executes its own child tests and each test reports its own results.

The absence of a distinct test execution object makes it difficult to support certain features, such as high-level setup and teardown, parallel test executiono and cancellation of running tests with restart.

In NUnit 3.0, the objects representing tests will no longer have the responsibility of running any subordinate tests and a separate test runner will sequence through the tests to execute them in the desired order.

See [Test Execution]

Framework Drivers are extensions that know how to create a driver for a particular framework. The NUnit engine provides drivers for both the NUnit 3.x and NUnit 2.x frameworks. Third parties may provide drivers for other frameworks by creating extensions.

The ExtensionPoint for framework drivers uses the Path “NUnit.Engine.DriverService” and accepts an extension of Type NUnit.Engine.Extensibility.IDriverFactory. The definition of a driver factory might look like this:

[Extension]
public class MyOwnFrameworkDriverFactory : IDriverFactory
{
    ...
}

The IDriverFactory interface is defined as follows:

public interface IDriverFactory
{
    // Gets a flag indicating whether the provided assembly name and version
    // represent a test framework supported by this factory
    bool IsSupportedTestFramework(string assemblyName, Version version);

    // Gets a driver for use with the specified framework assembly name and version
    IFrameworkDriver GetDriver(AppDomain domain, string assemblyName, Version version);
}

The IFrameworkDriver interface is defined as follows:

public interface IFrameworkDriver
{
    // Gets and sets the unique identfier for this driver
    string ID { get; set; }

    // Loads the tests in an assembly
    string Load(string testAssemblyPath, IDictionary<string, object> settings);

    // Counts the test cases that would be executed
    int CountTestCases(string filter);

    // Executes the tests in an assembly
    string Run(ITestEventListener listener, string filter);

    // Returns information about the tests in an assembly
    string Explore(string filter);

    // Cancels an ongoing test run.
    void StopRun(bool force);

The strings returned by Run and Explore are XML representations and the filter is also in XML format. See the source code for NUnit3FrameworkDriver and NUnit2FrameworkDriver for details. The NUnit Framework is the part of NUnit that is referenced by user tests. It contains the definition of all of NUnit’s Attributes, Constraints and Asserts as well as the code that discovers and executes tests. Most extensions to exactly how tests are recognized and how they execute are Framework extensions.

In this documentation, we refer to four different types of Framework extension:

[[Action Attributes]] are designed to better enable composability of test logic by creating attributes that encapsulate specific actions to be taken before or after a test is run.

[[Custom Attributes]] go beyond simple before/after logic and allow creation of new types of tests and suites, new sources of data and modification of the environment in which a test runs as well as its final result.

[[Custom Constraints]] allow the user to define new constraints for use in tests along with the associated fluent syntax that allows them to be used with Assert.That.

[[Custom Asserts]] are, for the most part, no longer needed as a result the constraint-based assertion model. However, it is still possible to create them and custom asserts created for NUnit V2 are still useable.

GreaterThanConstraint tests that one value is greater than another.

<h4>Constructor</h4>

GreaterThanConstraint(object expected)

<h4>Syntax</h4>

Is.GreaterThan(object expected)
Is.Positive // Equivalent to Is.GreaterThan(0)

<h4>Modifiers</h4>

...Using(IComparer comparer)
...Using<T>(IComparer<T> comparer)
...Using<T>(Comparison<T> comparer)

<h4>Examples of Use</h4>

Assert.That(7, Is.GreaterThan(3));
Assert.That(myOwnObject, 
    Is.GreaterThan(theExpected).Using(myComparer));
Assert.That(42, Is.Positive);

GreaterThanOrEqualConstraint tests that one value is greater than or equal to another.

<h4>Constructor</h4>

GreaterThanOrEqualConstraint(object expected)

<h4>Syntax</h4>

Is.GreaterThanOrEqualTo(object expected)
Is.AtLeast(object expected)

<h4>Modifiers</h4>

...Using(IComparer comparer)
...Using<T>(IComparer<T> comparer)
...Using<T>(Comparison<T> comparer)

<h4>Examples of Use</h4>

Assert.That(7, Is.GreaterThanOrEqualTo(3));
Assert.That(7, Is.AtLeast(3));
Assert.That(7, Is.GreaterThanOrEqualTo(7));
Assert.That(7, Is.AtLeast(7));
Assert.That(myOwnObject, 
    Is.GreaterThanOrEqualTo(theExpected).Using(myComparer));

IgnoreAttribute is used to indicate that a test should not be executed for some reason. Note that with NUnit 3.0, the reason must be specified. Ignored tests are displayed by the runners as warnings in order to provide a reminder that the test needs to be corrected or otherwise changed and re-instated.

0.7.2.8 Test Fixture Syntax

namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture]
  [Ignore("Ignore a fixture")]
  public class SuccessTests
  {
    // ...
  }
}

<h4>Test Syntax</h4>

namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture]
  public class SuccessTests
  {
    [Test]
    [Ignore("Ignore a test")]
    public void IgnoredTest()
    { /* ... */ }
}

0.7.2.9 Ignore Until

The Until named parameter allows you to ignore a test for a specific period of time, after which the test will run normally. The until date must be a string that can be parsed to a date.

[TestFixture]
[Ignore("Waiting for Joe to fix his bugs", Until = "2014-07-31 12:00:00Z"]
public class MyTests
{
 [Test]
 public void Test1() { /* ... */ }
}

In the above example, it’s assumed that the test would fail if run. With the IgnoreAttribute, it will give a warning until the specified date. After that time, it will run normally and either pass or fail. To get a copy of the NUnit 3, you can use various installation approaches.

  1. Zip file download.
  2. Full NUnit install via NuGet.
  3. NUnitLite install via NuGet.
  4. Combined Approach

0.7.3 Downloading the Zip File

Download the latest binary zip from our Download page or from GitHub. Unzip the file into any convenient directory.

If you would like be able to run nunit3-console from the command line, put the bin directory, containing nunit3-console.exe on your path.

In your test assemblies, add a reference to nunit.framework.dll, using the copy in the subdirectory for the appropriate runtime version. For example, if you are targeting .NET 4.0, you should reference the framework assembly in the net-4.0 subdirectory.

Run nunit3-console from the command line, giving it the path to your test assembly. To run NUnit’s own framework tests from within the NUnit bin directory, enter:

     nunit3-console net-2.0/nunit.framework.tests.dll

0.7.4 Using NuGet Packages

In Visual Studio, from the Tools menu, select NuGet Package Manager | Manage NuGet packages for solution…

Scroll or use search to locate the NUnit and NUnit.Console packages. Install both packages. The NUnit package should be referenced by each of your test assemblies, but not by any others.

Locate nunit3-console in the **packages.ConsoleRunner.3.X.X* directory under your solution. This is the location from which you must run nunit3-console when using this approach. We recommend you only use this approach when running under the control of a script on your build server.

0.7.5 Using NuGet NUnitLite Package

The NUnitLite approach provides a way to run NUnit tests without a full install of the NUnit runner and test engine assemblies. Only the framework and a small runner program are installed. Note that this is currently the only way to run tests under Silverlight 5.0 or the compact framework.

In Visual Studio, from the Tools menu, select NuGet Package Manager | Manage NuGet packages for solution…

Scroll or use search to locate the NUnitLite Version 3 and install it. The package should be referenced by each of your test assemblies, but not by any others.

As a result of your installation, a file named “Program.cs” will have been copied into each of your test projects. NUnitLite test projects are console applications and this file contains the Main() program for your tests. If you already have a Main() in some other class, you may delete the file. This file also contains comments describing how the main should call the NUnitLite runner.

To run your tests, simply run your executable test assembly. No other runner is needed.

0.7.6 Combined Approach

This approach is useful if you would like to use a single copy of nunit3-console with individual copies of the framework in each project.

Simply follow the zip file procedure to get a central copy of NUnit on your system. Then install the NUnit Version 3 NuGet package in each of your test assemblies. For desktop use by developers, this approach may give you the best of both worlds. InstanceOfTypeConstraint tests that an object is of the type supplied or a derived type.

<h4>Constructor</h4>

InstanceOfTypeConstraint( Type )

<h4>Syntax</h4>

Is.InstanceOf( Type )
Is.InstanceOf<T>()

<h4>Examples of Use</h4>

Assert.That("Hello", Is.InstanceOf(typeof(string)));
Assert.That(5, Is.Not.InstanceOf(typeof(string)));

Assert.That(5, Is.Not.InstanceOf<string>());

0.7.7 DRAFT

NUnit traps text output directed to the Console, Trace or a logging subsytem. This makes it difficult to use text output when debugging the framework itself, particularly when working on the classes that redirect output. For that reason, NUnit incorporates a simple internal tracing mechanism for use by developers and for debugging problems in the field.

0.7.8 User Stories

0.7.8.0.1 An NUnit User…
  • enables NUnit’s internal trace in order to collect information about an apparent bug or unexplained error.
0.7.8.0.2 An NUnit Developer…
  • makes use of the internal trace as a part of the development cycle.

0.7.9 Design

NUnit provides a simple internal trace facility using the InternalTrace class. The following public methods are provided for creating output:

public static void Error(string message, params object[] args)
public static void Warning(string message, params object[] args)
public static void Info(string message, params object[] args)
public static void Debug(string message, params object[] args)

By default, all output through InternalTrace is sent to the console, which means it is mixed with other console output. Note that the trace mechanism bypasses NUnit’s redirection of output and displays on the actual Console. Of course, if NUnit is being run without a console - from the Gui, for example - the output will be lost.

0.7.9.1 Redirecting Output

The InternalTrace.Open() method allows the programmer to specify a file to which the output will be written. Any absolute or relative path may be used as an argument. Relative paths are interpreted as relative to the location of the assembly being tested. A programmer working on NUnit may call this method at any point in the code. InternalTrace.Close() is also provided.

<note> Since the path is saved as a static property, it must be specified separately within each AppDomain. Writing to the same file from two different AppDomains is not supported. </note>

0.7.9.2 Specifying Verbosity

Verbosity is specified using the nested enum InternalTrace.TraceLevel. Although the values currently match those of System.Diagnostics.TraceLevel, they have no actual relation to that class. Additional values may be added in the future.

public class InternalTrace
{
    public enum TraceLevel
    {
        Off,
        Error,
        Warning,
        Info,
        Debug
    }
    ...
}

A programmer working on NUnit may set the TraceLevel at any point in the code by setting the InternalTrace.Level property.

0.7.10 Unresolved Issues

  • How should the destination and verbosity level of internal trace be specified by a user?

LessThanConstraint tests that one value is less than another.

<h4>Constructor</h4>

LessThanConstraint(object expected)

<h4>Syntax</h4>

Is.LessThan(object expected)
Is.Negative // Equivalent to Is.LessThan(0)

<h4>Modifiers</h4>

...Using(IComparer comparer)
...Using<T>(IComparer<T> comparer)
...Using<T>(Comparison<T> comparer)

<h4>Examples of Use</h4>

Assert.That(3, Is.LessThan(7));
Assert.That(myOwnObject, 
    Is.LessThan(theExpected).Using(myComparer));
Assert.That(-5, Is.Negative);

LessThanOrEqualConstraint tests that one value is less than or equal to another.

<h4>Constructor</h4>

LessThanOrEqualConstraint(object expected)

<h4>Syntax</h4>

Is.LessThanOrEqualTo(object expected)
Is.AtMost(object expected)

<h4>Modifiers</h4>

...Using(IComparer comparer)
...Using<T>(IComparer<T> comparer)
...Using<T>(Comparison<T> comparer)

<h4>Examples of Use</h4>

Assert.That(3, Is.LessThanOrEqualTo(7));
Assert.That(3, Is.AtMost(7));
Assert.That(3, Is.LessThanOrEqualTo(3));
Assert.That(3, Is.AtMost(3));
Assert.That(myOwnObject, 
    Is.LessThanOrEqualTo(theExpected).Using(myComparer));

This is an assembly-level attribute, which may be used to specify the level of parallelism, that is, the maximum number of worker threads executing tests in the assembly. It may be overridden using a command-line option in the console runner.

This attribute is optional. If it is not specified, NUnit uses the processor count or 2, whichever is greater. For example, on a four processor machine the default value is 4.

0.7.10.1 Example

The following code, which might be placed in AssemblyInfo.cs, sets the level of parallelism to 3:

  [assembly:LevelOfParallelism(3)]

0.7.10.2 See also…

  • [[Parallelizable Attribute]]

<h2>NUnit License</h2>

<p><b>Copyright (c) 2004-2015 Charlie Poole</b></p>

<p>Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:</p>

<p>The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.</p>

<p>THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.</p>

Unlike Constraint classes, <b>ListMapper</b> is used to modify the actual value argument to Assert.That(). It transforms the actual value, which must be a collection, creating a new collection to be tested against the supplied constraint. Currently, ListMapper supports one transformation: creating a collection of property values.

Normally, ListMapper will be used through the <b>List.Map()</b> syntax helper or the inherited syntax equivalent, <b>Map()</b>. The following example shows three forms of the same assert:

string[] strings = new string[] { "a", "ab", "abc" };
int[] lengths = new int[] { 1, 2, 3 };

Assert.That(List.Map(strings).Property("Length"), 
       Is.EqualTo(lengths));
       
Assert.That(new ListMapper(strings).Property("Length"),
       Is.EqualTo(lengths));

// Assuming inheritance from AssertionHelper
Expect(Map(strings).Property("Length"), EqualTo(lengths));

The <b>MaxtimeAttribute</b> is used on test methods to specify a maximum time in milliseconds for a test case. If the test case takes longer than the specified time to complete, it is reported as a failure.

0.7.10.3 Example

[Test, Maxtime(2000)]
public void TimedTest()
{
    ...
}

0.7.10.4 Notes:

  1. Any assertion failures take precedence over the elapsed time check.

  2. This attribute does not cancel the test if the time is exceeded. It merely waits for the test to complete and then compares the elapsed time to the specified maximum. If you want to cancel long-running tests, see [[Timeout Attribute]].

This is the original - now out of date - document created to describe the planned architecture for NUnit 3.0. We are keeping it for whatever historical interest it may have.

The diagram was created first and was shown for the first time by Charlie Poole at a Mono gathering in Madrid in 2008. The complete document was published the following year.

0.7.11 Summary

This specification describes the overall layered architecture of the NUnit Platform. Details of individual components are covered only as they are relevant to the overall architecture.

0.7.11.0.1 Launchpad Blueprint
  • http://blueprints.launchpad.net/nunit-3.0/+spec/nunit-3.0-architecture

0.7.12 User Stories

0.7.12.0.1 An NUnit User…
  • upgrades selected components of the NUnit platform without changing others.
  • runs tests using alternate runners and frameworks under the NUnit platform.
  • installs and uses addins and other extensions to the NUnit platform.
0.7.12.0.2 An NUnit Developer…
  • understands how each component fits into the overall architecture of NUnit.
  • chooses the proper component and project for making a fix or adding a feature.
  • is able to focus attention on the component being modified, without worrying about the details of other components.
0.7.12.0.3 An NUnit Extension Developer…
  • can develop an extension to a specific NUnit component, without dealing with other parts.
  • can develop extensions to multiple NUnit components, which work together to achieve their result.

0.7.13 Layers

The basic architecture of the NUnit Extended testing platform consists of three layers:

  • [[Architectural Overview#Test Runner Layer]]
  • [[Architectural Overview#Test Engine Layer]]
  • [[Architectural Overview#Framework Layer]]

[[images/nunit-xtp-2008.png]]

0.7.13.1 Test Runner Layer

The Test Runner or UI layer contains various runners, some provided by the NUnit team, others by independent projects leveraging the NUnit platform. Some runners are stand-alone programs, while others are tasks or plugins running under the control of an IDE or other application. This diversity of runners is part of the reason we refer to NUnit 3.0 as a Testing Platform – we expect many different runners to come into existence and will facilitate their development by providing reusable controls for several key environments.

Programs in this layer are able to participate in the NUnit platform plugin architecture, providing extension points that allow them to be extended. Plugins at this level will usually add some functionality to the UI. Some of them may be standalone while others may require the presence of specific test engine plugins in order to operate.

The NUnit project will continue to provide both a console runner and a WinForms-based GUI with extended capabilities those in NUnit 2.5. In addition, two new GUI runners will be developed, one based on WPF, the other on GTK#.

We’ll work with the NAnt project to provide updates to the NAnt task for use with NUnit 3.0, with the goal of keeping that task current as new versions of NUnit are released. We will provide an NUnit plugin for the new Gallio platform.

In the area of IDE integration, we will deliver a Visual Studio addin or package for running NUnit tests. Since other folks are already providing open source plugins for SharpDevelop and MonoDevelop, we’ll work with them to ensure compatibility.

There is a long-standing need for a runner that executes tests in an actual or simulated web server environment. While good practice calls for extracting as much functionality as possible into separately testable assemblies, more complex applications often contain code which can only be tested in such an environment. For that reason, NUnit 3.0 will feature a web runner, which allows tests to be executed on a web server, reporting results back to the desktop.

Each of the runners will have the option of participating in the NUnit plugin architecture and using functionality provided by NUnit to whatever degree desired. For the runners developed directly by the NUnit team, this capability will be used to the max, allowing others to add GUI features that function by themselves or in conjunction with other plugins operating at the level of the test engine.

0.7.13.2 Test Engine Layer

The Test Engine Layer is the core of the NUnit platform. It provides a public API for use by applications that want to locate, load and run tests and display test results. Many aspects of the Test Engine are already present in NUnit 2.4, while others are new. I’ll focus on the new features here.

NUnit 2.5 already supports running tests in a separate process, allowing selection of the CLR version under which the test is to be run. NUnit 3.0 will extend this feature to allow test processes to run on other machines. It will also support distribution of test execution across a network of remote Test Agents. This facility is intended to be used in several distinct scenarios: * Simple load sharing when test execution time is excessive * Testing applications on multiple platforms * Testing applications, which are themselves distributed A great deal of distributed function is already present in NUnit 2.5 through pNUnit, the distributed testing runner contributed to our project by Codice Software. With NUnit 3.0, we plan to integrate this function more completely into NUnit, allowing them to be executed by any runner that uses the NUnit Test Engine.

NUnit will support performance and coverage analysis, test result reporting and maintenance of a history of test results. These features will function only as called upon by the user. That is, we will not carry on data collection activities with the potential of impacting performance when the user simply wants to run tests.

Through use of plugins, NUnit will be able to support a wide variety of test types beyond low-level, isolated programmer tests. What is available in this area will be dependent on the interests of users and their willingness to contribute their efforts to creating them. Examples of the kinds of things we envision, some of which we will provide directly, are: * Randomization of test execution order * Deterministic test ordering for integration tests * Parameterized (data-driven) tests * Transactional test behavior * Timed tests and parameterized timeout failures * Dynamic test generation based on data input * Repetitive test execution * Tests written as non-managed code * Test generation for legacy applications

0.7.13.3 Framework Layer

In NUnit 3.0, the NUnit framework itself – the assembly that is referenced by user tests – will be split along two dimensions. First, there will be separate framework assemblies for different version levels of the Common Language Runtime. By splitting the framework in this way, we will be able to take advantage of newer features, and allow users to take advantage of them, without compromising basic support for older runtimes. For a list of what we currently expect to support see the [[Framework Distributions]] page.

The second split we plan is between the core framework capabilities and the syntactic features that make it easy to access those features. A key example of this is the fluent interface introduced in NUnit 2.4 – the “Assert.That” syntax. One thing we learned through that experiment is that the same syntactic “sugar” does not work well for different language environments. Many of the 2.4/2.5 constructs are unusable or very difficult to use in other languages – C++ for example. By a combination of separate namespaces and separate assemblies, we will allow users to select the appropriate syntax for the work they are doing. Other people will be able to build on the syntax we provide or create entirely new syntactic overlays for their own purposes.

Through use of plugins in the Test Engine layer, NUnit will be able to recognize, load and run tests written using other frameworks. Our focus will be on facilitating the creation of plugins in support of external frameworks by the projects that produce those frameworks or by interested users. For frameworks where that sort of support is not possible – commercial frameworks, for example – we have the option of creating the plugins ourselves.

In some cases, individuals with an idea for a new framework may be able to create them more simply by writing an adapter on top of the NUnit framework itself. We will encourage and support this by giving priority to framework modifications that provide the necessary infrastructure for such projects.

NUnit will also provide or re-package some framework extensions for specific types of applications, including Windows Forms development, WPF development, web page access, XML testing, performance measurement and load testing. At this point, it is not yet possible to state which of these will involve existing third-party packages and which will be developed from scratch, because discussions are ongoing.

The NUnitLite framework will be supported running on devices and reporting results back to the Test Engine through a communications channel.

Various popular mock frameworks will be supported. One framework will be selected for use by NUnit’s own self-tests and will be packaged with NUnit.

0.7.14 Further Details

More detailed specifications are being developed for each of the layers. Consult the [[Specifications]] index for their current status. <p>The nunit-agent.exe program is used by other runners when the tests are being run in a separate process. It is not intended for direct execution by users.

<p>NUnit runs tests in a separate process by default.

<p>When running under the Gui, NUnit will continue to use the same process to reload tests so that it is not normally necessary to re-attach to a new process. However, if the settings are changed in a way that requires a different process - for example, by changing the version of the runtime that is being used - the old process will be terminated and a new one created. In that case, it’s necessary to re-attach to the new process.

<h4>Debugging</h4>

<p>When debugging tests that are run in a separate process, it is not possible to do so by simply running the console or gui runner under the debugger. Rather, it is necessary to attach the debugger to the nunit-agent process after the tests have been loaded.

This documentation covers NUnit 3.0 and higher.

Where applicable, we have marked sections with the version in which a feature first appeared.

If you are new to NUnit, we suggest you begin by reading the Getting Started section of this site. Those who have used earlier releases may want to begin with the [[Upgrading]] section.

See the [[Release Notes]] for more information on each release.

0.7.14.1 Internals

0.8 <NUnitProject>

The required root element for any NUnit project file. * Containing Element: None * Contained Elements: <Settings>, <Config> * Attributes: None

0.9 <Settings>

Contains global settings that apply to all configurations in the project. May be empty or absent. * Containing Element: <NUnitProject> * Contained Elements: None * Attributes: * activeConfig The name of the config to use if none is specified by the user. Using nunit-console, you may override this through the --config command-line option. * appbase The application base to use in loading and running tests. Defaults to the location of the .nunit project file. If the path is relative, as it normally is, it is taken as relative to the location of the project file. * processModel Specifies how NUnit should create processes for executing test assemblies. Possible values are: Default, Single (no separate processes are created), Separate (tests are run in a single, separate process) and Multiple (each assembly is run in it’s own separate process). The default value is Multiple, provided there are multiple assemblies, otherwise Separate. * domainUsage Specifies how NUnit should create AppDomains within each process for running tests. Possible values are: Default, None (no domain is created), Single (a single domain is created) and Multiple (a separate domain is created for each assembly). The default is Multiple if multiple assemblies are run in the same process, Single if only one assembly is run in a process.

0.10 <Config>

Describes a specific configuration of the project. This may map to traditional compiler configs like Debug or Release or may be used to specify any arbitrary collection of assemblies to be tested together. At least one configuration should be specified or the project will not be usable. * Containing Element: <NUnitProject> * Contained Elements: <assembly> * Attributes: * name The name of this configuration. (Required) * appbase The application base to use in loading and running tests under this config. Defaults to the appbase specified in the <Settings> element. The path should normally be relative to that global application base or to the location of project file if there is no global appbase. * binpath The probing path used to search for assemblies, consisting of a number of directory paths separated by semicolons. The directory paths should be relative to the application base and must be under it. Specifying this attribute automatically sets the binpathType to ‘manual’. * binpathType Indicates how the probing path is determined. Possible values are: Auto (the probing path is determined from the location of the test assemblies), Manual (the path is specified by the binpath attribute) and None (no probing path is used). It is an error to specify a value other than Manual if the binpath attribute is also used. * configFile Specifies the path to a config file to be used when running tests under this configuration. The path is relative to the application base. * processModel Specifies how NUnit should create processes for executing test assemblies under this configuration. Possible values are: Default, Single (no separate processes are created), Separate (tests are run in a single, separate process) and Multiple (each assembly is run in it’s own separate process). The default is the value specified globally or Multiple if nothing has been specified. * domainUsage Specifies how NUnit should create AppDomains within each process for running tests under this configuration. Possible values are: Default, None (no domain is created), Single (a single domain is created) and Multiple (a separate domain is created for each assembly). The default is the value specified globally, if provided, otherwise Multiple if multiple assemblies are run in the same process, Single if only one assembly is run in a process. * runtimeFramework Specifies a runtime framework to be used in running tests. Abbreviations are the same as those accepted by the nunit-console command-line. If none is specified, tests are run under the target runtime specified when the assembly was compiled.

0.11 <assembly>

Specifies a single assembly containing tests. * Containing Element: <Config> * Contained Elements: None * Attributes: * path The path to the test assembly, relative to the application base for its configuration.

Although you may simply enter multiple assembly names on the console command-line, running tests from multiple assemblies is facilitated by the use of NUnit test projects. These are files with the extension .nunit containing information about the assemblies to be loaded. The following is an example of a hypothetical test project file:

<NUnitProject>
  <Settings activeconfig="Debug"/>
  <Config name="Debug">
    <assembly path="LibraryCore\bin\Debug\Library.dll"/>
    <assembly path="LibraryUI\bin\Debug\LibraryUI.dll"/>
  </Config>
  <Config name="Release">
    <assembly path="LibraryCore\bin\Release\Library.dll"/>
    <assembly path="LibraryUI\bin\Release\LibraryUI.dll"/>
  </Config>
</NUnitProject>

This project contains two configurations, each of which contains two assemblies. The Debug configuration is currently active. By default, the assemblies will be loaded using the directory containing this file as the ApplicationBase. The PrivateBinPath will be set automatically to LibraryCore\bin\Debug;LibraryUI\bin\Debug or to the corresonding release path. XML attributes are used to specify non-default values for the ApplicationBase, Configuration File and PrivateBinPath.

The [[Project Editor]] may be used to create or modify NUnit projects. If you edit the XML manually, you should make sure to use a path relative to the directory containing the project file as the location of any assemblies.

For details, see [[NUnit Project XML Format]]

0.11.0.1 Command-line Overrides

The following command-line options override what is specified in the NUnit project file:

  • –config
  • –domain
  • –process
  • –framework

<h3>NUnitLite Options</h3>

The NUnitLite runner is invoked by executing the test program. If desired, any of the following options may be specified:

<table class=“nunit” style=“margin-left:0; max-width:700px”>

<tr><th>–test=NAMES</th> <td>Comma-separated list of NAMES of tests to run or explore. This option may be repeated.</td></tr>

<tr><th>–testlist=PATH</th> <td>File PATH containing a list of tests to run, one per line. This option may be repeated.</td></tr>

<tr><th>–include=CATEGORIES</th> <td>Test CATEGORIES to be included. May be a single category, a comma-separated list of categories or a category expression.</td></tr>

<tr><th>–exclude=CATEGORIES</th> <td>Test CATEGORIES to be excluded. May be a single category, a comma-separated list of categories or a category expression.</td></tr>

<tr><th>–timeout=MILLISECONDS</th> <td>Set timeout for each test case in MILLISECONDS.</td></tr>

<tr><th>–seed=SEED</th> <td>Set the random SEED used to generate test cases.</td></tr>

<tr><th>–workers=NUMBER</th> <td>Specify the NUMBER of worker threads to be used in running tests.</td></tr>

<tr><th>–stoponerror</th> <td>Stop run immediately upon any test failure or error.</td></tr>

<tr><th>–wait</th> <td>Wait for input before closing console window.</td></tr>

<tr><th>–pause</th> <td>Pause before run to allow debugging.</td> </tr> <tr><th>–work=PATH</th> <td>PATH of the directory to use for output files.</td></tr>

<tr><th>–output, –out=PATH</th> <td>File PATH to contain text output from the tests.</td></tr>

<tr><th>–err=PATH</th> <td>File PATH to contain error output from the tests.</td></tr>

<tr><th>–result=SPEC</th> <td>An output SPEC for saving the test results. This option may be repeated.</td></tr>

<tr><th>–explore[=SPEC]</th> <td>Display or save test info rather than running tests. Optionally provide an output SPEC for saving the test info. This option may be repeated.</td></tr>

<tr><th>–noresult</th> <td>Don’t save any test results.</td></tr>

<tr><th>–labels=VALUE</th> <td>Specify whether to write test case names to the output. Values: Off, On, All</td></tr>

<tr><th>–trace=LEVEL</th> <td>Set internal trace LEVEL. Values: Off, Error, Warning, Info, Verbose (Debug)</td></tr>

<tr><th>–teamcity</th> <td>Turns on use of TeamCity service messages.</td></tr>

<tr><th>–noheader, –noh</th> <td>Suppress display of program information at start of run.</td></tr>

<tr><th>–nocolor, –noc</th> <td>Displays console output without color.</td></tr>

<tr><th>–verbose, -v</th> <td>Display additional information as the test runs.</td></tr>

<tr><th>–help, -h</th> <td>Display this message and exit.</td></tr>

</table>

<h3>Description</h3>

<p>By default, this command runs the tests contained in the assemblies and projects specified. If the <b>–explore</b> option is used, no tests are executed but a description of the tests is saved in the specified or default format.</p>

<p>Several options that specify processing of XML output take an output specification as a value. A SPEC may take one of the following forms:</p>

<ul> <li>–OPTION:filename <li>–OPTION:filename;format=formatname <li>–OPTION:filename;transform=xsltfile </ul>

<p>The –result option may use any of the following formats: <ul> <li>nunit3 - the native XML format for NUnit 3.0 <li>nunit2 - legacy XML format used by earlier releases of NUnit</p> </ul>

<p>The –explore option may use any of the following formats: <ul> <li>nunit3 - the native XML format for NUnit 3.0 <li>cases - a text file listing the full names of all test cases.</p> </ul>

<p>If –explore is used without any specification following, a list of test cases is output to the console.</p>

<p>If none of the options {–result, –explore, –noxml} is used, NUnit saves the results to TestResult.xml in nunit3 format.</p>

<p>Any transforms provided must handle input in the native nunit3 format.</p>

NUnitLite started out life as a separate version of the NUnit framework, with fewer features and a built-in test runner. It ran on devices as well as on desktop .NET and mono and allowed users to create executable tests without the overhead of a full NUnit installation.

With NUnit 3.0, the NUnitLite framework has been merged with the full NUnit framework. There is now only one framework, <b>nunit.framework.dll</b>. The built-in runner, which was a part of the old NUnitLite, is now packaged as <b>nunitlite.dll</b>.

<h4>Using NUnitLite</h4>

To run tests under NUnitLite, proceed as follows:

  1. Create a console application to use as your test assembly. If you have already placed tests in a dll, you can change the project so that it produces a console application and reload it.

  2. Make sure your test assembly references both <b>nunit.framework</b> and <b>nunitlite</b>.

  3. The Main() for your console application should look something like this… C# public static int Main(string[] args) { return new AutoRun().Execute(args); }

  4. Execute your test application in order to run the tests.

If you install the NUnitLite runner via the NuGet package, steps 2 is handled automatically. Both assemblies are installed and referenced for you.

In addition, if you are using C#, step 3 is handled as well. A file, Program.cs, is added to your project with a Main() like that above.

Note: If you are not working in C#, you should delete the Program.cs file that the NuGet package added to your test project.

<h4>NUnitLite Output</h4>

As seen in the following screen shot, the output from an NUnitLite run is quite similar to that from the console runner.

Screenshot of NUnitLite
Screenshot of NUnitLite

NaNConstraint tests that a value is floating-point NaN.

<h4>Constructor</h4>

NaNConstraint()

<h4>Syntax</h4>

Is.NaN

<h4>Examples of Use</h4>

Assert.That( aDouble, Is.NaN );
Assert.That( aDouble, Is.Not.NaN );

NoItemConstraint applies a constraint to each item in a collection, succeeding only if all of them fail. An exception is thrown if the actual value passed does not implement IEnumerable.

<h4>Constructor</h4>

NoItemConstraint(Constraint itemConstraint)

<h4>Syntax</h4>

Has.None...

<h4>Examples of Use</h4>

int[] iarray = new int[] { 1, 2, 3 };
string[] sarray = new string[] { "a", "b", "c" };
Assert.That( iarray, Has.None.Null );
Assert.That( sarray, Has.None.EqualTo("d") );
Assert.That( iarray, Has.None.LessThan(0) );

NotConstraint reverses the effect of another constraint. If the base constraint fails, NotConstraint succeeds. If the base constraint succeeds, NotConstraint fails.

<h4>Constructor</h4>

NotConstraint()

<h4>Syntax</h4>

Is.Not...

<h4>Examples of Use</h4>

Assert.That( collection, Is.Not.Unique );
Assert.That( 2 + 2, Is.Not.EqualTo(5) );

NullConstraint tests that a value is null.

<h4>Constructor</h4>

    NullConstraint()

<h4>Syntax</h4>

    Is.Null

<h4>Examples of Use</h4>

   Assert.That( anObject, Is.Null );
   Assert.That( anObject, Is.Not.Null );

<p>This attribute is to identify methods that are called once prior to executing any of the tests in a fixture. It may appear on methods of a TestFixture or a SetUpFixture.

<p>OneTimeSetUp methods may be either static or instance methods and you may define more than one of them in a fixture. Normally, multiple OneTimeSetUp methods are only defined at different levels of an inheritance hierarchy, as explained below.

<p>If a OneTimeSetUp method fails or throws an exception, none of the tests in the fixure are executed and a failure or error is reported.

<h4>Example:</h4>

namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture]
  public class SuccessTests
  {
    [OneTimeSetUp]
    public void Init()
    { /* ... */ }

    [OneTimeTearDown]
    public void Cleanup()
    { /* ... */ }

    [Test]
    public void Add()
    { /* ... */ }
  }
}

<h3>Inheritance</h3>

<p>The OneTimeSetUp attribute is inherited from any base class. Therefore, if a base class has defined a OneTimeSetUp method, that method will be called before any methods in the derived class.

<p>You may define a OneTimeSetUp method in the base class and another in the derived class. NUnit will call base class OneTimeSetUp methods before those in the derived classes.

<h4>Notes:</h4> <ol> <li>Although it is possible to define multiple OneTimeSetUp methods in the same class, you should rarely do so. Unlike methods defined in separate classes in the inheritance hierarchy, the order in which they are executed is not guaranteed. <li>OneTimeSetUp methods may be async if running under .NET 4.0 or higher. </ol>

<h4>See also…</h4> * [[SetUp Attribute]] * [[TearDown Attribute]] * [[OneTimeTearDown Attribute]]

<p>This attribute is to identify methods that are called once after executing all the tests in a fixture. It may appear on methods of a TestFixture or a SetUpFixture.

<p>OneTimeTearDown methods may be either static or instance methods and you may define more than one of them in a fixture. Normally, multiple OneTimeTearDown methods are only defined at different levels of an inheritance hierarchy, as explained below.

<p>So long as any OneTimeSetUp method runs without error, the OneTimeTearDown method is guaranteed to run. It will not run if a OneTimeSetUp method fails or throws an exception.</p>

<h4>Example:</h4>

namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture]
  public class SuccessTests
  {
    [OneTimeSetUp]
    public void Init()
    { /* ... */ }

    [OneTimeTearDown]
    public void Cleanup()
    { /* ... */ }

    [Test]
    public void Add()
    { /* ... */ }
  }
}

<h3>Inheritance</h3>

<p>The OneTimeTearDown attribute is inherited from any base class. Therefore, if a base class has defined a OneTimeTearDown method, that method will be called after any test methods in the derived class.

<p>You may define a OneTimeTearDown method in the base class and another in the derived class. NUnit will call base class OneTimeTearDown methods after those in the derived classes.

<h4>Notes:</h4> <ol> <li>Although it is possible to define multiple OneTimeTearDown methods in the same class, you should rarely do so. Unlike methods defined in separate classes in the inheritance hierarchy, the order in which they are executed is not guaranteed. <li>OneTimeTearDown methods may be async if running under .NET 4.0 or higher. </ol>

<h4>See also…</h4> * [[SetUp Attribute]] * [[TearDown Attribute]] * [[OneTimeSetUp Attribute]] * [[TestFixture Attribute]] * [[SetUpFixture Attribute]]

OrConstraint combines two other constraints and succeeds if either of them succeeds.

<h4>Constructor</h4>

OrConstraint(Constraint left, Constraint right)

<h4>Syntax</h4>

<Constraint>.Or.<Constraint>

<h4>Examples of Use</h4>

Assert.That( 3, Is.LessThan( 5 ).Or.GreaterThan( 10 ) ); 

0.11.0.2 See also…

  • [[AndConstraint]]

The OrderAttribute may be placed on a test method to specify the order in which tests are run. Ordering is given by the required order argument to the attribute, an int.

0.11.0.3 Example

The following tests will be run in the order: * TestA * TestB * TestC

public class MyFixture
{
    [Test, Order(1)]
    public void TestA() { ... }


    [Test, Order(2)]
    public void TestB() { ... }

    [Test]
    public void TestC() { ... }
}

0.11.0.4 Notes:

  1. Tests with an OrderAttribute argument are started before any tests without the attribute.

  2. Ordered tests are started in ascending order of the order argument.

  3. Among tests with the same order value or without the attribute, execution order is indeterminate.

  4. Tests do not wait for prior tests to finish. If multiple threads are in use, a test may be started while some earlier tests are still being run.

<p>The <b>PairwiseAttribute</b> is used on a test to specify that NUnit should generate test cases in such a way that all possible pairs of values are used. This is a well-known approach for combatting the combinatorial explosion of test cases when more than two features (parameters) are involved.

<h4>Example</h4>

<p>Using the Combinatorial attribute, the following test would be executed 12 (3x2x2) times. With <b>Pairwise</b> it is executed only enough times so that each possible pair is covered..

[Test, Pairwise]
public void MyTest(
    [Values("a", "b", "c")] string a,
    [Values("+", "-")] string b,
    [Values("x", "y")] string c)
{
    Console.WriteLine("{0} {1} {2}", a, b, c);
}

<p>For this test, NUnit currently calls the method six times, producing the following output:

    a + y
    a - x
    b - y
    b + x
    c - x
    c + y

<p>Note that this is not the optimal output. The pairs (-, x) and (+, y) appear twice. NUnit uses a heuristic algorithm to reduce the number of test cases as much as it can. Improvements may be made in the future.

<h4>Limitations</h4>

<p>When used on a generic method the programmer must ensure that all possible combinations of arguments are valid. When multiple parameters use the same generic type (e.g.: T) this may not be possible and the attribute may generate invalid test cases.

<h4>See also…</h4> * [[Sequential Attribute]] * [[Combinatorial Attribute]]

NUnit 3.0 offers two forms of parallel execution, allowing tests to be run more quickly.

0.12 Parallel Execution of Multiple Assemblies

The NUnit test engine is able to offer a certain degree of parallelization by running the tests in each test assembly in a different Process. If tests are already split across multiple assemblies, this is the simplest way to improve performance through parallel execution.

In order to use this level of parallelism is simple to implement, that the tests in each assembly must not use any common external resources such as files or databases. This is the default behavior when running multiple assemblies together.

Normally, all the test processes run simultaneously. If you need to reduce the number of processes allowed to run at one time, you may specify a value for the --agents option on the nunit-console command-line. For example, if you are running tests in 10 different processes, a setting of --agents=3 will allow no more than three of them to execute simultaneously.

Note: This facility does not depend on the test framework used in any way. Test assemblies that use older versions of NUnit may be run in parallel processes just as easily as those using NUnit 3.0. If extensions are created to support additional frameworks, the NUnit engine will run those assemblies in parallel as well.

0.13 Parallel Execution within an Assembly

The NUnit 3.0 framework can run tests in parallel within an assembly. By default, no parallel excecution takes place. Tests that are eligible to be run in parallel with other tests must be identified using the [[Parallelizable Attribute]].

The framework creates worker threads for running tests. The default number of threads is equal to the number of processors or 2, whichever is greater. The user can control the number of threads by use of the assembly-level [[LevelOfParallelism Attribute]] or the --workers option on the command-line. The command-line option takes precedence, if specified.

Parallel execution in the framework requires some attention to how tests are written. Use of non-readonly statics or access to common internal or external resources can easily break your tests or hang the test run. Race conditions may result in problems that arise only intermittently and are very difficult to resolve. Once we implement parallel execution of test cases, shared use of instance members will be dangerous as well. Tests to be run in parallel should generally be stateless.

0.13.0.1 How It Works

We use multiple queues organized into “shifts”. A WorkShift consists of one or more queues of work items, which may be active at the same time. As the name suggests, no two shifts are active simultaneously. NUnit runs one WorkShift until all available work is complete and then switches to the next shift. When there is no work for any shift, the run is complete.

There are three shifts, listed here with their associated queues…

Shift Queues Workers Usage
Parallel Shift Parallel Queue LoP* Parallelizable tests run in the MTA
Parallel STA Queue 1 Parallelizable tests run in the STA
Non-Parallel Shift Non-Parallel Queue 1 Non-parallelizable tests run in the MTA
Non-Parallel STA Shift Non-Parallel STA Queue 1 Non-parallelizable tests run in the STA

* Depends on Level of Parallelism

NOTES:

  1. As can be seen in the table, if --workers=n is specified, n + 3 workers are actually created and up to n + 1 workers may be active at one time. This extra worker has little impact on throughput, since tasks in the STA tend to spend a good deal of time waiting.

  2. For efficiency, each queue is created when the first test is added to it. At the time of creation, all workers for that queue are also created and initialized.

  3. If the command line specifies zero workers, all use of the dispatcher and its queues is currently bypassed and tests are run sequentially on a single thread.

  4. In the current implementation, test methods or cases are never run in parallel with one another, even if the ParallelizableAttribute is specified on them.

0.13.0.2 Text Output from Tests

In the past, NUnit was able to capture text output (Console, Trace and Log) and associate it with the correct test. This was possible because only one test could run at a time, therefore any output received between the start and end of a particular test could be identified with that test.

In an environment where multiple tests may be running at the same time, this is no longer possible. To solve this problem, output is now collected in the result for each test and is not displayed until the test completes.

0.13.0.3 Platform Support

This feature is supported by .NET 2.0, 4.0, 4.5 and Compact Framework builds of the NUnit framework. It is not available in the Silverlight or Portable builds. Where not supported, the related attributes are still accepted but are ignored.

This attribute is used to indicate whether the test and/or its descendants may be run in parallel with other tests. By default, no parallel execution takes place.

The constructor takes an optional <b>ParallelScope</b> enumeration argument (see below), which defaults to <b>ParallelScope.Self</b>. The attribute may be used at the assembly, class or method level and the word “test” in the rest of this description refers to the suite or test case that corresponds to the item on which the attribute appears.

<p>The Scope may also be specified using the named property <b>Scope=</b>.

<h4>ParallelScope Enumeration</h4>

This is a <b>[Flags]</b> enumeration used to specify which tests may run in parallel. It applies to the test upon which it appears and any subordinate tests. It is defined as follows:

[Flags]
public enum ParallelScope
{
    None,     // the test may not be run in parallel with other tests
    Self,     // the test itself may be run in parallel with other tests
    Children, // child tests may be run in parallel with one another
    Fixtures  // fixtures may be run in parallel with one another
}

Note: Parallel execution of methods within a class is not yet implemented. Parallel execution only applies down to the TestFixture level. ParallelScope.Children works as ParallelScope.Fixtures and any ParallelizableAttribute placed on a method is ignored.

Values that apply to a higher level test than the test on which the scope appears - for example, ParallelScope.Fixtures appearing on a method - are ignored without warning or affect.

0.13.0.4 Specifying Parallelizable at Multiple Test Levels

The <b>ParallelizableAttribute</b> may be specified on multiple levels of the tests. Settings at a higher level may affect lower level tests, unless those lower-level tests override the inherited settings. Thus, if the assembly has <b>ParallelScope.None</b> either by use of the attribute or by default, classes with <b>ParallelScope.Self</b> may be run in parallel as may their children if an appropriate scope is used.

Note that a lower-level test cannot change the settings on higher-level tests. Thus, allowing parallel execution for methods of a class that does not allow it, results in those methods running in parallel with one another, but not in parallel with test methods under any other classes. This is the natural outcome of the fact that the execution of a test method is, in fact, part of the execution of the test represented by the class.

0.13.0.5 See also…

  • [[LevelOfParallelism Attribute]]

NUnit supports parameterized tests. Test methods may have parameters and various attributes are available to indicate what arguments should be supplied by NUnit.

Multiple sets of arguments cause the creation of multiple tests. All arguments are created at the point of loading the tests, so the individual test cases are available for display and selection in the Gui, if desired.

Some attributes allow you to specify arguments inline - directly on the attribute - while others use a separate method, property or field to hold the arguments. In addition, some attributes identify complete test cases, including all the necessary arguments, while others only provide data for a single argument. This gives rise to four groups of attributes, as shown in the following table.

Complete Test Cases Data for One Argument
Inline [[TestCase Attribute]] [[Random Attribute]]<br/>[[Range Attribute]]<br/>[[Values Attribute]]
Separate [[TestCaseSource Attribute]] [[ValueSource Attribute]]

In addition, when data is specified for individual arguments, special attributes may be added to the test method itself in order to tell NUnit how to go about combining the arguments. Currently, the following attributes are provided: * [[Combinatorial Attribute]] (default) * [[Pairwise Attribute]] * [[Sequential Attribute]]

0.13.1 Order of Execution

The individual test cases are executed in the order in which NUnit discovers them. This order does <b>not</b> necessarily follow the lexical order of the attributes and will often vary between different compilers or different versions of the CLR.

The following specific rules for ordering apply: * If all arguments are specified in a <b>single TestCaseSource</b> attribute, the ordering of the cases provided will be maintained. * If each parameter has a single <b>Values</b>, <b>ValueSource</b> or <b>Range</b> attribute and the <b>Sequential</b> combining strategy is used - or there is only one argument - the ordering will be maintained. * In all other cases, including using multiple <b>TestCase</b> attributes or a combination of different types of attributes, the ordering of the test cases is undefined.

The Platform attribute is used to specify platforms for which a test or fixture should be run. Platforms are specified using case-insensitive string values and may be either included or excluded from the run by use of the Include or Exclude properties respectively. Platforms to be included may alternatively be specified as an argument to the PlatformAttribute constructor. In either case, multiple comma-separated values may be specified.

If a test or fixture with the Platform attribute does not satisfy the specified platform requirements it is skipped. The test does not affect the outcome of the run at all: it is not considered as ignored and is not even counted in the total number of tests. [Ed.: Check this.] In the gui, the tree node for the test remains gray and the status bar color is not affected.

0.13.1.1 Test Fixture Syntax

namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture]
  [Platform("NET-2.0")]
  public class DotNetTwoTests
  {
    // ...
  }
}

0.13.1.2 Test Syntax

namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture]
  public class SuccessTests
  {
    [Test]
    [Platform(Exclude="Win98,WinME")]
    public void SomeTest()
    { /* ... */ }
}

0.13.1.3 Platform Specifiers

The following values are recognized as platform specifiers. They may be expressed in upper, lower or mixed case.

0.13.1.3.0.1 Operating System

<ul> <li>Win</li> <li>Win32</li> <li>Win32S</li> <li>Win32Windows</li> <li>Win32NT</li> <li>WinCE</li> <li>Win95</li> <li>Win98</li> <li>WinMe</li> <li>NT3</li> <li>NT4</li> <li>NT5</li> <li>NT6</li> <li>Win2K</li> <li>WinXP</li> <li>Win2003Server</li> <li>Vista</li> <li>Win2008Server</li> <li>Win2008ServerR2</li> <li>Windows7</li> <li>Win2012Server</li> <li>Windows8</li> <li>Unix</li> <li>Linux</li> <li>MacOsX</li> <li>XBox</li> </ul>

0.13.1.3.0.2 Architecture
  • 32-Bit
  • 32-Bit-Process
  • 32-Bit-OS (.NET 4.0 and higher only)
  • 64-Bit
  • 64-Bit-Process
  • 64-Bit-OS (.NET 4.0 and higher only)
0.13.1.3.0.3 Runtime

<ul> <li>Net</li> <li>Net-1.0</li> <li>Net-1.1</li> <li>Net-2.0</li> <li>Net-3.0 (1)</li> <li>Net-3.5 (2)</li> <li>Net-4.0</li> <li>Net-4.5 (3)</li> <li>NetCF</li> <li>SSCLI</li> <li>Rotor</li> <li>Mono</li> <li>Mono-1.0</li> <li>Mono-2.0</li> <li>Mono-3.0 (4)</li> <li>Mono-3.5 (5)</li> <li>Mono-4.0</li> </ul>

0.13.1.4 Notes:

  1. Includes Net-2.0
  2. Includes Net-2.0 and Net-3.0
  3. Includes Net-4.0
  4. Includes Mono-2.0
  5. Includes Mono-2.0 and Mono-3.0

If your tests are compiled x86 or x64, NUnit will run each assembly by default in a process targeted at the appropriate platform as well as the targeted runtime. If you run multiple assemblies in the same process, you may not mix x86 and x64 assemblies.

On an x64 machine, if your test assembly is compiled x86, you <b>must not</b> use <b>/process=Single</b> on the command line. This is because NUnit is already running in a 64-bit process and will fail when it tries to load your assembly into the same process.

If your test assembly references any x86 or x64 assemblies, it should be built for the same platform. Consequently, you may not mix x86 and x64 references.

Project Loaders are extensions that know how to load a project in a particular format and create a package suitable for running tests under NUnit. NUnit itself provides two of them: * NUnitProjectLoader * VisualStudioProjectLoader

The extension point for project loaders accepts extensions that implement the NUnit.Engine.Extensibility.IProjectLoader interface. The definition of a project loader extension might look something like the following…

[Extension]
[ExtensionProperty("FileExtension", ".xxx")]
[ExtensionProperty("FileExtension", ".yyy")]
public class SomeProjectLoader : IProjectLoader
{
    ...
}

The engine will only load the extension if it encounters a potential project file using the indicated file extensions.

The IProjectLoader interface is defined as follows:

/// <summary>
/// The IProjectLoader interface is implemented by any class
/// that knows how to load projects in a specific format.
/// </summary>
[TypeExtensionPoint(
    Description = "Recognizes and loads assemblies from various types of project formats.")]
public interface IProjectLoader
{
    /// <summary>
    /// Returns true if the file indicated is one that this
    /// loader knows how to load.
    /// </summary>
    /// <param name="path">The path of the project file</param>
    /// <returns>True if the loader knows how to load this file, otherwise false</returns>
    bool CanLoadFrom( string path );

    /// <summary>
    /// Loads a project of a known format.
    /// </summary>
    /// <param name="path">The path of the project file</param>
    /// <returns>An IProject interface to the loaded project or null if the project cannot be loaded</returns>
    IProject LoadFrom(string path);
}

An IProject, which is returned by LoadFrom is defined as:

/// <summary>
/// Interface for the various project types that the engine can load.
/// </summary>
public interface IProject
{
    /// <summary>
    /// Gets the path to the file storing this project, if any.
    /// If the project has not been saved, this is null.
    /// </summary>
    string ProjectPath { get; }

    /// <summary>
    /// Gets the active configuration, as defined
    /// by the particular project.
    /// </summary>
    string ActiveConfigName { get; }

    /// <summary>
    /// Gets a list of the configs for this project
    /// </summary>
    IList<string> ConfigNames { get; }

    /// <summary>
    /// Gets a test package for the primary or active
    /// configuration within the project. The package 
    /// includes all the assemblies and any settings
    /// specified in the project format.
    /// </summary>
    /// <returns>A TestPackage</returns>
    TestPackage GetTestPackage();

    /// <summary>
    /// Gets a TestPackage for a specific configuration
    /// within the project. The package includes all the
    /// assemblies and any settings specified in the 
    /// project format.
    /// </summary>
    /// <param name="configName">The name of the config to use</param>
    /// <returns>A TestPackage for the named configuration.</returns>
    TestPackage GetTestPackage(string configName);
}

TestPackage is defined in the nunit.engine.api assembly and includes a list of assemblies together with a dictionary of settings used by the package.

PropertyAttribute provides a generalized approach to setting named properties on any test case or fixture, using a name/value pair. In the example below, the fixture class MathTests is given a Location value of 723 while the test case AdditionTest is given a Severity of “Critical”

0.13.1.5 Example:

namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture, Property("Location",723)]
  public class MathTests
  {
    [Test, Property("Severity", "Critical")]
    public void AdditionTest()
    { /* ... */ }
  }
}

0.13.1.6 Usage Note

The PropertyAttribute is not currently used for any purpose by NUnit itself, other than to display them in the XML output file and in the Test Properties dialog of the gui. You may also use use properties with the --where option on the command-line in order to select tests to run. See [[Test Selection Language]].

User tests may access properties through the [[TestContext]] or by reflection.

0.13.1.7 Custom Property Attributes

Users can define custom attributes that derive from PropertyAttribute and have them recognized by NUnit. PropertyAttribute provides a protected constructor that takes the value of the property and sets the property name to the name of the derived class with the ‘Attribute’ suffix removed.

Here’s an example that creates a Severity property. It works just like any other property, but has a simpler syntax and is type-safe. A custom test reporting system might make use of the property to provide special reports.

public enum SeverityLevel
{
    Critical,
    Major,
    Normal,
    Minor
}

[AttributeUsage(AttributeTargets.Method, AllowMultiple=false)]
public class SeverityAttribute : PropertyAttribute
{
    public SeverityAttribute( SeverityLevel level )
        : base( level ); 
}

...

[Test, Severity( SeverityLevel.Critical)]
public void MyTest()
{ /*...*/ }

A PropertyAttribute may contain multiple name/value pairs. This capability is not exposed publicly but may be used by derived property classes.

PropertyConstraint tests for the existence of a named property on an object and then applies a constraint test to the property value.

<h4>Constructor</h4>

PropertyConstraint(string name, IConstraint baseConstraint)

<h4>Syntax</h4>

Has.Property(string name)... // followed by further constraint syntax

<h4>Examples of Use</h4>

Assert.That(someObject, Has.Property("Version").EqualTo("2.0"));
Assert.That(collection, Has.Property("Count").GreaterThan(10));
Assert.That(collection, Has.Count.GreaterThan(10);

Note: As shown in the example, certain common properties are known to NUnit and may be tested using a shorter form. The following properties are supported:

Has.Length...
Has.Count...
Has.Message...
Has.InnerException...

0.13.1.8 See also…

  • [[PropertyExistsConstraint]] The PropertyExistsConstraint tests for the existence of a named property on an object.

<h4>Constructor</h4>

PropertyExistsConstraint(string name)

<h4>Syntax</h4>

Has.Property( string )

<h4>Examples of Use</h4>

Assert.That( someObject, Has.Property( "Version" ) );

0.13.1.9 See also…

  • [[PropertyConstraint]]

<style><!– div.code { width: 34em } –></style>

<h2>NUnit Quick Start</h2>

<pre class=“prettyprint”>using System; using NUnit.Framework;

namespace Bank { public class Account { private decimal balance; private decimal minimumBalance = 10m;

public void Deposit(decimal amount)
{
  balance += amount;
}

public void Withdraw(decimal amount)
{
  balance -= amount;
}

public void TransferFunds(Account destination, decimal amount)
{
  if (balance - amount < minimumBalance)
    throw new InsufficientFundsException();

  destination.Deposit(amount);

  Withdraw(amount);
}

public decimal Balance
{
  get { return balance; }
}

public decimal MinimumBalance
{
  get { return minimumBalance; }
}

}

public class InsufficientFundsException : ApplicationException { }

[TestFixture] public class AccountTest { Account source; Account destination;

[SetUp]
public void Init()
{
  source = new Account();
  source.Deposit(200m);

  destination = new Account();
  destination.Deposit(150m);
}

[Test]
public void TransferFunds()
{
  source.TransferFunds(destination, 100m);

  Assert.AreEqual(250m, destination.Balance);
  Assert.AreEqual(100m, source.Balance);
}

[Test]
[ExpectedException(typeof(InsufficientFundsException))]
public void TransferWithInsufficientFunds()
{
  source.TransferFunds(destination, 300m);
}

[Test]
[Ignore("Decide how to implement transaction management")]
public void TransferWithInsufficientFundsAtomicity()
{
  try
  {
    source.TransferFunds(destination, 300m);
  }
  catch (InsufficientFundsException expected)
  {
  }

  Assert.AreEqual(200m, source.Balance);
  Assert.AreEqual(150m, destination.Balance);
}

} }</pre>

<b>Listing - QuickStart Example</b>

<p>The <b>RandomAttribute</b> is used to specify a set of random values to be provided for an individual parameter of a parameterized test method. Since NUnit combines the data provided for each parameter into a set of test cases, data must be provided for all parameters if it is provided for any of them.

<p>By default, NUnit creates test cases from all possible combinations of the datapoints provided on parameters - the combinatorial approach. This default may be modified by use of specific attributes on the test method itself.

<p>RandomAttribute supports the following constructors:

public Random( int count );
public Random( int min, int max, int count );
public Random( uint min, uint max, int count );
public Random( long min, long max, int count );
public Random( ulong min, ulong max, int count );
public Random( short min, short max, int count );
public Random( ushort min, ushort max, int count );
public Random( byte min, byte max, int count );
public Random( sbyte min, sbyte max, int count );
public Random( double min, double max, int count );
public Random( float min, float max, int count );

In the first form, without minimum and maximum values, the attribute automatically generates values of the appropriate type for the argument, provided that using the Randomizer object associated with the current context. See [[Randomizer Methods]] for details.

In general, the forms that specify a minimum and maximum should be used on arguments of the same type. However, the following exceptions are supported:

  • You may use an int range on arguments of type short, ushort, byte, sbyte and decimal.

  • You may use a double range on arguments of type decimal.

Note that there is no constructor taking decimal values for min and max. This is because .NET does not support use of decimal in an attribute constructor.

<h4>Example</h4>

<p>The following test will be executed fifteen times, three times for each value of x, each combined with 5 random doubles from -1.0 to +1.0.

[Test]
public void MyTest(
    [Values(1,2,3)] int x,
    [Random(-1.0, 1.0, 5)] double d)
{
    ...
}

<h4>See also…</h4> * [[Values Attribute]] * [[Range Attribute]] * [[Sequential Attribute]] * [[Combinatorial Attribute]] * [[Pairwise Attribute]]

The Randomizer object exposed by TestContext.CurrentContext.Random extends System.Random to provide random data for a wide range of numeric types as well as enums and strings. Each test context has access to it’s own Randomizer which is used to generate random values for the RandomAttribute as well as for use by the user callng its methods.

The benefit of using Randomizer rather than System.Random directly is twofold: 1. A wide range of types are provided in a uniform manner. 2. Randomizer produces repeatable values for the test run so long as no tests are changed and the same seed is used to initialize the run. A future extension is planned, which would provide repeatability at the individual method level, even if other methods are changed.

0.13.2 Int (System.Int32)

0.13.2.0.1 Next()

Returns the next random non-negative int. (Inherited from System.Random)

0.13.2.0.2 Next(int max)

Returns the next random non-negative int less than max, which must be positive. (Inherited from System.Random)

0.13.2.0.3 Next(int min, int max)

Returns the next random int in the range min to max, excluding max. (Inherited from System.Random)

0.13.3 UInt (System.UInt32)

0.13.3.0.1 NextUInt()

Returns the next random uint.

0.13.3.0.2 NextUInt(uint max)

Returns the next random uint less than max.

0.13.3.0.3 NextUInt(uint min, uint max)

Returns the next random uint in the range min to max, excluding max.

0.13.4 Long (System.Int64)

0.13.4.0.1 NextLong()

Returns the next random non-negative long.

0.13.4.0.2 NextLong(long max)

Returns the next random non-negative long less than max, which must be positive.

0.13.4.0.3 NextLong(long min, long max)

Returns the next random long in the range min to max, excluding max.

0.13.5 ULong (System.UInt64)

0.13.5.0.1 NextULong()

Returns the next random ulong.

0.13.5.0.2 NextULong(ulong max)

Returns the next random ulong less than max.

0.13.5.0.3 NextULong(ulong min, ulong max)

Returns the next random ulong in the range min to max, excluding max.

0.13.6 Short (System.Int16)

0.13.6.0.1 NextShort()

Returns the next random non-negative short.

0.13.6.0.2 NextShort(short max)

Returns the next random non-negative short less than max, which must be positive.

0.13.6.0.3 NextShort(short min, short max)

Returns the next random short in the range min to max, excluding max.

0.13.7 UShort (System.UInt16)

0.13.7.0.1 NextUShort()

Returns the next random ushort.

0.13.7.0.2 NextUShort(ushort max)

Returns the next random ushort less than max.

0.13.7.0.3 NextUShort(ushort min, ushort max)

Returns the next random ushort in the range min to max, excluding max.

0.13.8 SByte (System.SByte)

0.13.8.0.1 NextSByte()

Returns the next random non-negative sbyte.

0.13.8.0.2 NextSByte(sbyte max)

Returns the next random non-negative sbyte less than max, which must be positive.

0.13.8.0.3 NextSByte(sbyte min, sbyte max)

Returns the next random sbyte in the range min to max, excluding max.

0.13.9 Byte (System.Byte)

0.13.9.0.1 NextByte()

Returns the next random byte.

0.13.9.0.2 NextByte(byte max)

Returns the next random byte less than max.

0.13.9.0.3 NextByte(byte min, byte max)

Returns the next random byte in the range min to max, excluding max.

0.13.10 Double (System.Double)

0.13.10.0.1 NextDouble()

Returns the next double in the range 0.0 to 1.0, exclusive. (Inherited from System.Random.)

0.13.10.0.2 NextDouble(double max)

Returns the next non-negative double less than max.

0.13.10.0.3 NextDouble(double min, double max)

Returns the next double in the range min to max, excluding max.

0.13.11 Float (System.Float)

0.13.11.0.1 NextFloat()

Returns the next float in the range 0.0f to 1.0f, exclusive.

0.13.11.0.2 NextFloat(float max)

Returns the next non-negative float less than max.

0.13.11.0.3 NextFloat(float min, float max)

Returns the next float in the range min to max, excluding max.

0.13.12 Decimal (System.Decimal)

0.13.12.0.1 NextDecimal()

Returns the next non-negative random decimal.

0.13.12.0.2 NextDecimal(decimal max)

Returns the next non-negative decimal less than max.

0.13.12.0.3 NextDecimal(decimal min, decimal max)

Returns the next decimal in the range min to max, excluding max.

0.13.12.0.3.1 Notes on Decimal Implementation
  1. In the current implementation, the scale is always set to zero. That is, the values are all integral, with no decimal places. This may be enhanced in a future release by allowing the user to specify the scale.
  2. In the third form, an exception is currently thrown if the range is greater than decimal.MaxValue. This can only occur if min is negative and max is positive.

0.13.13 Bool (System.Boolean)

0.13.13.0.1 NextBool()

Returns a random bool with equal probability of true or false.

0.13.13.0.2 NextBool(double probability)

Returns a random bool with the specified probability of being true. Probability argument must be in the range 0.0 to 1.0, inclusive.

0.13.14 Enum (System.Enum)

0.13.14.0.1 NextEnum<T>()

Returns a random enum value of type T. All the values of the enum are returned with equal probability. Note that this may not be useful in all cases for enums with the FlagsAttribute specified.

0.13.14.0.2 NextEnum(Type type)

Returns a random enum value of the type specified as an object, which the caller will normally cast to the specified type.

0.13.15 String (System.String)

0.13.15.0.1 GetString()

Returns a random string of default length, composed using a default set of characters. In the current implementation, the default length is hard-coded as 25 and the default characters are “abcdefghijkmnopqrstuvwxyzABCDEFGHJKLMNOPQRSTUVWXYZ0123456789_”.

0.13.15.0.2 GetString(int length)

Returns a random string of the specified length.

0.13.15.0.3 GetString(int outputLength, string allowedChars)

Returns a random string of the specified length using the characters in the string given as the second argument.

The <b>RangeAttribute</b> is used to specify a range of values to be provided for an individual parameter of a parameterized test method. Since NUnit combines the data provided for each parameter into a set of test cases, data must be provided for all parameters if it is provided for any of them.

By default, NUnit creates test cases from all possible combinations of the datapoints provided on parameters - the combinatorial approach. This default may be modified by use of specific attributes on the test method itself.

RangeAttribute supports the following constructors:

public RangeAttribute( int from, int to );
public RangeAttribute( int from, int to, int step );
public RangeAttribute( long from, long to, long step );
public RangeAttribute( float from, float to, float step );
public RangeAttribute( double from, double to, double step );

0.13.15.1 Example

The following test will be executed nine times.

[Test]
public void MyTest(
    [Values(1,2,3) int x,
    [Range(0.2,0.6,0.2] double d)
{
    ...
}

The MyTest method is called nine times, as follows:

    MyTest(1, 0.2)
    MyTest(1, 0.4)
    MyTest(1, 0.6)
    MyTest(2, 0.2)
    MyTest(2, 0.4)
    MyTest(2, 0.6)
    MyTest(3, 0.2)
    MyTest(3, 0.4)
    MyTest(3, 0.6)

0.13.15.2 See also…

  • [[Values Attribute]]
  • [[Random Attribute]]
  • [[Sequential Attribute]]
  • [[Combinatorial Attribute]]
  • [[Pairwise Attribute]]

RangeConstraint tests that a value is in an (inclusive) range.

<h4>Constructor</h4>

RangeConstraint(IComparable from, IComparable to)

<h4>Syntax</h4>

Is.InRange(IComparable from, IComparable to)

<h4>Modifiers</h4>

...Using(IComparer comparer)
...Using<T>(IComparer<T> comparer)
...Using<T>(Comparison<T> comparer)

<h4>Examples of Use</h4>

int[] iarray = new int[] { 1, 2, 3 }

Assert.That( 42, Is.InRange(1, 100) );
Assert.That( iarray, Is.All.InRange(1, 3) );
Assert.That(myOwnObject, 
    Is.InRange(lowExpected, highExpected).Using(myComparer));

RegexConstraint tests that a pattern is matched.

<h4>Constructor</h4>

RegexConstraint(string pattern)

<h4>Syntax</h4>

Does.Match(string pattern)
Matches(string pattern)

<h4>Modifiers</h4>

...IgnoreCase

<h4>Examples of Use</h4>

string phrase = "Make your tests fail before passing!"

Assert.That( phrase, Does.Match( "Make.*tests.*pass" ) );
Assert.That( phrase, Does.Not.Match( "your.*passing.*tests" ) );
Expect( phrase, Matches( "Make.*pass" ) );

<h4>Notes</h4> 1. <b>Matches</b> may appear only in the body of a constraint expression or when the inherited syntax is used.

0.13.16 NUnit 3.5 - October 3, 2015

This is the first version of NUnit where the framework will be released separately from the console runner, engine and other extensions. From this point forward, the NUnit Framework will be released on its own schedule that is not bound to that of any other NUnit project and version numbers may diverge over time.

This is also the first release where the NUnit Framework will not be included in the installer. Only the console runner, engine and extensions will be available as an MSI installer. We recommend that you use the NUnit NuGet packages for the framework, but a ZIP file with the binaries will also be available.

0.13.16.1 Framework

  • Added Assert.Zero and Assert.NotZero methods
  • You can now pass a Func<string> to Asserts to lazily evaluate exception messages
  • Added the ability to Assert on the order of multiple properties in a collection
  • Tests with a Timeout will no longer timeout while you are debugging

0.13.16.2 Issues Resolved

  • 144 Pass a Func<string> to lazily evaluate an exception message
  • 995 Enable Warning as Error
  • 1106 Move various Assembly Info files under Properties for CF
  • 1334 Add Assert.Zero and Assert.NotZero
  • 1479 Don’t enforce [Timeout] when debugger is attached
  • 1540 Remove old .NET Core Projects
  • 1553 Allow ordering tests to be done in multiple properties
  • 1575 Escaping control chars in custom message
  • 1596 Eliminate code sharing across projects to be split
  • 1598 Split framework and console/engine into separate projects
  • 1610 Refactor dependencies in build.cake
  • 1615 Appveyor error in TestCF
  • 1621 Remove console and command-line option files from common
  • 1640 When submitting only part of optional parameters, all are overriden by defaults
  • 1641 Create OSX CI Build on Travis
  • 1663 Find way to hide NUnit.Compatability.Path from intellisense
  • 1681 NUnitLite under .net core doesn’t support TeamCity output
  • 1683 Existence of SerializableAttribute in .NET Core
  • 1693 2 unit tests fail due to localization
  • 1716 Move installer to new repository
  • 1717 Change suffix for master builds
  • 1723 Remove Cake target TestAll
  • 1739 Create separate copies of MockAssembly for framework, engine and extensions
  • 1751 Serializable attribute exists in both System.Runtime.Serialization.Formatters and nunit.framework
  • 1775 Support NUnit assertions in partial trust code.
  • 1800 Remove Console/Engine projects from nunit.linux.sln
  • 1805 Error message “arguments provided for method not taking any” seems incomplete / doesn’t make much sense
  • 1815 Prevent NullReferenceException in SubPathConstraint

0.13.17 NUnit 3.4.1 - June 30, 2016

0.13.17.1 Console Runner

  • A new option, –list-extensions, will display all the engine extensions that have been installed by the engine.

0.13.17.2 Issues Resolved

  • 1623 NUnit 3.4 is not integrated with TeamCity
  • 1626 NUnit.ConsoleRunner is not picking up NUnit.Extension.NUnitV2ResultWriter
  • 1628 Agent’s process stays in memory when it was failed to unload AppDomain
  • 1635 Console option to list loaded extensions

0.13.18 NUnit 3.4 - June 25, 2016

0.13.18.1 Framework

  • Improvements in comparing equality using IEquatable<T>
  • Test case names will only be truncated if the runner requests it or it is overridden on the command line with the –test-name-format option
  • The .NET 2.0 version of the framework now includes LINQ. If your tests target .NET 2.0, you can now use LINQ queries in your tests

0.13.18.2 Engine

  • The TeamCity event listener has been separated out into an engine extension
  • Fixed numerous issues around thread safety of parallel test runs
  • Additional fixes to reduce memory usage
  • Fixes for Mono 4.4

0.13.18.3 Console Runner

  • There is a new –params command line option that allows you to pass parameters to your tests which can be retrieved using TestContext.Parameters
  • Another new command line option –loaduserprofile causes the User Profile to be loaded into the NUnit Agent process.

0.13.18.4 Issues Resolved

  • 329 (CLI) Runner does not report AppDomain unloading timeout
  • 720 Need a way to get test-specific command-line arguments at runtime
  • 1010 Need to control engine use of extensions
  • 1139 Nunit3 console doesn’t show test output continously
  • 1225 The –teamcity option should really be an extension
  • 1241 Make TestDirectory accessible when TestCaseSource attributes are evaluated
  • 1366 Classname for inherited test is not correct
  • 1371 Support dotnet test in .NET CLI and .NET Core
  • 1379 Console returns 0 for invalid fixtures
  • 1422 Include TestListWithEmptyLine.tst in ZIP Package
  • 1423 SingleThreaded attribute should raise an error if a thread is required
  • 1425 Lazy initialization of OutWriter in TestResult is not thread safe
  • 1427 Engine extensions load old packages
  • 1430 TestObjects are retained for lifetime of test run, causing high memory usage
  • 1432 NUnit hangs when reporting to TeamCity
  • 1434 TestResult class needs to be thread-safe
  • 1435 Parallel queue creation needs to be thread-safe
  • 1436 CurrentFramework and Current Platform need to be more thread-safe
  • 1439 EqualConstraint does Not use Equals Override on the Expected Object
  • 1441 Add Linq for use internally in .NET 2.0 code
  • 1446 TestOrderAttributeTests is not public
  • 1450 Silverlight detection doesn’t work when building on 32-bit OS
  • 1457 Set the 2.0 build to ignore missing xml dcoumentation
  • 1463 Should TestResult.AssertCount have a public setter?
  • 1464 TNode.EscapeInvalidXmlCharacters recreates Regex continually
  • 1470 Make EventQueue and associated classes lock-less and thread safe
  • 1476 Examine need for “synchronous” events in event queue
  • 1481 TestCase with generic return type causes NullReferenceException
  • 1483 Remoting exceptions during test execution
  • 1484 Comparing Equality using IEquatable<T> Should Use Most Specific Method
  • 1493 NUnit 2 test results report ParameterizedMethod but should be ParameterizedTest
  • 1507 NullReferenceException when null arguments are used in TestFixtureAttribute
  • 1513 Add new teamcity extension to packages
  • 1518 NUnit does not send the “testStarted” TeamCity service message when exception was thrown from SetUp/OneTimeSetUp
  • 1520 Detect Portable, Silverlight and Compact and give error message
  • 1528 Use of Sleep(0) in NUnit
  • 1543 Blank name attribute in nunit2-formatted XML result file test-run element
  • 1547 Create separate assembly for System.Linq compatibility classes
  • 1548 Invalid Exception when engine is in a 32-bit process
  • 1549 Changing default behavior for generating test case names
  • 1551 Path in default .addins file for ConsoleRunner package may not exist
  • 1555 EndsWith calls in Constraint constructor can cause major perf issues
  • 1560 Engine writes setting file unnecessarily
  • 1573 Move Nunit.Portable.Agent to new Repo
  • 1579 NUnit v3 dangerously overrides COMPLUS_Version environment variable
  • 1582 Mono 4.4.0 Causes Test Failures
  • 1593 Nunit Console Runner 3.2.1 and Mono 4.4 throws RemotingException
  • 1597 Move Portable agent to its own repository
  • 1605 TeamCity package has no pre-release suffix
  • 1607 nunit.nuget.addins discovery pattern is wrong then restored through project.json
  • 1617 Load user profile on test runners

0.13.19 NUnit 3.2.1 - April 19, 2016

0.13.19.1 Framework

  • The output and error files are now thread safe when running tests in parallel
  • Added a .NET 3.5 build of the framework preventing conflicts with the compatiblity classes in the 2.0 framework
  • Added a SingleThreadedAttribute to be added to a TestFixture to indicate all child tests should run on the same thread

0.13.19.2 Engine

  • Unless required, run all tests within a fixture on the same thread
  • Added an EventListener extension point
  • Reduced memory usage

0.13.19.3 Console Runner

  • No longer probes for newer versions of the engine, instead uses the engine that is included with the console

0.13.19.4 Issues Resolved

  • 1413 Switch console to use a local engine
  • 1410 Make OutFile and ErrFile streamwriters synchronized
  • 1406 TextCapture is not threadsafe but is used to intercept calls that are expected to be threadsafe
  • 1405 ITestRunner.StopRun throws exception of type ‘System.MissingMethodException’
  • 1399 Fixed NullReference issue introduced by the fix for #681
  • 1390 –testlist with file with blank first line causes IndexOutOfRangeException
  • 1386 Bug when using Assert.Equals() with types that explicitly implement IEquatable<T>
  • 1383 Skip Silverlight build if SDK not installed
  • 1359 Compilation error in NUnitPortableDriverTests.cs
  • 1352 Tests losing data setup on thread
  • 1349 Collision on System.Func from nunit.framework with System.Core in .Net 3.5 (CS0433)
  • 1338 BUILDING.txt is outdated
  • 1333 Single Thread per Worker
  • 1329 Unable to build without Compact Framework
  • 1327 TestCaseSource in NUnit 3 converts an argument declared as String[] to String
  • 1318 NUnit console runner fails with error code -100
  • 1317 Getting CF framework unit tests running on CI build
  • 1302 Create EventListener ExtensionPoint for the Engine
  • 1266 SetCultureAttribute does not work if set on assembly level
  • 1247 Potential memory issue
  • 1231 Build may silently fail some tests
  • 1208 Apartment on assembly level broken
  • 1161 NUnit3-Console should disallow the combination of –inprocess and –x86, giving an error message
  • 1141 Explicit Tests get run when using –where with some filters
  • 773 Upgrade Travis CI from Legacy Infrastructure
  • 640 Keep CF Build (and other future builds) in Sync
  • 332 Add CF to the Appveyor CI build

0.13.20 NUnit 3.2 - March 5, 2016

0.13.20.1 Framework

  • Added an Order attribute that defines the order in which tests are run
  • Added Assert.ThrowsAsync for testing if async methods throw an exception
  • You can now compare unlike collections using Is.EquivalentTo().Using(…)
  • Added the ability to add custom message formatters to MsgUtils
  • TestCaseSourceAttribute now optionally takes an array of parameters that can be passed to the source method
  • Added Is.Zero and Is.Not.Zero to the fluent syntax as a shorter option for Is.EqualTo(0) and Is.Not.EqualTo(0)

0.13.20.2 Engine

  • Engine extensions can be installed via NuGet packages

0.13.20.3 Issues Resolved

  • 170 Test Order Attribute
  • 300 Create an NUnit Visual Studio Template
  • 464 Async delegate assertions
  • 532 Batch runner for Silverlight tests
  • 533 Separate NUnitLite runner and autorunner
  • 681 NUnit agent cannot resolve test dependency assemblies when mixed mode initialization runs in the default AppDomain
  • 793 Replace CoreEngine by use of Extensions
  • 907 Console report tests are too fragile
  • 922 Wrap Console in NUnitLite
  • 930 Switch from MSBuild based build system to Cake
  • 981 Define NUnit Versioning for post-3.0 Development
  • 1004 Poor formatting of results for Assert.AreEqual(DateTimeOffset, DateTimeOffset)
  • 1018 ArgumentException when 2.x version of NUnit Framework is in the bin directory
  • 1022 Support Comparing Unlike Collections using Is.EquivalentTo().Using(…)
  • 1044 Re-order Test Summary Errors/Failures
  • 1066 ApartmentAttribute and TestCaseAttribute(s) do not work together
  • 1103 Can’t use TestCaseData from base class
  • 1109 NullReferenceException when using inherited property for ValueSource
  • 1113 Console runner and xml output consistency
  • 1117 Fix misbehaviour of Throws.Exception with non-void returning functions
  • 1120 NUnitProject should parse .nunit project files containing Xml Declarations
  • 1121 Usage of field set to null as value source leads to somewhat cryptic error
  • 1122 Region may be disposed before test delegate is executed
  • 1133 Provide a way to install extensions as nuget packages
  • 1136 Don’t allow V2 framework to update in V2 driver tests
  • 1171 A bug when using Assert.That() with Is.Not.Empty
  • 1185 Engine finds .NET 4.0 Client Profile twice
  • 1187 ITestAssemblyRunner.StopRun as implemented by NUnitTestAssemblyRunner
  • 1195 name attribute in test-suite and test-results element of output xml is different to nunit 2.6.4 using nunit2-format
  • 1196 Custom value formatter for v3 via MsgUtils
  • 1210 Available runtimes issues
  • 1230 Add ability for testcasedatasource to have parameters passed to methods
  • 1233 Add TestAssemblyRunner tests to both portable and silverlight builds
  • 1234 Have default NUnitLite Runner Program.cs return exit code
  • 1236 Make Appveyor NuGet feed more useable
  • 1246 Introduce Is.Zero syntax to test for zero
  • 1252 Exception thrown when any assembly is not found
  • 1261 TypeHelper.GetDisplayName generates the wrong name for generic types with nested classes
  • 1278 Fix optional parameters in TestCaseAttribute
  • 1282 TestCase using Params Behaves Oddly
  • 1283 Engine should expose available frameworks.
  • 1286 value of the time attribute in nunit2 outputs depends on the machine culture
  • 1297 NUnit.Engine nuget package improvements
  • 1301 Assert.AreNotSame evaluates ToString unnecessarily

0.13.21 NUnit 3.0.1 (3.0.5813) - December 1, 2015

0.13.21.1 Console Runner

  • The Nunit.Runners NuGet package was updated to become a meta-package that pulls in the NUnit.Console package
  • Reinstated the –pause command line option that will display a message box allowing you to attach a debugger if the –debug option does not work

0.13.21.2 Issues Resolved

  • 994 Add max number of Agents to the NUnit project file
  • 1014 Ensure NUnit API assembly updates with MSI installs
  • 1024 Added –pause flag to console runner
  • 1030 Update Nunit.Runners package to 3.0
  • 1033 “No arguments were provided” with Theory and Values combination
  • 1035 Check null arguments
  • 1037 Async tests not working on Windows 10 Universal
  • 1041 NUnit2XmlResult Writer is reporting Sucess when test fails
  • 1042 NUnit2 reports on 3.0 is different than 2.6.4
  • 1046 FloatingPointNumerics.AreAlmostEqualUlps throws OverflowException
  • 1049 Cannot select Generic tests from command line
  • 1050 Do not expose System.Runtime.CompilerServices.ExtensionAttribute to public
  • 1054 Create nuget feeds for CI builds on Appveyor
  • 1055 nunit3 console runner –where option does not return error on invalid selection string
  • 1060 Remove “Version 3” from NUnit Nuget Package
  • 1061 Nunit30Settings.xml becomes corrupted
  • 1062 Console.WriteLine statements in “OneTimeSetUp” and “OneTimeTearDown” annotated methods are not directed to the console when using nunit3-console.exe runner
  • 1063 Error in Random Test

0.13.22 NUnit 3.0.0 Final Release (3.0.5797) - November 15, 2015

0.13.22.1 Issues Resolved

  • 635 Mono 4.0 Support

0.13.23 NUnit 3.0.0 Release Candidate 3 (3.0.5795) - November 13, 2015

0.13.23.1 Engine

  • The engine now only sets the config file for project.nunit to project.config if project.config exists. Otherwise, each assembly uses its own config, provided it is run in a separate AppDomain by itself.

NOTE: It is not possible for multiple assemblies in the same AppDomain to use different configs. This is not an NUnit limitation, it’s just how configs work!

0.13.23.2 Issues Resolved

  • 856 Extensions support for third party runners in NUnit 3.0
  • 1003 Delete TeamCityEventHandler as it is not used
  • 1015 Specifying .nunit project and –framework on command line causes crash
  • 1017 Remove Assert.Multiple from framework

0.13.24 NUnit 3.0.0 Release Candidate 2 (3.0.5790) - November 8, 2015

0.13.24.1 Engine

  • The IDriverFactory extensibility interface has been modified.

0.13.24.2 Issues Resolved

  • 970 Define PARALLEL in CF build of nunitlite
  • 978 It should be possible to determine version of NUnit using nunit console tool
  • 983 Inconsistent return codes depending on ProcessModel
  • 986 Update docs for parallel execution
  • 988 Don’t run portable tests from NUnit Console
  • 990 V2 driver is passing invalid filter elements to NUnit
  • 991 Mono.Options should not be exposed to public directly
  • 993 Give error message when a regex filter is used with NUnit V2
  • 997 Add missing XML Documentation
  • 1008 NUnitLite namespace not updated in the NuGet Packages

0.13.25 NUnit 3.0.0 Release Candidate (3.0.5783) - November 1, 2015

0.13.25.1 Framework

  • The portable build now supports ASP.NET 5 and the new Core CLR.

NOTE: The nunit3-console runner cannot run tests that reference the portable build. You may run such tests using NUnitLite or a platform-specific runner.

  • TestCaseAttribute and TestCaseData now allow modifying the test name without replacing it entirely.
  • The Silverlight packages for are now separate downloads.

0.13.25.2 NUnitLite

  • The NUnitLite runner now produces the same output display and XML results as the console runner.

0.13.25.3 Engine

  • The format of the XML result file has been finalized and documented.

0.13.25.4 Console Runner

  • The console runner program is now called nunit3-console.
  • Console runner output has been modified so that the summary comes at the end, to reduce the need for scrolling.

0.13.25.5 Issues Resolved

  • 59 Length of generated test names should be limited
  • 68 Customization of test case name generation
  • 404 Split tests between nunitlite.runner and nunit.framework
  • 575 Add support for ASP.NET 5 and the new Core CLR
  • 783 Package separately for Silverlight
  • 833 Intermittent failure of WorkItemQueueTests.StopQueue_WithWorkers
  • 859 NUnit-Console output - move Test Run Summary to end
  • 867 Remove Warnings from Ignored tests
  • 868 Review skipped tests
  • 887 Move environment and settings elements to the assembly suite in the result file
  • 899 Colors for ColorConsole on grey background are too light
  • 904 InternalPreserveStackTrace is not supported on all Portable platforms
  • 914 Unclear error message from console runner when assembly has no tests
  • 916 Console runner dies when test agent dies
  • 918 Console runner –where parameter is case sensitive
  • 920 Remove addins.engine.api.dll from NuGet package
  • 929 Rename nunit-console.exe
  • 931 Remove beta warnings from NuGet packages
  • 936 Explicit skipped tests not displayed
  • 939 Installer complains about .NET even if already installed
  • 940 Confirm or modify list of packages for release
  • 947 Breaking API change in ValueSourceAttribute
  • 949 Update copyright in NUnit Console
  • 954 NUnitLite XML output is not consistent with the engine’s
  • 955 NUnitLite does not display the where clause
  • 959 Restore filter options for NUnitLite portable build
  • 960 Intermittent failure of CategoryFilterTests
  • 967 Run Settings Report is not being displayed.

0.13.26 NUnit 3.0.0 Beta 5 (3.0.5767) - October 16, 2015

0.13.26.1 Framework

  • Parameterized test cases now support nullable arguments.
  • The NUnit framework may now be built for the .NET Core framework. Note that this is only available through building the source code. A binary will be available in the next release.

0.13.26.2 Engine

  • The engine now runs multiple test assemblies in parallel by default
  • The output XML now includes more information about the test run, including the text of the command used, any engine settings and the filter used to select tests.
  • Extensions may now specify data in an identifying attribute, for use by the engine in deciding whether to load that extension.

0.13.26.3 Console Runner

  • The console now displays all settings used by the engine to run tests as well as the filter used to select tests.
  • The console runner accepts a new option –maxagents. If multiple assemblies are run in separate processes, this value may be used to limit the number that are executed simultaneously in parallel.
  • The console runner no longer accepts the –include and –exclude options. Instead, the new –where option provides a more general way to express which tests will be executed, such as –where “cat==Fast && Priority==High”. See the docs for details of the syntax.
  • The new –debug option causes NUnit to break in the debugger immediately before tests are run. This simplifies debugging, especially when the test is run in a separate process.

0.13.26.4 Issues Resolved

  • 41 Check for zeroes in Assert messages
  • 254 Finalize XML format for test results
  • 275 NUnitEqualityComparer fails to compare IEquatable<T> where second object is derived from T
  • 304 Run test Assemblies in parallel
  • 374 New syntax for selecting tests to be run
  • 515 OSPlatform.IsMacOSX doesn’t work
  • 573 nunit-console hangs on Mac OS X after all tests have run
  • 669 TeamCity service message should have assembly name as a part of test name.
  • 689 The TeamCity service message “testFinished” should have an integer value in the “duration” attribute
  • 713 Include command information in XML
  • 719 We have no way to configure tests for several assemblies using NUnit project file and the common installation from msi file
  • 735 Workers number in xml report file cannot be found
  • 784 Build Portable Framework on Linux
  • 790 Allow Extensions to provide data through an attribute
  • 794 Make it easier to debug tests as well as NUnit itself
  • 801 NUnit calls Dispose multiple times
  • 814 Support nullable types with TestCase
  • 818 Possible error in Merge Pull Request #797
  • 821 Wrapped method results in loss of result information
  • 822 Test for Debugger in NUnitTestAssemblyRunner probably should not be in CF build
  • 824 Remove unused System.Reflection using statements
  • 826 Randomizer uniqueness tests fail randomly!
  • 828 Merge pull request #827 (issue 826)
  • 830 Add ability to report test results synchronously to test runners
  • 837 Enumerators not disposed when comparing IEnumerables
  • 840 Add missing copyright notices
  • 844 Pull Request #835 (Issue #814) does not build in CF
  • 847 Add new –process:inprocess and –inprocess options
  • 850 Test runner fails if test name contains invalid xml characters
  • 851 ‘Exclude’ console option is not working in NUnit Lite
  • 853 Cannot run NUnit Console from another directory
  • 860 Use CDATA section for message, stack-trace and output elements of XML
  • 863 Eliminate core engine
  • 865 Intermittent failures of StopWatchTests
  • 869 Tests that use directory separator char to determine platform misreport Linux on MaxOSX
  • 870 NUnit Console Runtime Environment misreports on MacOSX
  • 874 Add .NET Core Framework
  • 878 Cannot exclude MacOSX or XBox platforms when running on CF
  • 892 Fixed test runner returning early when executing more than one test run.
  • 894 Give nunit.engine and nunit.engine.api assemblies strong names
  • 896 NUnit 3.0 console runner not placing test result xml in –work directory

0.13.27 NUnit 3.0.0 Beta 4 (3.0.5715) - August 25, 2015

0.13.27.1 Framework

  • A new RetryAttribute allows retrying of failing tests.
  • New SupersetConstraint and Is.SupersetOf syntax complement SubsetConstraint.
  • Tests skipped due to ExplicitAttribute are now reported as skipped.

0.13.27.2 Engine

  • We now use Cecil to examine assemblies prior to loading them.
  • Extensions are no longer based on Mono.Addins but use our own extension framework.

0.13.27.3 Issues Resolved

  • 125 3rd-party dependencies should be downloaded on demand
  • 283 What should we do when a user extension does something bad?
  • 585 RetryAttribute
  • 642 Restructure MSBuild script
  • 649 Change how we zip packages
  • 654 ReflectionOnlyLoad and ReflectionOnlyLoadFrom
  • 664 Invalid “id” attribute in the report for case “test started”
  • 685 In the some cases when tests cannot be started NUnit returns exit code “0”
  • 728 Missing Assert.That overload
  • 741 Explicit Tests get run when using –exclude
  • 746 Framework should send events for all tests
  • 747 NUnit should apply attributes even if test is non-runnable
  • 749 Review Use of Mono.Addins for Engine Extensibility
  • 750 Include Explicit Tests in Test Results
  • 753 Feature request: Is.SupersetOf() assertion constraint
  • 755 TimeOut attribute doesn’t work with TestCaseSource Attribute
  • 757 Implement some way to wait for execution to complete in ITestEngineRunner
  • 760 Packaging targets do not run on Linux
  • 766 Added overloads for True()/False() accepting booleans
  • 778 Build and build.cmd scripts invoke nuget.exe improperly
  • 780 Teamcity fix
  • 782 No sources for 2.6.4

0.13.28 NUnit 3.0.0 Beta 3 (3.0.5674) - July 15, 2015

0.13.28.1 Framework

  • The RangeAttribute has been extended to support more data types including uint, long and ulong
  • Added platform support for Windows 10 and fixed issues with Windows 8 and 8.1 support
  • Added async support to the portable version of NUnit Framework
  • The named members of the TestCaseSource and ValueSource attributes must now be static.
  • RandomAttribute has been extended to add support for new data types including uint, long, ulong, short, ushort, float, byte and sbyte
  • TestContext.Random has also been extended to add support for new data types including uint, long, ulong, short, ushort, float, byte, sbyte and decimal
  • Removed the dependency on Microsoft.Bcl.Async from the NUnit Framework assembly targeting .NET 4.0. If you want to write async tests in .NET 4.0, you will need to reference the NuGet package yourself.
  • Added a new TestFixtureSource attribute which is the equivalent to TestCaseSource but provides for instantiation of fixtures.
  • Significant improvements have been made in how NUnit deduces the type arguments of generic methods based on the arguments provided.

0.13.28.2 Engine

  • If the target framework is not specified, test assemblies that are compiled to target .NET 4.5 will no longer run in .NET 4.0 compatibility mode

0.13.28.3 Console

  • If the console is run without arguments, it will now display help

0.13.28.4 Issues Resolved

  • 47 Extensions to RangeAttribute
  • 237 System.Uri .ctor works not properly under Nunit
  • 244 NUnit should properly distinguish between .NET 4.0 and 4.5
  • 310 Target framework not specified on the AppDomain when running against .Net 4.5
  • 321 Rationalize how we count tests
  • 472 Overflow exception and DivideByZero exception from the RangeAttribute
  • 524 int and char do not compare correctly?
  • 539 Truncation of string arguments
  • 544 AsyncTestMethodTests for 4.5 Framework fails frequently on Travis CI
  • 656 Unused parameter in Console.WriteLine found
  • 670 Failing Tests in TeamCity Build
  • 673 Ensure proper disposal of engine objects
  • 674 Engine does not release test assemblies
  • 679 Windows 10 Support
  • 682 Add Async Support to Portable Framework
  • 683 Make FrameworkController available in portable build
  • 687 TestAgency does not launch agent process correctly if runtime type is not specified (i.e. v4.0)
  • 692 PlatformAttribute_OperatingSystemBitNess fails when running in 32-bit process
  • 693 Generic Test<T> Method cannot determine type arguments for fixture when passed as IEnumerable<T>
  • 698 Require TestCaseSource and ValueSource named members to be static
  • 703 TeamCity non-equal flowid for ‘testStarted’ and ‘testFinished’ messages
  • 712 Extensions to RandomAttribute
  • 715 Provide a data source attribute at TestFixture Level
  • 718 RangeConstraint gives error with from and two args of differing types
  • 723 Does nunit.nuspec require dependency on Microsoft.Bcl.Async?
  • 724 Adds support for Nullable<bool> to Assert.IsTrue and Assert.IsFalse
  • 734 Console without parameters doesn’t show help

0.13.29 NUnit 3.0.0 Beta 2 (3.0.5610) - May 12, 2015

0.13.29.1 Framework

  • The Compact Framework version of the framework is now packaged separately and will be distributed as a ZIP file and as a NuGet package.
  • The NUnit 2.x RepeatAttribute was added back into the framework.
  • Added Throws.ArgumentNullException
  • Added GetString methods to NUnit.Framework.Internal.RandomGenerator to create repeatable random strings for testing
  • When checking the equality of DateTimeOffset, you can now use the WithSameOffset modifier
  • Some classes intended for internal usage that were public for testing have now been made internal. Additional classes will be made internal for the final 3.0 release.

0.13.29.2 Engine

  • Added a core engine which is a non-extensible, minimal engine for use by devices and similar situations where reduced functionality is compensated for by reduced size and simplicity of usage. See https://github.com/nunit/dev/wiki/Core-Engine for more information.

0.13.29.3 Issues Resolved

  • 22 Add OSArchitecture Attribute to Environment node in result xml
  • 24 Assert on Dictionary Content
  • 48 Explicit seems to conflict with Ignore
  • 168 Create NUnit 3.0 documentation
  • 196 Compare DateTimeOffsets including the offset in the comparison
  • 217 New icon for the 3.0 release
  • 316 NUnitLite TextUI Runner
  • 320 No Tests found: Using parametrized Fixture and TestCaseSource
  • 360 Better exception message when using non-BCL class in property
  • 454 Rare registry configurations may cause NUnit to fail
  • 478 RepeatAttribute
  • 481 Testing multiple assemblies in nunitlite
  • 538 Potential bug using TestContext in constructors
  • 546 Enable Parallel in NUnitLite/CF (or more) builds
  • 551 TextRunner not passing the NumWorkers option to the ITestAssemblyRunner
  • 556 Executed tests should always return a non-zero duration
  • 559 Fix text of NuGet packages
  • 560 Fix PackageVersion property on wix install projects
  • 562 Program.cs in NUnitLite NuGet package is incorrect
  • 564 NUnitLite Nuget package is Beta 1a, Framework is Beta 1
  • 565 NUnitLite Nuget package adds Program.cs to a VB Project
  • 568 Isolate packaging from building
  • 570 ThrowsConstraint failure message should include stack trace of actual exception
  • 576 Throws.ArgumentNullException would be nice
  • 577 Documentation on some members of Throws falsely claims that they return TargetInvocationException constraints
  • 579 No documentation for recommended usage of TestCaseSourceAttribute
  • 580 TeamCity Service Message Uses Incorrect Test Name with NUnit2Driver
  • 582 Test Ids Are Not Unique
  • 583 TeamCity service messages to support parallel test execution
  • 584 Non-runnable assembly has incorrect ResultState
  • 609 Add support for integration with TeamCity
  • 611 Remove unused –teamcity option from CF build of NUnitLite
  • 612 MaxTime doesn’t work when used for TestCase
  • 621 Core Engine
  • 622 nunit-console fails when use –output
  • 628 Modify IService interface and simplify ServiceContext
  • 631 Separate packaging for the compact framework
  • 646 ConfigurationManager.AppSettings Params Return Null under Beta 1

0.13.30 NUnit 3.0.0 Beta 1 (3.0.5562) - March 25, 2015

0.13.30.1 General

  • There is now a master windows installer for the framework, engine and console runner.

0.13.30.2 Framework

  • We no longer create a separate framework build for .NET 3.5. The 2.0 and 3.5 builds were essentially the same, so the former should now be used under both runtimes.
  • A new Constraint, <b>DictionaryContainsKeyConstraint</b>, may be used to test that a specified key is present in a dictionary.
  • <b>LevelOfParallelizationAttribute</b> has been renamed to <b>LevelOfParallelismAttribute</b>.
  • The Silverlight runner now displays output in color and includes any text output created by the tests.
  • The class and method names of each test are included in the output xml where applicable.
  • String arguments used in test case names are now truncated to 40 rather than 20 characters.

0.13.30.3 Engine

  • The engine API has now been finalized. It permits specifying a minimum version of the engine that a runner is able to use. The best installed version of the engine will be loaded. Third-party runners may override the selection process by including a copy of the engine in their installation directory and specifying that it must be used.
  • The V2 framework driver now uses the event listener and test listener passed to it by the runner. This corrects several outstanding issues caused by events not being received and allows selecting V2 tests to be run from the command-line, in the same way that V3 tests are selected.

0.13.30.4 Console

  • The console now defaults to not using shadowcopy. There is a new option –shadowcopy to turn it on if needed.

0.13.30.5 Issues Resolved

  • 224 Silverlight Support
  • 318 TestActionAttribute: Retrieving the TestFixture
  • 428 Add ExpectedExceptionAttribute to C# samples
  • 440 Automatic selection of Test Engine to use
  • 450 Create master install that includes the framework, engine and console installs
  • 477 Assert does not work with ArraySegment
  • 482 nunit-console has multiple errors related to -framework option
  • 483 Adds constraint for asserting that a dictionary contains a particular key
  • 484 Missing file in NUnit.Console nuget package
  • 485 Can’t run v2 tests with nunit-console 3.0
  • 487 NUnitLite can’t load assemblies by their file name
  • 488 Async setup and teardown still don’t work
  • 497 Framework installer shold register the portable framework
  • 504 Option –workers:0 is ignored
  • 508 Travis builds with failure in engine tests show as successful
  • 509 Under linux, not all mono profiles are listed as available
  • 512 Drop the .NET 3.5 build
  • 517 V2 FrameworkDriver does not make use of passed in TestEventListener
  • 523 Provide an option to disable shadowcopy in NUnit v3
  • 528 V2 FrameworkDriver does not make use of passed in TestFilter
  • 530 Color display for Silverlight runner
  • 531 Display text output from tests in Silverlight runner
  • 534 Add classname and methodname to test result xml
  • 541 Console help doesn’t indicate defaults

<h3>NUnit 3.0.0 Alpha 5 (3.0.5509) - January 30, 2015</h3>

<h4>General</h4>

<ul> <li>A Windows installer is now included in the release packages. </ul>

<h4>Framework</h4>

<ul> <li>TestCaseAttribute now allows arguments with default values to be omitted. Additionaly, it accepts IncludePlatform and ExcludePlatform properties to specify the platforms on which the test case should be run. <li>TestFixture and TestCase attributes now enforce the requirement that a reason needs to be provided when ignoring a test. <li>SetUp, TearDown, OneTimeSetUp and OneTimeTearDown methods may now be async. <li>String arguments over 20 characters in length are truncated when used as part of a test name. </ul>

<h4>Engine</h4>

<ul> <li>The engine is now extensible using Mono.Addins. In this release, extension points are provided for FrameworkDrivers, ProjectLoaders and OutputWriters. The following addins are bundled as a part of NUnit: <ul> <li>A FrameworkDriver that allows running NUnit V2 tests under NUnit 3.0. <li>ProjectLoaders for NUnit and Visual Studio projects. <li>An OutputWriter that creates XML output in NUnit V2 format. </ul> <li>DomainUsage now defaults to Multiple if not specified by the runner </ul>

<h4>Console</h4>

<ul> <li>New options supported: <ul> <li>–testlist provides a list of tests to run in a file <li>–stoponerror indicates that the run should terminate when any test fails. </ul> </ul>

<h4>Issues Resolved</h4>

<ul> <li>20 TestCaseAttribute needs Platform property. <li>60 NUnit should support async setup, teardown, fixture setup and fixture teardown. <li>257 TestCaseAttribute should not require parameters with default values to be specified. <li>266 Pluggable framework drivers. <li>368 Create addin model. <li>369 Project loader addins <li>370 OutputWriter addins <li>403 Move ConsoleOptions.cs and Options.cs to Common and share… <li>419 Create Windows Installer for NUnit. <li>427 [TestFixture(Ignore=true)] should not be allowed. <li>437 Errors in tests under Linux due to hard-coded paths. <li>441 NUnit-Console should support –testlist option <li>442 Add –stoponerror option back to nunit-console. <li>456 Fix memory leak in RuntimeFramework. <li>459 Remove the Mixed Platforms build configuration. <li>468 Change default domain usage to multiple. <li>469 Truncate string arguments in test names in order to limit the length. </ul>

<h3>NUnit 3.0.0 Alpha 4 (3.0.5476) - December 30, 2014</h3>

<h4>Framework</h4>

<ul> <li>ApartmentAttribute has been added, replacing STAAttribute and MTAAttribute. <li>Unnecessary overloads of Assert.That and Assume.That have been removed. <li>Multiple SetUpFixtures may be specified in a single namespace. <li>The Pairwise strategy test case generation algorithm has been improved. </ul>

<h4>Engine</h4>

<ul> <li>A driver is now included, which allows running NUnit 2.x tests under NUnit 3.0. <li>The engine can now load and run tests specified in a number of project formats: <ul> <li>NUnit (.nunit) <li>Visual Studio C# projects (.csproj) <li>Visual Studio F# projects (.vjsproj) <li>Visual Studio Visual Basic projects (.vbproj) <li>Visual Studio solutions (.sln) <li>Legacy C++ and Visual JScript projects (.csproj and .vjsproj) are also supported <li>Support for the current C++ format (.csxproj) is not yet available </ul> <li>Creation of output files like TestResult.xml in various formats is now a service of the engine, available to any runner. </ul>

<h4>Console</h4>

<ul> <li>The command-line may now include any number of assemblies and/or supported projects. </ul>

<h4>Issues Resolved</h4>

<ul> <li>37 Multiple SetUpFixtures should be permitted on same namespace <li>210 TestContext.WriteLine in an AppDomain causes an error <li>227 Add support for VS projects and solutions <li>231 Update C# samples to use NUnit 3.0 <li>233 Update F# samples to use NUnit 3.0 <li>234 Update C++ samples to use NUnit 3.0 <li>265 Reorganize console reports for nunit-console and nunitlite <li>299 No full path to assembly in XML file under Compact Framework <li>301 Command-line length <li>363 Make Xml result output an engine service <li>377 CombiningStrategyAttributes don’t work correctly on generic methods <li>388 Improvements to NUnitLite runner output <li>390 Specify exactly what happens when a test times out <li>396 ApartmentAttribute <li>397 CF nunitlite runner assembly has the wrong name <li>407 Assert.Pass() with ]]> in message crashes console runner <li>414 Simplify Assert overloads <li>416 NUnit 2.x Framework Driver <li>417 Complete work on NUnit projects <li>420 Create Settings file in proper location </ul>

<h3>NUnit 3.0.0 Alpha 3 (3.0.5446) - November 29, 2014</h3>

<h4>Breaking Changes</h4>

<ul> <li>NUnitLite tests must reference both the nunit.framework and nunitlite assemblies. </ul>

<h4>Framework</h4>

<ul> <li>The NUnit and NUnitLite frameworks have now been merged. There is no longer any distinction between them in terms of features, although some features are not available on all platforms. <li>The release includes two new framework builds: compact framework 3.5 and portable. The portable library is compatible with .NET 4.5, Silverlight 5.0, Windows 8, Windows Phone 8.1, Windows Phone Silverlight 8, Mono for Android and MonoTouch. <li>A number of previously unsupported features are available for the Compact Framework: <ul> <li>Generic methods as tests <li>RegexConstraint <li>TimeoutAttribute <li>FileAssert, DirectoryAssert and file-related constraints </ul> </ul>

<h4>Engine</h4>

<ul> <li>The logic of runtime selection has now changed so that each assembly runs by default in a separate process using the runtime for which it was built. <li>On 64-bit systems, each test process is automatically created as 32-bit or 64-bit, depending on the platform specified for the test assembly. </ul>

<h4>Console</h4>

<ul> <li>The console runner now runs tests in a separate process per assembly by default. They may still be run in process or in a single separate process by use of command-line options. <li>The console runner now starts in the highest version of the .NET runtime available, making it simpler to debug tests by specifying that they should run in-process on the command-line. <li>The -x86 command-line option is provided to force execution in a 32-bit process on a 64-bit system. <li>A writeability check is performed for each output result file before trying to run the tests. <li>The -teamcity option is now supported. </ul>

<h4>Issues Resolved</h4>

<ul> <li>12 Compact framework should support generic methods <li>145 NUnit-console fails if test result message contains invalid xml characters <li>155 Create utility classes for platform-specific code <li>223 Common code for NUnitLite console runner and NUnit-Console <li>225 Compact Framework Support <li>238 Improvements to running 32 bit tests on a 64 bit system <li>261 Add portable nunitlite build <li>284 NUnitLite Unification <li>293 CF does not have a CurrentDirectory <li>306 Assure NUnit can write resultfile <li>308 Early disposal of runners <li>309 NUnit-Console should support incremental output under TeamCity <li>325 Add RegexConstraint to compact framework build <li>326 Add TimeoutAttribute to compact framework build <li>327 Allow generic test methods in the compact framework <li>328 Use .NET Stopwatch class for compact framework builds <li>331 Alpha 2 CF does not build <li>333 Add parallel execution to desktop builds of NUnitLite <li>334 Include File-related constraints and syntax in NUnitLite builds <li>335 Re-introduce ‘Classic’ NUnit syntax in NUnitLite <li>336 Document use of separate obj directories per build in our projects <li>337 Update Standard Defines page for .NET 3.0 <li>341 Move the NUnitLite runners to separate assemblies <li>367 Refactor XML Escaping Tests <li>372 CF Build TestAsesemblyRunnerTests <li>373 Minor CF Test Fixes <li>378 Correct documentation for PairwiseAttribute <li>386 Console Output Improvements </ul>

<h3>NUnit 3.0.0 Alpha 2 (3.0.5419) - November 2, 2014</h3>

<h4>Breaking Changes</h4>

<ul> <li>The console runner no longer displays test results in the debugger. <li>The NUnitLite compact framework 2.0 build has been removed. <li>All addin support has been removed from the framework. Documentation of NUnit 3.0 extensibility features will be published in time for the beta release. In the interim, please ask for support on the nunit-discuss list. </ul>

<h4>General</h4>

<ul> <li>A separate solution has been created for Linux <li>We now have continuous integration builds under both Travis and Appveyor <li>The compact framework 3.5 build is now working and will be supported in future releases. </ul>

<h4>New Features</h4>

<ul> <li>The console runner now automatically detects 32- versus 64-bit test assemblies. <li>The NUnitLite report output has been standardized to match that of nunit-console. <li>The NUnitLite command-line has been standardized to match that of nunit-console where they share the same options. <li>Both nunit-console and NUnitLite now display output in color. <li>ActionAttributes now allow specification of multiple targets on the attribute as designed. This didn’t work in the first alpha. <li>OneTimeSetUp and OneTimeTearDown failures are now shown on the test report. Individual test failures after OneTimeSetUp failure are no longer shown. <li>The console runner refuses to run tests build with older versions of NUnit. A plugin will be available to run older tests in the future. </ul>

<h4>Issues Resolved</h4>

<ul> <li>222 Color console for NUnitLite <li>229 Timing failures in tests <li>241 Remove reference to Microslft BCL packages <li>243 Create solution for Linux <li>245 Multiple targets on action attributes not implemented <li>246 C++ tests do not compile in VS2013 <li>247 Eliminate trace display when running tests in debug <li>255 Add new result states for more precision in where failures occur <li>256 ContainsConstraint break when used with AndConstraint <li>264 Stacktrace displays too many entries <li>269 Add manifest to nunit-console and nunit-agent <li>270 OneTimeSetUp failure results in too much output <li>271 Invalid tests should be treated as errors <li>274 Command line options should be case insensitive <li>276 NUnit-console should not reference nunit.framework <li>278 New result states (ChildFailure and SetupFailure) break NUnit2XmlOutputWriter <li>282 Get tests for NUnit2XmlOutputWriter working <li>288 Set up Appveyor CI build <li>290 Stack trace still displays too many items <li>315 NUnit 3.0 alpha: Cannot run in console on my assembly <li>319 CI builds are not treating test failures as failures of the build <li>322 Remove Stopwatch tests where they test the real .NET Stopwatch </ul>

<h3>NUnit 3.0.0 Alpha 1 (3.0.5378) - September 22, 2014</h3>

<h4>Breaking Changes</h4>

<ul> <li>Legacy suites are no longer supported <li>Assert.NullOrEmpty is no longer supported (Use Is.Null.Or.Empty) </ul>

<h4>General</h4>

<ul> <li>MsBuild is now used for the build rather than NAnt <li>The framework test harness has been removed now that nunit-console is at a point where it can run the tests. </ul>

<h4>New Features</h4>

<ul> <li>Action Attributes have been added with the same features as in NUnit 2.6.3. <li>TestContext now has a method that allows writing to the XML output. <li>TestContext.CurrentContext.Result now provides the error message and stack trace during teardown. <li>Does prefix operator supplies several added constraints. </ul>

<h4>Issues Resolved</h4> <ul> <li>6 Log4net not working with NUnit <li>13 Standardize commandline options for nunitlite runner <li>17 No allowance is currently made for nullable arguents in TestCase parameter conversions <li>33 TestCaseSource cannot refer to a parameterized test fixture <li>54 Store message and stack trace in TestContext for use in TearDown <li>111 Implement Changes to File, Directory and Path Assertions <li>112 Implement Action Attributes <li>156 Accessing multiple AppDomains within unit tests result in SerializationException <li>163 Add –trace option to NUnitLite <li>167 Create interim documentation for the alpha release <li>169 Design and implement distribution of NUnit packages <li>171 Assert.That should work with any lambda returning bool <li>175 Test Harness should return an error if any tests fail <li>180 Errors in Linux CI build <li>181 Replace NAnt with MsBuild / XBuild <li>183 Standardize commandline options for test harness <li>188 No output from NUnitLite when selected test is not found <li>189 Add string operators to Does prefix <li>193 TestWorkerTests.BusyExecutedIdleEventsCalledInSequence fails occasionally <li>197 Deprecate or remove Assert.NullOrEmpty <li>202 Eliminate legacy suites <li>203 Combine framework, engine and console runner in a single solution and repository <li>209 Make Ignore attribute’s reason mandatory <li>215 Running 32-bit tests on a 64-bit OS <li>219 Teardown failures are not reported </ul>

<h4>Console Issues Resolved (Old nunit-console project, now combined with nunit)</h4> <ul> <li>2 Failure in TestFixtureSetUp is not reported correctly <li>5 CI Server for nunit-console <li>6 System.NullReferenceException on start nunit-console-x86 <li>21 NUnitFrameworkDriverTests fail if not run from same directory <li>24 ‘Debug’ value for /trace option is deprecated in 2.6.3 <li>38 Confusing Excluded categories output </ul>

<h3>NUnit 2.9.7 - August 8, 2014</h3>

<h4>Breaking Changes</h4>

<ul> <li>NUnit no longer supports void async test methods. You should use a Task return Type instead. <li>The ExpectedExceptionAttribute is no longer supported. Use Assert.Throws() or Assert.That(…, Throws) instead for a more precise specification of where the exception is expected to be thrown. </ul>

<h4>New Features</h4>

<ul> <li>Parallel test execution is supported down to the Fixture level. Use ParallelizableAttribute to indicate types that may be run in parallel. <li>Async tests are supported for .NET 4.0 if the user has installed support for them. <li>A new FileExistsConstraint has been added along with FileAssert.Exists and FileAssert.DoesNotExist <li>ExpectedResult is now supported on simple (non-TestCase) tests. <li>The Ignore attribute now takes a named parameter Until, which allows specifying a date after which the test is no longer ignored. <li>The following new values are now recognized by PlatformAttribute: Win7, Win8, Win8.1, Win2012Server, Win2012ServerR2, NT6.1, NT6.2, 32-bit, 64-bit <li>TimeoutAttribute is now supported under Silverlight <li>ValuesAttribute may be used without any values on an enum or boolean argument. All possible values are used. <li>You may now specify a tolerance using Within when testing equality of DateTimeOffset values. <li>The XML output now includes a start and end time for each test. </ul>

<h4>Issues Resolved</h4>

<ul> <li>8 [SetUpFixture] is not working as expected <li>14 CI Server for NUnit Framework <li>21 Is.InRange Constraint Ambiguity <li>27 Values attribute support for enum types <li>29 Specifying a tolerance with “Within” doesn’t work for DateTimeOffset data types <li>31 Report start and end time of test execution <li>36 Make RequiresThread, RequiresSTA, RequiresMTA inheritable <li>45 Need of Enddate together with Ignore <li>55 Incorrect XML comments for CollectionAssert.IsSubsetOf <li>62 Matches(Constraint) does not work as expected <li>63 Async support should handle Task return type without state machine <li>64 AsyncStateMachineAttribute should only be checked by name <li>65 Update NUnit Wiki to show the new location of samples <li>66 Parallel Test Execution within test assemblies <li>67 Allow Expected Result on simple tests <li>70 EquivalentTo isn’t compatible with IgnoreCase for dictioneries <li>75 Async tests should be supported for projects that target .NET 4.0 <li>82 nunit-framework tests are timing out on Linux <li>83 Path-related tests fail on Linux <li>85 Culture-dependent NUnit tests fail on non-English machine <li>88 TestCaseSourceAttribute documentation <li>90 EquivalentTo isn’t compatible with IgnoreCase for char <li>100 Changes to Tolerance definitions <li>110 Add new platforms to PlatformAttribute <li>113 Remove ExpectedException <li>118 Workarounds for missing InternalPreserveStackTrace in mono <li>121 Test harness does not honor the –worker option when set to zero <li>129 Standardize Timeout in the Silverlight build <li>130 Add FileAssert.Exists and FileAssert.DoesNotExist <li>132 Drop support for void async methods <li>153 Surprising behavior of DelayedConstraint pollingInterval <li>161 Update API to support stopping an ongoing test run </ul>

<h3>NUnit 2.9.6 - October 4, 2013</h3>

<h4>Main Features</h4>

<ul> <li>Separate projects for nunit-console and nunit.engine <li>New builds for .NET 4.5 and Silverlight <li>TestContext is now supported and includes an additional property, <b>Random</b>, which may be used to generate repeatable random values for use in a test. <li>The external framework API is now stable; internal interfaces are separate from API <li>Tests may be run in parallel on separate threads <li>Solutions and projects now use VS2012 (except for Compact framework) </ul>

<h4>Bug Fixes</h4>

<ul> <li>463470 We should encapsulate references to pre-2.0 collections <li>498690 Assert.That() doesn’t like properties with scoped setters <li>501784 Theory tests do not work correctly when using null parameters <li>531873 Feature: Extraction of unit tests from NUnit test assembly and calling appropriate one <li>611325 Allow Teardown to detect if last test failed <li>611938 Generic Test Instances disappear <li>655882 Make CategoryAttribute inherited <li>664081 Add Server2008 R2 and Windows 7 to PlatformAttribute <li>671432 Upgrade NAnt to Latest Release <li>676560 Assert.AreEqual does not support IEquatable<T> <li>691129 Add Category parameter to TestFixture <li>697069 Feature request: dynamic location for TestResult.xml <li>708173 NUnit’s logic for comparing arrays - use Comparer<T[]> if it is provided <li>709062 “System.ArgumentException : Cannot compare” when the element is a list <li>712156 Tests cannot use AppDomain.SetPrincipalPolicy <li>719184 Platformdependency in src/ClientUtilities/util/Services/DomainManager.cs:40 <li>719187 Using Path.GetTempPath() causes conflicts in shared temporary folders <li>735851 Add detection of 3.0, 3.5 and 4.0 frameworks to PlatformAttribute <li>736062 Deadlock when EventListener performs a Trace call + EventPump synchronisation <li>756843 Failing assertion does not show non-linear tolerance mode <li>766749 net-2.0-console-x86.exe.config should have a <startup /> element and also enable loadFromRemoteSources <li>770471 Assert.IsEmpty does not support IEnumerable <li>785460 Add Category parameter to TestCaseSourceAttribute <li>787106 EqualConstraint provides inadequate failure information for IEnumerables <li>792466 TestContext MethodName <li>794115 HashSet incorrectly reported <li>800089 Assert.Throws() hides details of inner AssertionException <li>848713 Feature request: Add switch for console to break on any test case error <li>878376 Add ‘Exactly(n)’ to the NUnit constraint syntax <li>882137 When no tests are run, higher level suites display as Inconclusive <li>882517 NUnit 2.5.10 doesn’t recognize TestFixture if there are only TestCaseSource inside <li>885173 Tests are still executed after cancellation by user <li>885277 Exception when project calls for a runtime using only 2 digits <li>885604 Feature request: Explicit named parameter to TestCaseAttribute <li>890129 DelayedConstraint doesn’t appear to poll properties of objects <li>892844 Not using Mono 4.0 profile under Windows <li>893919 DelayedConstraint fails polling properties on references which are initially null <li>896973 Console output lines are run together under Linux <li>897289 Is.Empty constraint has unclear failure message <li>898192 Feature Request: Is.Negative, Is.Positive <li>898256 IEnumerable<T> for Datapoints doesn’t work <li>899178 Wrong failure message for parameterized tests that expect exceptions <li>904841 After exiting for timeout the teardown method is not executed <li>908829 TestCase attribute does not play well with variadic test functions <li>910218 NUnit should add a trailing separator to the ApplicationBase <li>920472 CollectionAssert.IsNotEmpty must dispose Enumerator <li>922455 Add Support for Windows 8 and Windows 2012 Server to PlatformAttribute <li>928246 Use assembly.Location instead of assembly.CodeBase <li>958766 For development work under TeamCity, we need to support nunit2 formatted output under direct-runner <li>1000181 Parameterized TestFixture with System.Type as constructor arguments fails <li>1000213 Inconclusive message Not in report output <li>1023084 Add Enum support to RandomAttribute <li>1028188 Add Support for Silverlight <li>1029785 Test loaded from remote folder failed to run with exception System.IODirectory <li>1037144 Add MonoTouch support to PlatformAttribute <li>1041365 Add MaxOsX and Xbox support to platform attribute <li>1057981 C#5 async tests are not supported <li>1060631 Add .NET 4.5 build <li>1064014 Simple async tests should not return Task<T> <li>1071164 Support async methods in usage scenarios of Throws constraints <li>1071343 Runner.Load fails on CF if the test assembly contains a generic method <li>1071861 Error in Path Constraints <li>1072379 Report test execution time at a higher resolution <li>1074568 Assert/Assume should support an async method for the ActualValueDelegate <li>1082330 Better Exception if SetCulture attribute is applied multiple times <li>1111834 Expose Random Object as part of the test context <li>1111838 Include Random Seed in Test Report <li>1172979 Add Category Support to nunitlite Runner <li>1203361 Randomizer uniqueness tests sometimes fail <li>1221712 When non-existing test method is specified in -test, result is still “Tests run: 1, Passed: 1” <li>1223294 System.NullReferenceException thrown when ExpectedExceptionAttribute is used in a static class <li>1225542 Standardize commandline options for test harness </ul>

<h3>NUnit 2.9.5 - July 30, 2010</h3>

<h4>Bug Fixes</h4>

<ul> <li>483836 Allow non-public test fixtures consistently <li>487878 Tests in generic class without proper TestFixture attribute should be invalid <li>498656 TestCase should show array values in GUI <li>513989 Is.Empty should work for directories <li>519912 Thread.CurrentPrincipal Set In TestFixtureSetUp Not Maintained Between Tests <li>532488 constraints from ConstraintExpression/ConstraintBuilder are not reusable <li>590717 categorie contains dash or trail spaces is not selectable <li>590970 static TestFixtureSetUp/TestFixtureTearDown methods in base classes are not run <li>595683 NUnit console runner fails to load assemblies <li>600627 Assertion message formatted poorly by PropertyConstraint <li>601108 Duplicate test using abstract test fixtures <li>601645 Parametered test should try to convert data type from source to parameter <li>605432 ToString not working properly for some properties <li>606548 Deprecate Directory Assert in 2.5 and remove it in 3.0 <li>608875 NUnit Equality Comparer incorrectly defines equality for Dictionary objects </ul>

<h3>NUnit 2.9.4 - May 4, 2010</h3>

<h4>Bug Fixes</h4>

<ul> <li>419411 Fixture With No Tests Shows as Non-Runnable <li>459219 Changes to thread princpal cause failures under .NET 4.0 <li>459224 Culture test failure under .NET 4.0 <li>462019 Line endings needs to be better controlled in source <li>462418 Assume.That() fails if I specify a message <li>483845 TestCase expected return value cannot be null <li>488002 Should not report tests in abstract class as invalid <li>490679 Category in TestCaseData clashes with Category on ParameterizedMethodSuite <li>501352 VS2010 projects have not been updated for new directory structure <li>504018 Automatic Values For Theory Test Parameters Not Provided For bool And enum <li>505899 ‘Description’ parameter in both TestAttribute and TestCaseAttribute is not allowed <li>523335 TestFixtureTearDown in static class not executed <li>556971 Datapoint(s)Attribute should work on IEnumerable<T> as well as on Arrays <li>561436 SetCulture broken with 2.5.4 <li>563532 DatapointsAttribute should be allowed on properties and methods </ul>

<h3>NUnit 2.9.3 - October 26, 2009</h3>

<h4>Main Features</h4>

<ul> <li>Created new API for controlling framework <li>New builds for .Net 3.5 and 4.0, compact framework 3.5 <li>Support for old style tests has been removed <li>New adhoc runner for testing the framework </ul>

<h4>Bug Fixes</h4>

<ul> <li>432805 Some Framework Tests don’t run on Linux <li>440109 Full Framework does not support “Contains” </ul>

<h3>NUnit 2.9.2 - September 19, 2009</h3>

<h4>Main Features</h4>

<ul> <li>NUnitLite code is now merged with NUnit <li>Added NUnitLite runner to the framework code <li>Added Compact framework builds </ul>

<h4>Bug Fixes</h4>

<ul> <li>430100 Assert.Catch<T> should return T <li>432566 NUnitLite shows empty string as argument <li>432573 Mono test should be at runtime </ul>

<h3>NUnit 2.9.1 - August 27, 2009</h3>

<h4>Main Features</h4>

<ul> <li>Created a separate project for the framework and framework tests <li>Changed license to MIT / X11 <li>Created Windows installer for the framework </ul>

<h4>Bug Fixes</h4>

<ul> <li>400502 NUnitEqualityComparer.StreamsE­qual fails for same stream <li>400508 TestCaseSource attirbute is not working when Type is given <li>400510 TestCaseData variable length ctor drops values <li>417557 Add SetUICultureAttribute from NUnit 2.5.2 <li>417559 Add Ignore to TestFixture, TestCase and TestCaseData <li>417560 Merge Assert.Throws and Assert.Catch changes from NUnit 2.5.2 <li>417564 TimeoutAttribute on Assembly </ul>

<h3>Earlier Releases</h3>

<ul> <li>Release Notes for <a href=“http://www.nunit.org/?p=releaseNotes&r=2.6.4”>NUnit 2.6 through 2.6.4</a> <li>Release Notes for <a href=“http://www.nunit.org/?p=releaseNotes&r=2.5.10”>NUnit 2.5 through 2.5.10</a> <li>Release Notes for <a href=“http://www.nunit.org/?p=releaseNotes&r=2.4.8”>NUnit 2.4 through 2.4.8</a> <li>Release Notes for <a href=“http://www.nunit.org/?p=releaseNotes&r=2.2.10”>NUnit 2.0 through 2.2.10</a> </ul>

<b>RepeatAttribute</b> is used on a test method to specify that it should be executed multiple times. If any repetition fails, the remaining ones are not run and a failure is reported.

0.13.30.6 Notes:

  1. If RepeatAttribute is used on a parameterized method, each individual test case created for that method is repeated.

  2. It is not currently possible to use RepeatAttribute on a TestFixture or any higher level suite. Only test cases may be repeated.

The <b>RequiresThreadAttribute</b> is used to indicate that a test method, class or assembly should be run on a separate thread. Optionally, the desired apartment for the thread may be specified in the constructor.

<b>Note:</b> This attribute, used with or without an ApartmentState argument will <b>always</b> result in creation of a new thread. To create a thread <b>only</b> if the current ApartmentState is not appropriate, use the <b>ApartmentAttribute</b>.

<h4>Examples</h4>

// A thread will be created and used to run
// all the tests in the assembly
[assembly:RequiresThread]

...

// TestFixture requiring a separate thread
[TestFixture, RequiresThread]
public class FixtureOnThread
{
  // A separate thread will be created and all
  // tests in the fixture will run on it.
}

[TestFixture]
public class AnotherFixture
{
  [Test, RequiresThread]
  public void TestRequiringThread()
  {
    // A separate thread will be created for this test
  }
  
  [Test, RequiresThread(ApartmentState.STA)]
  public void TestRequiringSTAThread()
  {
    // A separate STA thread will be created for this test.
  }
}

<h4>See also…</h4>

  • [[Apartment Attribute]]

Result Writers take the result of a test run, in NUnit 3.0 XML format, and use it to create a result file in some other format. NUnit itself provides a two result writers, one to create output in NUnit V2 format and another to write test cases to the console. The definition of a result writer extension might look something like this:

[Extension]
[ExtensionProperty("Format", "custom")]
public class CustomResultWriterFactory : IResultWriter
{
    ...
}

You must provide an ExtensionPropertyAttribute giving the name of the format you support. Users would access your format from the nunit-console command-line by using that name in a result specification, like

   nunit-console test.dll --result=CustomResult.xml;format=custom

The IResultWriter interface, which you must implement, is defined as follows:

/// <summary>
/// Common interface for objects that process and write out test results
/// </summary>
[TypeExtensionPoint(
    Description = "Supplies a writer to write the result of a test to a file using a specific format.")]
public interface IResultWriter
{
    /// <summary>
    /// Checks if the output path is writable. If the output is not
    /// writable, this method should throw an exception.
    /// </summary>
    /// <param name="outputPath"></param>
    void CheckWritability(string outputPath);

    /// <summary>
    /// Writes result to the specified output path.
    /// </summary>
    /// <param name="resultNode">XmlNode for the result</param>
    /// <param name="outputPath">Path to which it should be written</param>
    void WriteResultFile(XmlNode resultNode, string outputPath);

    /// <summary>
    /// Writes result to a TextWriter.
    /// </summary>
    /// <param name="resultNode">XmlNode for the result</param>
    /// <param name="writer">TextWriter to which it should be written</param>
    void WriteResultFile(XmlNode resultNode, TextWriter writer);
}

The engine calls the CheckWritability method at the start of the run, before executing any tests. The WriteResultFile method is called after the run is complete. The writer may check writability in any way desired, including writing an abbreviated output file, which it will later overwrite. RetryAttribute is used on a test method to specify that it should be rerun if it fails, up to a maximum number of times.

Notes:

  1. It is not currently possible to use RetryAttribute on a TestFixture or any other type of test suite. Only single tests may be repeated.

  2. If a test has an unexpected exception, an error result is returned and it is not retried. Only assertion failures can trigger a retry. To convert an unexpected exception into an assertion failure, see the [[ThrowsConstraint]].

Normally constraints just work. However, attempting to reuse the same constraint in several places can lead to unexpected results.

Consider the following code as an example:

    Constraint myConstraint = Is.Not.Null;
    Assert.That("not a null", myConstraint); // Passes, of course
    Assert.That("not a null", myConstraint); // Fails! What's that about?

We’ll save the technical explanation for later and show the solution first:

    ReusableConstraint myConstraint = Is.Not.Null;
    Assert.That("not a null", myConstraint); // Passes
    Assert.That("not a null", myConstraint); // Passes

Or alternatively..

    var myConstraint = new ReusableConstraint(Is.Not.Null);
    Assert.That("not a null", myConstraint); // Passes
    Assert.That("not a null", myConstraint); // Passes

0.13.31 Technical Explanation

In the original example, the value assigned to myConstraint is known as an <b>unresolved</b> constraint. In fact, it’s an unresolved NullConstraint, because that was the last constraint encountered in the expression. It’s associated with a <b>Not</b> operator that has not yet been applied.

That’s OK for use with Assert.That(), because the method knows how to resolve a constraint before using it. Assert.That() resolves this constraint to a NotConstraint referencing the original NullConstraint.

Of course, the original reference in myConstraint is left unchanged in all of this. But the EqualConstraint it points to has now been resolved. It is now a <b>resolved</b> constraint and can’t be resolved again by the second Assert.That(), which only sees the NullConstraint and not the NotConstraint.

So, for reusability, what we want to save is the result of resolving the constraint, in this case

    NotConstraint => NullConstraint

That’s what <b>ReusableConstraint</b> does for us. It resolves the full expression and saves the result. Then it passes all operations on to that saved result.

<h3>When to Use It</h3>

Use this constraint any time you want to reuse a constraint expression and you’ll be safe.

If you like to take chances, you’ll find that you can avoid using it in the following cases…

<ol> <li> With a simple constraint involving no operators, like…

    Constraint myConstraint = Is.Null;
    Constraint myConstraint = Is.EqualTo(42);

<li> With any constraint you construct using new, without using the “dotted” constraint syntax…

    Constraint myConstraint = new NotConstraint(new NullConstraint());
    Constraint myConstraint = new AndConstraint(
        new GreaterThanConstraint(0), 
        new LessThanConstraint(100));

However, there is no significant penalty to using <b>ReusableConstraint</b>. It makes your intent much clearer and the exceptions listed are accidents of the internal implementation and could disappear in future releases.

Nunit provides three different runners, which may be used to load and run your tests.

  • The [Console Runner], nunit-console.exe, is used for batch execution.
  • The [[Gui Runner]], nunit.exe, provides interactive loading and running of tests.

0.13.32 NUnit Agent

When running tests in a separate process, the console and gui runners make use of the [NUnit Agent] program, nunit-agent.exe. Although not directly run by users, nunit-agent does load and execute tests and users need to be aware of it, especially when debugging is involved.

0.13.33 Third-Party Runners

Various third-party applications are available for loading and running NUnit tests. Some of these actually use NUnit to load the tests, while others provide their own emulation and may not work in the same way that NUnit does.

Because the status of such projects may change from time to time, we don’t discuss them individually here. For the latest information, consult the manufacturer of any third-party software or ask other users on our <a href=“http://groups.google.com/group/nunit-discuss”>discussion list</a>.

0.13.34 Additional Information

For additional general information on how tests are loaded and run, see

  • [[Runtime Selection]]
  • [[Assembly Isolation]]
  • [[Configuration Files]]
  • [[Multiple Assemblies]]
  • [[Visual Studio Support]]

By default, NUnit runs tests under under the runtime version for which the test assembly was built, provided it is available on the test machine. If it is not available, NUnit runs the assembly under the best available runtime that will allow it to run. If no suitable runtime can be found, an error is reported.

0.13.34.1 Overriding the Defaults

The default runtime framework may be overridden using command line options. In all cases, NUnit will attempt to honor the options given, issuing an error message if the assembly cannot be loaded. See [[Console Command Line]] for more information.

  • The <b>/framework</b> option of console runner allows you to specify the framework type and version to be used for a test run. When this option is used, NUnit will attempt to run the tests under the framework specified even if the assembly targets a different runtime.

  • The <b>/process:Single</b> command-line option indicates that tests should be run in the NUnit process itself. This forces usage of the runtime under which NUnit is already running.

  • The <b>process:Separate</b> causes a single separate process to be used for all assemblies. In this case, NUnit will use the highest level runtime targeted by any of the assemblies, if it is available.

A SameAsConstraint is used to test whether the object passed as an actual value has the same identity as the object supplied in its constructor.

0.13.34.2 Constructor

SameAsConstraint( object expected )

<h4>Syntax</h4>

Is.SameAs( object expected )

<h4>Examples of Use</h4>

Exception ex1 = new Exception();
Exception ex2 = ex1;
Assert.That( ex2, Is.SameAs( ex1 ) );
Exception ex3 = new Exception();
Assert.That( ex3, Is.Not.SameAs( ex1 ) );

SamePathConstraint tests that two paths are equivalent.

<h4>Constructor</h4>

SamePathConstraint( string expectedPath )

<h4>Syntax</h4>

Is.SamePath( string expectedPath )

<h4>Modifiers</h4>

...IgnoreCase
...RespectCase

<h4>Examples of Use</h4>

Assert.That( "/folder1/./junk/../folder2", 
    Is.SamePath( "/folder1/folder2" ) );
Assert.That( "/folder1/./junk/../folder2/x", 
    Is.Not.SamePath( "/folder1/folder2" ) );

Assert.That( @"C:\folder1\folder2",
    Is.SamePath( @"C:\Folder1\Folder2" ).IgnoreCase );
Assert.That( "/folder1/folder2",
    Is.Not.SamePath( "/Folder1/Folder2" ).RespectCase );

0.13.34.3 Notes

  1. Path constraints perform tests on paths, without reference to any actual files or directories. This allows testing paths that are created by an application for reference or later use, without any effect on the environment.

  2. Path constraints are intended to work across multiple file systems, and convert paths to a canonical form before comparing them.

  3. It is usually not necessary to know the file system of the paths in order to compare them. Where necessary, the programmer may use the <b>IgnoreCase</b> and <b>RespectCase</b> modifiers to provide behavior other than the system default.

0.13.35 See also…

  • [[SubPathConstraint]]
  • [[SamePathOrUnderConstraint]] SamePathOrUnderConstraint tests that one path is equivalent to another path or that it is under it.

<h4>Constructor</h4>

SamePathOrUnderConstraint( string expectedPath )

<h4>Syntax</h4>

Is.SamePathOrUnder( string expectedPath )

<h4>Modifiers</h4>

...IgnoreCase
...RespectCase

<h4>Examples of Use</h4>

Assert.That( "/folder1/./junk/../folder2", 
    Is.SamePathOrUnder( "/folder1/folder2" ) );
Assert.That( "/folder1/junk/../folder2/./folder3",
    Is.SamePathOrUnder( "/folder1/folder2" ) );
Assert.That( "/folder1/junk/folder2/folder3",
    Is.Not.SamePathOrUnder( "/folder1/folder2" ) );

Assert.That( @"C:\folder1\folder2\folder3",
    Is.SamePathOrUnder( @"C:\Folder1\Folder2" ).IgnoreCase );
Assert.That( "/folder1/folder2/folder3",
    Is.Not.SamePathOrUnder( "/Folder1/Folder2" ).RespectCase );

0.13.35.1 Notes:

  1. Path constraints perform tests on paths, without reference to any actual files or directories. This allows testing paths that are created by an application for reference or later use, without any effect on the environment.

  2. Path constraints are intended to work across multiple file systems, and convert paths to a canonical form before comparing them.

  3. It is usually not necessary to know the file system of the paths in order to compare them. Where necessary, the programmer may use the <b>IgnoreCase</b> and <b>RespectCase</b> modifiers to provide behavior other than the system default.

0.13.35.2 See also…

  • [[SamePathConstraint]]
  • [[SubPathConstraint]]

Samples of NUnit usage are available in each of the languages listed below.

  • C# Samples
  • VB.NET Samples
  • F# Samples
  • C++ Samples <p>The <b>SequentialAttribute</b> is used on a test to specify that NUnit should generate test cases by selecting individual data items provided for the parameters of the test, without generating additional combinations.

<p><b>Note:</b> If parameter data is provided by multiple attributes, the order in which NUnit uses the data items is not guaranteed. However, it can be expected to remain constant for a given runtime and operating system. For best results with <b>SequentialAttribute</b> use only one data attribute on each parameter.

<h4>Example</h4>

<p>The following test will be executed three times.

[Test, Sequential]
public void MyTest(
    [Values(1,2,3)] int x,
    [Values("A","B")] string s)
{
    ...
}

<p>MyTest is called three times, as follows:

    MyTest(1, "A")
    MyTest(2, "B")
    MyTest(3, null)

<h4>See also…</h4> * [[Combinatorial Attribute]] * [[Pairwise Attribute]]

The SetCulture attribute is used to set the current Culture for the duration of a test. It may be specified at the level of a test or a fixture. The culture remains set until the test or fixture completes and is then reset to its original value. If you wish to use the current culture setting to decide whether to run a test, use the Culture attribute instead of this one.

Only one culture may be specified. Running a test under multiple cultures is a planned future enhancement. At this time, you can achieve the same result by factoring out your test code into a private method that is called by each individual test method.

<h4>Examples:</h4>

namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture]
  [SetCulture("fr-FR")]
  public class FrenchCultureTests
  {
    // ...
  }
}

<h4>See also…</h4> * [[Culture Attribute]]

The SetUICulture attribute is used to set the current UI Culture for the duration of a test. It may be specified at the level of a test or a fixture. The UI culture remains set until the test or fixture completes and is then reset to its original value. If you wish to use the current culture setting to decide whether to run a test, use the Culture attribute instead of this one.

Only one culture may be specified. Running a test under multiple cultures is a planned future enhancement. At this time, you can achieve the same result by factoring out your test code into a private method that is called by each individual test method.

<h4>Examples:</h4>

namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture]
  [SetUICulture("fr-FR")]
  public class FrenchCultureTests
  {
    // ...
  }
}

<h4>See also…</h4> * [[Culture Attribute]]] * [[SetCulture Attribute]]

This attribute is used inside a [[TestFixture|TestFixture-Attribute]] to provide a common set of functions that are performed just before each test method is called.

SetUp methods may be either static or instance methods and you may define more than one of them in a fixture. Normally, multiple SetUp methods are only defined at different levels of an inheritance hierarchy, as explained below.

If a SetUp method fails or throws an exception, the test is not executed and a failure or error is reported.

<h4>Example:</h4>

namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture]
  public class SuccessTests
  {
    [SetUp] public void Init()
    { /* ... */ }

    [TearDown] public void Cleanup()
    { /* ... */ }

    [Test] public void Add()
    { /* ... */ }
  }
}

<h4>Inheritance</h4>

The SetUp attribute is inherited from any base class. Therefore, if a base class has defined a SetUp method, that method will be called before each test method in the derived class.

You may define a SetUp method in the base class and another in the derived class. NUnit will call base class SetUp methods before those in the derived classes.

<h4>Notes:</h4>

  1. Although it is possible to define multiple SetUp methods in the same class, you should rarely do so. Unlike methods defined in separate classes in the inheritance hierarchy, the order in which they are executed is not guaranteed.

  2. SetUp methods may be async if running under .NET 4.0 or higher.

<h4>See also…</h4>

  • [[TearDown Attribute]]
  • [[OneTimeSetUp Attribute]]
  • [[OneTimeTearDown Attribute]]
  • [[TestFixture Attribute]]

This page describes significant changes in SetUp and TearDown in NUnit 3.0

Existing NUnit 2.6.4 attributes used for SetUp and TearDown were

  • SetUpAttribute
  • TearDownAttribute
  • TestFixtureSetUpAttribute
  • TestFixtureTearDownAttribute
  • SetUpFixtureAttribute

Taken together, these attributes provided per-test setup and teardown at the fixture level and one-time setup and teardown at the fixture, namespace and assembly levels.

These features were somewhat confusing:

  • SetUpFixture seems not very well understood by users in general.
  • TestFixtureSetUp and TestFixtureTearDown could do with better names.
  • SetUp and TearDown designate per-test setup/teardown within a test fixture, one-time setup/teardown within a setup fixture

For NUnit 3.0 we standardized the use of attributes for setup and teardown and renamed some of them to make their function clearer.

0.13.35.3 Attribute Usage

  • SetUpAttribute is now used exclusively for per-test setup.

  • TearDownAttribute is now used exclusively for per-test teardown.

  • OneTimeSetUpAttribute is used for one-time setup.

  • OneTimeTearDownAttribute is used for one-time teardown.

  • SetUpFixtureAttribute continues to be used as at before, but with changed method attributes.

0.13.35.4 Attribute Usage by Fixture Type

TestFixture SetUpFixture
OneTimeSetUp Supported Supported
OneTimeTearDown Supported Supported
TestFixtureSetUp Deprecated Not Allowed
TestFixtureTearDown Deprecated Not Allowed
SetUp Supported Not Allowed
TearDown Supported Not Allowed

0.13.35.5 Backward Compatibility

TestFixtureSetUpAttribute and TestFixtureTearDownAttribute continue to be supported as synonyms for OneTimeSetUpAttribute and OneTimeTearDownAttribute in test fixtures, but are deprecated.

Since SetUpAttribute and TearDownAttribute are used in two different ways, it’s not possible to simply deprecate their usage in SetUpFixture. They have been disallowed in that context, which is a [[breaking change|breaking-changes]].

0.13.35.6 How Setup and TearDown Methods Are Called

Multiple SetUp, OneTimeSetUp, TearDown and OneTimeTearDown methods may exist within a class. The rules for how the setup methods are called will be the same in NUnit 3.0 as in NUnit 2.6. However, there is a change in the calling of the teardown methods.

Setup methods (both types) are called on base classes first, then on derived classes. If any setup method throws an exception, no further setups are called. This is the same as in NUnit 2.6.

Teardown methods (again, both types) are called on derived classes first, then on the base class. In NUnit 2.6, all teardown methods were called so long as any setup method was called. It was entirely up to the teardown method to determine how much of the initialization took place.

In NUnit 3.0, the teardown methods at any level in the inheritance hierarchy will be called only if a setup method at the same level was called. The following example is illustrates the difference.

public class BaseClass
{
   [SetUp]
   public void BaseSetUp() { ... } // Exception thrown!

   [TearDown]
   public void BaseTearDown() { ... }
}

[TestFixture]
public class DerivedClass : BaseClass
{
   [SetUp]
   public void DerivedSetUp() { ... }

   [TearDown]
   public void DerivedTearDown() { ... }

   [Test]
   public void TestMethod() { ... }
}

Assume that an exception is thrown in BaseSetUp. In NUnit 2.6, methods would be executed as follows: * BaseSetUp * DerivedTearDown * BaseTearDown

In NUnit 3.0, execution will proceed as follows: * BaseSetUp * BaseTearDown

This is potentially a [[breaking change|breaking-changes]] for some users.

0.13.35.7 Unresolved Issues

  • We need to define how setup and teardown methods are ordered with respect to the newly introduced [[Action Attributes]] and how they interact. This is the attribute that marks a class that contains the one-time setup or teardown methods for all the test fixtures under a given namespace. The class may contain at most one method marked with the OneTimeSetUpAttribute and one method marked with the OneTimeTearDownAttribute.

There are a few restrictions on a class that is used as a setup fixture.

  • It must be a publicly exported type or NUnit will not see it.

  • It must have a default constructor or NUnit will not be able to construct it.

The OneTimeSetUp method in a SetUpFixture is executed once before any of the fixtures contained in its namespace. The OneTimeTearDown method is executed once after all the fixtures have completed execution. In the examples below, the method RunBeforeAnyTests() is called before any tests or setup methods in the NUnit.Tests namespace. The method RunAfterAnyTests() is called after all the tests in the namespace as well as their individual or fixture teardowns have completed execution.

Multiple SetUpFixtures may be created in a given namespace. The order of execution of such fixtures is indeterminate.

A SetUpFixture outside of any namespace provides SetUp and TearDown for the entire assembly.

0.13.35.8 Example:

using System;
using NUnit.Framework;

namespace NUnit.Tests
{
  [SetUpFixture]
  public class MySetUpClass
  {
    [OneTimeSetUp]
    public void RunBeforeAnyTests()
    {
      // ...
    }

    [OneTimeTearDown]
    public void RunAfterAnyTests()
    {
      // ...
    }
  }
}

Note: Prior to NUnit 3.0, SetUpFixture used the SetUp and TearDown attributes rather than OneTimeSetUp and OneTimeTearDown. The older attributes are no longer supported in SetUpFixutres in NUnit 3.0 and later.

0.13.35.9 See also…

  • [[SetUp Attribute]]
  • [[TearDown Attribute]]
  • [[OneTimeSetUp Attribute]]
  • [[OneTimeTearDown Attribute]]

SingleThreadedAttribute is used on a TestFixture and indicates that the OneTimeSetUp, OneTimeTearDown and all the child tests must run on the same thread.

SomeItemsConstraint applies a constraint to each item in an IEnumerable, succeeding if at least one of them succeeds. An exception is thrown if the actual value passed does not implement IEnumerable.

<h4>Constructor</h4>

SomeItemsConstraint(Constraint itemConstraint)

<h4>Syntax</h4>

Has.Some...

<h4>Examples of Use</h4>

int[] iarray = new int[] { 1, 2, 3 };
string[] sarray = new string[] { "a", "b", "c" };
Assert.That( iarray, Has.Some.GreaterThan(2) );
Assert.That( sarray, Has.Some.Length(1) );

StartsWithConstraint tests for an initial string.

<h4>Constructor</h4>

StartsWithConstraint(string expected)

<h4>Syntax</h4>

Does.StartWith(string expected)
StartsWith(string expected)

<h4>Modifiers</h4>

...IgnoreCase

<h4>Examples of Use</h4>

string phrase = "Make your tests fail before passing!"

Assert.That( phrase, Does.StartWith( "Make" ) );
Assert.That( phrase, Does.Not.StartWith( "Break" ) );
Expect( phrase, StartsWith( "Make" ) );

<h4>Notes</h4>

  1. StartsWith may appear only in the body of a constraint expression or when the inherited syntax is used.

The StringAssert class provides a number of methods that are useful when examining string values.

StringAssert.Contains( string expected, string actual );
StringAssert.Contains( string expected, string actual, 
                string message, params object[] args );

StringAssert.DoesNotContain( string expected, string actual );
StringAssert.DoesNotContain( string expected, string actual, 
                string message, params object[] args );

StringAssert.StartsWith( string expected, string actual );
StringAssert.StartsWith( string expected, string actual, 
                string message, params object[] args 
    
StringAssert.DoesNotStartsWith( string expected, string actual );
StringAssert.DoesNotStartsWith( string expected, string actual, 
                string message, params object[] args );

StringAssert.EndsWith( string expected, string actual );
StringAssert.EndsWith( string expected, string actual, 
                string message, params object[] args );
    
StringAssert.DoesNotEndWith( string expected, string actual );
StringAssert.DoesNotEndWith( string expected, string actual, 
                string message, params object[] args );

StringAssert.AreEqualIgnoringCase( string expected, string actual );
StringAssert.AreEqualIgnoringCase( string expected, string actual,
                string message params object[] args );    

StringAssert.AreNotEqualIgnoringCase( string expected, string actual );
StringAssert.AreNotEqualIgnoringCase( string expected, string actual,
                string message params object[] args );
                
StringAssert.IsMatch( string regexPattern, string actual );
StringAssert.IsMatch( string regexPattern, string actual, 
                string message, params object[] args );    
                
StringAssert.DoesNotMatch( string regexPattern, string actual );
StringAssert.DoesNotMatch( string regexPattern, string actual, 
                string message, params object[] args );

0.13.35.10 See also…

SubPathConstraint tests that one path is under another path.

<h4>Constructor</h4>

SubPathConstraint( string expectedPath )

<h4>Syntax</h4>

Is.SubPath( string expectedPath )

<h4>Modifiers</h4>

...IgnoreCase
...RespectCase

<h4>Examples of Use</h4>

Assert.That( "/folder1/./junk/../folder2", 
    Is.SubPath( "/folder1/folder2" ) );
Assert.That( "/folder1/junk/folder2",
    Is.Not.SubPath( "/folder1/folder2" ) );

Assert.That( @"C:\folder1\folder2\folder3",
    Is.SubPath( @"C:\Folder1\Folder2/Folder3" ).IgnoreCase );
Assert.That( "/folder1/folder2/folder3",
    Is.Not.SubPath( "/Folder1/Folder2/Folder3" ).RespectCase );

0.13.35.11 Notes:

  1. Path constraints perform tests on paths, without reference to any actual files or directories. This allows testing paths that are created by an application for reference or later use, without any effect on the environment.

  2. Path constraints are intended to work across multiple file systems, and convert paths to a canonical form before comparing them.

  3. It is usually not necessary to know the file system of the paths in order to compare them. Where necessary, the programmer may use the <b>IgnoreCase</b> and <b>RespectCase</b> modifiers to provide behavior other than the system default.

0.13.35.12 See also…

  • [[SamePathConstraint]]
  • [[SamePathOrUnderConstraint]]

SubstringConstraint tests for a substring.

<h4>Constructor</h4>

SubstringConstraint(string expected)

<h4>Syntax</h4>

Does.Contain(string expected)

<h4>Modifiers</h4>

...IgnoreCase

<h4>Examples of Use</h4>

string phrase = "Make your tests fail before passing!"

Assert.That( phrase, Does.Contain( "tests fail" ) );
Assert.That( phrase, Does.Not.Contain( "tests pass" ) );
Assert.That( phrase, Does.Contain( "make" ).IgnoreCase );

0.13.36 UNDER CONSTRUCTION

NUnit 3.0 is based on NUnit version 2, but with substantial redesign and a new codebase. This page summarizes what’s different between NUnit 2.6.4, the last version 2 release, and NUnit 3.0.

0.14 General Changes

  • The previous NUnitLite framework, released as version 1.0, is now merged with the NUnit framework.

  • The license for NUnit is now MIT / X11.

0.15 Framework

0.15.1 Platforms

  • The NUnit framework is now built for a number of different platforms:
  • .NET 4.5 (also used for runtimes greater than 4.5)
  • .Net 4.0
  • .NET 2.0
  • Portable Class Library supporting
    • .NET 4.5+
    • .NET Core (Universal Windows Apps 10+, DNX Core 5+)
    • Windows 8
    • Windows Phone 8 (Silverlight)
    • Universal (Windows Phone 8.1+, Windows 8.1+),
    • Xamarin (MonoTouch, MonoAndroid, Xamarin iOS Universal)
  • Silverlight 5.0
  • .NET Compact Framework 3.5

  • The Silverlight and Compact Framework builds are separate downloads for those who need them.

  • Users targeting .NET 4.0 who want to write async tests will need to reference the Microsoft.Bcl.Async NuGet package.

0.15.2 Attributes

  • The following new attributes have been added:
  • TestFixtureSourceAttribute is equivalent to TestCaseSource for fixtures.
  • RetryAttribute allows retrying of failing tests.
  • LevelOfParallelismAttribute and ParallelizableAttribute support test execution in parallel.
  • ApartmentAttribute is used to specify the apartment in which a test should be run.

  • The following attributes have been modified:
  • TestCaseAttribute and TestCaseData now allow modifying the test name without replacing it entirely.
  • The RangeAttribute has been extended to support more data types including uint, long and ulong
  • The named members of the TestCaseSource and ValueSource attributes must now be static.
  • RandomAttribute has been extended to add support for new data types including uint, long, ulong, short, ushort, float, byte and sbyte
  • TestCaseAttribute now allows arguments with default values to be omitted. Additionaly, it accepts IncludePlatform and ExcludePlatform properties to specify the platforms on which the test case should be run.
  • TestFixture and TestCase attributes now enforce the requirement that a reason needs to be provided when ignoring a test.
  • SetUp, TearDown, OneTimeSetUp and OneTimeTearDown methods may now be async.
  • Multiple SetUpFixtures may be specified in a single namespace.
  • ActionAttributes now allow specification of multiple targets on the attribute as designed. This didn’t work in the first alpha.
  • Action Attributes have been added with the same features as in NUnit 2.6.3.

  • Parameterized test cases now support nullable arguments.
  • Tests skipped due to ExplicitAttribute are now reported as skipped.
  • Significant improvements have been made in how NUnit deduces the type arguments of generic methods based on the arguments provided.
  • The ExpectedExceptionAttribute is no longer supported. Use Assert.Throws() or Assert.That(…, Throws) instead for a more precise specification of where the exception is expected to be thrown.
  • ExpectedResult is now supported on simple (non-TestCase) tests.
  • The Ignore attribute now takes a named parameter Until, which allows specifying a date after which the test is no longer ignored.
  • The following new values are now recognized by PlatformAttribute: Win7, Win8, Win8.1, Win2012Server, Win2012ServerR2, NT6.1, NT6.2, 32-bit, 64-bit
  • ValuesAttribute may be used without any values on an enum or boolean argument. All possible values are used.

0.15.3 Asserts

  • Unnecessary overloads of Assert.That and Assume.That have been removed.
  • Assert.NullOrEmpty is no longer supported (Use Is.Null.Or.Empty)
  • FileAssert.Exists and FileAssert.DoesNotExist

0.15.4 Constraints

  • New SupersetConstraint and Is.SupersetOf syntax complement SubsetConstraint.
  • Added Throws.ArgumentNullException
  • When checking the equality of DateTimeOffset, you can now use the WithSameOffset modifier
  • A new Constraint, <b>DictionaryContainsKeyConstraint</b>, may be used to test that a specified key is present in a dictionary.
  • Does prefix operator supplies several added constraints.
  • A new FileExistsConstraint has been added.
  • You may now specify a tolerance using Within when testing equality of DateTimeOffset values.

0.15.5 Other

  • TestContext.Random has also been extended to add support for new data types including uint, long, ulong, short, ushort, float, byte, sbyte and decimal
  • Added GetString methods to NUnit.Framework.Internal.RandomGenerator to create repeatable random strings for testing
  • Some classes intended for internal usage that were public for testing have now been made internal. Additional classes will be made internal for the final 3.0 release.
  • The class and method names of each test are included in the output xml where applicable.
  • String arguments used in test case names are now truncated to 40 rather than 20 characters.
  • String arguments over 20 characters in length are truncated when used as part of a test name.
  • The Pairwise strategy test case generation algorithm has been improved.
  • All addin support has been removed from the framework. Documentation of NUnit 3.0 extensibility features will be published in time for the beta release. In the interim, please ask for support on the nunit-discuss list.
  • Legacy suites are no longer supported
  • TestContext now has a method that allows writing to the XML output.
  • TestContext.CurrentContext.Result now provides the error message and stack trace during teardown.
  • NUnit no longer supports void async test methods. You should use a Task return Type instead.
  • Parallel test execution is supported down to the Fixture level. Use ParallelizableAttribute to indicate types that may be run in parallel.
  • Async tests are supported for .NET 4.0 if the user has installed support for them.
  • The XML output now includes a start and end time for each test.
  • TestContext is now supported and includes an additional property, <b>Random</b>, which may be used to generate repeatable random values for use in a test.
  • The external framework API is now stable; internal interfaces are separate from API
  • Tests may be run in parallel on separate threads
  • Created new API for controlling framework
  • Support for old style tests has been removed

0.16 NUnitLite

  • The NUnitLite report output has been standardized to match that of nunit-console.
  • The NUnitLite command-line has been standardized to match that of nunit-console where they share the same options.
  • The NUnitLite runner now produces the same output display and XML results as the console runner.
  • NUnitLite tests must reference both the nunit.framework and nunitlite assemblies.
  • NUnitLite code is now merged with NUnit
  • Added NUnitLite runner to the framework code
  • The Silverlight runner now displays output in color and includes any text output created by the tests.
  • The NUnit and NUnitLite frameworks have now been merged. There is no longer any distinction between them in terms of features, although some features are not available on all platforms.

0.17 Engine

  • The format of the XML result file has been finalized and documented.
  • The engine now runs multiple test assemblies in parallel by default
  • The output XML now includes more information about the test run, including the text of the command used, any engine settings and the filter used to select tests.
  • Extensions may now specify data in an identifying attribute, for use by the engine in deciding whether to load that extension.
  • We now use Cecil to examine assemblies prior to loading them.
  • Extensions are no longer based on Mono.Addins but use our own extension framework.
  • If the target framework is not specified, test assemblies that are compiled to target .NET 4.5 will no longer run in .NET 4.0 compatibility mode
  • The engine API has now been finalized. It permits specifying a minimum version of the engine that a runner is able to use. The best installed version of the engine will be loaded. Third-party runners may override the selection process by including a copy of the engine in their installation directory and specifying that it must be used.
  • The V2 framework driver now uses the event listener and test listener passed to it by the runner. This corrects several outstanding issues caused by events not being received and allows selecting V2 tests to be run from the command-line, in the same way that V3 tests are selected.
  • The engine is now extensible using Mono.Addins. In this release, extension points are provided for FrameworkDrivers, ProjectLoaders and OutputWriters. The following addins are bundled as a part of NUnit:
  • A FrameworkDriver that allows running NUnit V2 tests under NUnit 3.0.
  • ProjectLoaders for NUnit and Visual Studio projects.
  • An OutputWriter that creates XML output in NUnit V2 format.
  • DomainUsage now defaults to Multiple if not specified by the runner
  • A driver is now included, which allows running NUnit 2.x tests under NUnit 3.0.
  • The engine can now load and run tests specified in a number of project formats:
  • NUnit (.nunit)
  • Visual Studio C# projects (.csproj)
  • Visual Studio F# projects (.vjsproj)
  • Visual Studio Visual Basic projects (.vbproj)
  • Visual Studio solutions (.sln)
  • Legacy C++ and Visual JScript projects (.csproj and .vjsproj) are also supported
  • Support for the current C++ format (.csxproj) is not yet available
  • Creation of output files like TestResult.xml in various formats is now a service of the engine, available to any runner.
  • The logic of runtime selection has now changed so that each assembly runs by default in a separate process using the runtime for which it was built.
  • On 64-bit systems, each test process is automatically created as 32-bit or 64-bit, depending on the platform specified for the test assembly.

0.18 Console Runner

NOTE: The nunit3-console runner cannot run tests that reference the portable build. You may run such tests using NUnitLite or a platform-specific runner. * The console runner program is now called nunit3-console. * Console runner output has been modified so that the summary comes at the end, to reduce the need for scrolling. * The console now displays all settings used by the engine to run tests as well as the filter used to select tests. * The console runner accepts a new option –maxagents. If multiple assemblies are run in separate processes, this value may be used to limit the number that are executed simultaneously in parallel. * The console runner no longer accepts the –include and –exclude options. Instead, the new –where option provides a more general way to express which tests will be executed, such as –where “cat==Fast && Priority==High”. See the docs for details of the syntax. * The new –debug option causes NUnit to break in the debugger immediately before tests are run. This simplifies debugging, especially when the test is run in a separate process. * OneTimeSetUp and OneTimeTearDown failures are now shown on the test report. Individual test failures after OneTimeSetUp failure are no longer shown. * If the console is run without arguments, it will now display help * The console now defaults to not using shadowcopy. There is a new option –shadowcopy to turn it on if needed. * New options supported: * –testlist provides a list of tests to run in a file * –stoponerror indicates that the run should terminate when any test fails. * The command-line may now include any number of assemblies and/or supported projects. * The console runner now runs tests in a separate process per assembly by default. They may still be run in process or in a single separate process by use of command-line options. * The console runner now starts in the highest version of the .NET runtime available, making it simpler to debug tests by specifying that they should run in-process on the command-line. * The -x86 command-line option is provided to force execution in a 32-bit process on a 64-bit system. * A writeability check is performed for each output result file before trying to run the tests. * The -teamcity option is now supported. * The console runner no longer displays test results in the debugger. * The console runner now automatically detects 32- versus 64-bit test assemblies. * Both nunit-console and NUnitLite now display output in color. * The console runner refuses to run tests build with older versions of NUnit. A plugin will be available to run older tests in the future.

0.19 Issues Resolved

0.19.0.1 Github Issues

  • 6 Log4net not working with NUnit
  • 8 [SetUpFixture] is not working as expected
  • 12 Compact framework should support generic methods
  • 14 CI Server for NUnit Framework
  • 13 Standardize commandline options for nunitlite runner
  • 17 No allowance is currently made for nullable arguents in TestCase parameter conversions
  • 20 TestCaseAttribute needs Platform property.
  • 21 Is.InRange Constraint Ambiguity
  • 22 Add OSArchitecture Attribute to Environment node in result xml
  • 24 Assert on Dictionary Content
  • 27 Values attribute support for enum types
  • 29 Specifying a tolerance with “Within” doesn’t work for DateTimeOffset data types
  • 31 Report start and end time of test execution
  • 33 TestCaseSource cannot refer to a parameterized test fixture
  • 36 Make RequiresThread, RequiresSTA, RequiresMTA inheritable
  • 37 Multiple SetUpFixtures should be permitted on same namespace
  • 41 Check for zeroes in Assert messages
  • 45 Need of Enddate together with Ignore
  • 47 Extensions to RangeAttribute
  • 48 Explicit seems to conflict with Ignore
  • 54 Store message and stack trace in TestContext for use in TearDown
  • 55 Incorrect XML comments for CollectionAssert.IsSubsetOf
  • 59 Length of generated test names should be limited
  • 60 NUnit should support async setup, teardown, fixture setup and fixture teardown.
  • 62 Matches(Constraint) does not work as expected
  • 63 Async support should handle Task return type without state machine
  • 64 AsyncStateMachineAttribute should only be checked by name
  • 65 Update NUnit Wiki to show the new location of samples
  • 66 Parallel Test Execution within test assemblies
  • 67 Allow Expected Result on simple tests
  • 68 Customization of test case name generation
  • 70 EquivalentTo isn’t compatible with IgnoreCase for dictioneries
  • 75 Async tests should be supported for projects that target .NET 4.0
  • 82 nunit-framework tests are timing out on Linux
  • 83 Path-related tests fail on Linux
  • 85 Culture-dependent NUnit tests fail on non-English machine
  • 88 TestCaseSourceAttribute documentation
  • 90 EquivalentTo isn’t compatible with IgnoreCase for char
  • 100 Changes to Tolerance definitions
  • 110 Add new platforms to PlatformAttribute
  • 111 Implement Changes to File, Directory and Path Assertions
  • 112 Implement Action Attributes
  • 113 Remove ExpectedException
  • 118 Workarounds for missing InternalPreserveStackTrace in mono
  • 121 Test harness does not honor the –worker option when set to zero
  • 125 3rd-party dependencies should be downloaded on demand
  • 129 Standardize Timeout in the Silverlight build
  • 130 Add FileAssert.Exists and FileAssert.DoesNotExist
  • 132 Drop support for void async methods
  • 145 NUnit-console fails if test result message contains invalid xml characters
  • 153 Surprising behavior of DelayedConstraint pollingInterval
  • 155 Create utility classes for platform-specific code
  • 156 Accessing multiple AppDomains within unit tests result in SerializationException
  • 161 Update API to support stopping an ongoing test run
  • 163 Add –trace option to NUnitLite
  • 167 Create interim documentation for the alpha release
  • 168 Create NUnit 3.0 documentation
  • 169 Design and implement distribution of NUnit packages
  • 171 Assert.That should work with any lambda returning bool
  • 175 Test Harness should return an error if any tests fail
  • 180 Errors in Linux CI build
  • 181 Replace NAnt with MsBuild / XBuild
  • 183 Standardize commandline options for test harness
  • 188 No output from NUnitLite when selected test is not found
  • 189 Add string operators to Does prefix
  • 193 TestWorkerTests.BusyExecutedIdleEventsCalledInSequence fails occasionally
  • 196 Compare DateTimeOffsets including the offset in the comparison
  • 197 Deprecate or remove Assert.NullOrEmpty
  • 202 Eliminate legacy suites
  • 203 Combine framework, engine and console runner in a single solution and repository
  • 209 Make Ignore attribute’s reason mandatory
  • 210 TestContext.WriteLine in an AppDomain causes an error
  • 215 Running 32-bit tests on a 64-bit OS
  • 217 New icon for the 3.0 release
  • 219 Teardown failures are not reported
  • 222 Color console for NUnitLite
  • 223 Common code for NUnitLite console runner and NUnit-Console
  • 224 Silverlight Support
  • 225 Compact Framework Support
  • 227 Add support for VS projects and solutions
  • 229 Timing failures in tests
  • 231 Update C# samples to use NUnit 3.0
  • 233 Update F# samples to use NUnit 3.0
  • 234 Update C++ samples to use NUnit 3.0
  • 237 System.Uri .ctor works not properly under Nunit
  • 238 Improvements to running 32 bit tests on a 64 bit system
  • 241 Remove reference to Microslft BCL packages
  • 243 Create solution for Linux
  • 244 NUnit should properly distinguish between .NET 4.0 and 4.5
  • 245 Multiple targets on action attributes not implemented
  • 246 C++ tests do not compile in VS2013
  • 247 Eliminate trace display when running tests in debug
  • 254 Finalize XML format for test results
  • 255 Add new result states for more precision in where failures occur
  • 256 ContainsConstraint break when used with AndConstraint
  • 257 TestCaseAttribute should not require parameters with default values to be specified.
  • 261 Add portable nunitlite build
  • 264 Stacktrace displays too many entries
  • 265 Reorganize console reports for nunit-console and nunitlite
  • 266 Pluggable framework drivers.
  • 269 Add manifest to nunit-console and nunit-agent
  • 270 OneTimeSetUp failure results in too much output
  • 271 Invalid tests should be treated as errors
  • 274 Command line options should be case insensitive
  • 275 NUnitEqualityComparer fails to compare IEquatable<T> where second object is derived from T
  • 276 NUnit-console should not reference nunit.framework
  • 278 New result states (ChildFailure and SetupFailure) break NUnit2XmlOutputWriter
  • 282 Get tests for NUnit2XmlOutputWriter working
  • 283 What should we do when a user extension does something bad?
  • 284 NUnitLite Unification
  • 288 Set up Appveyor CI build
  • 290 Stack trace still displays too many items
  • 293 CF does not have a CurrentDirectory
  • 299 No full path to assembly in XML file under Compact Framework
  • 301 Command-line length
  • 304 Run test Assemblies in parallel
  • 306 Assure NUnit can write resultfile
  • 308 Early disposal of runners
  • 309 NUnit-Console should support incremental output under TeamCity
  • 310 Target framework not specified on the AppDomain when running against .Net 4.5
  • 315 NUnit 3.0 alpha: Cannot run in console on my assembly
  • 316 NUnitLite TextUI Runner
  • 318 TestActionAttribute: Retrieving the TestFixture
  • 319 CI builds are not treating test failures as failures of the build
  • 320 No Tests found: Using parametrized Fixture and TestCaseSource
  • 321 Rationalize how we count tests
  • 322 Remove Stopwatch tests where they test the real .NET Stopwatch
  • 325 Add RegexConstraint to compact framework build
  • 326 Add TimeoutAttribute to compact framework build
  • 327 Allow generic test methods in the compact framework
  • 328 Use .NET Stopwatch class for compact framework builds
  • 331 Alpha 2 CF does not build
  • 333 Add parallel execution to desktop builds of NUnitLite
  • 334 Include File-related constraints and syntax in NUnitLite builds
  • 335 Re-introduce ‘Classic’ NUnit syntax in NUnitLite
  • 336 Document use of separate obj directories per build in our projects
  • 337 Update Standard Defines page for .NET 3.0
  • 341 Move the NUnitLite runners to separate assemblies
  • 360 Better exception message when using non-BCL class in property
  • 363 Make Xml result output an engine service
  • 367 Refactor XML Escaping Tests
  • 368 Create addin model.
  • 369 Project loader addins
  • 370 OutputWriter addins
  • 372 CF Build TestAsesemblyRunnerTests
  • 373 Minor CF Test Fixes
  • 374 New syntax for selecting tests to be run
  • 377 CombiningStrategyAttributes don’t work correctly on generic methods
  • 378 Correct documentation for PairwiseAttribute
  • 386 Console Output Improvements
  • 388 Improvements to NUnitLite runner output
  • 390 Specify exactly what happens when a test times out
  • 396 ApartmentAttribute
  • 397 CF nunitlite runner assembly has the wrong name
  • 403 Move ConsoleOptions.cs and Options.cs to Common and share…
  • 404 Split tests between nunitlite.runner and nunit.framework
  • 407 Assert.Pass() with ]]> in message crashes console runner
  • 414 Simplify Assert overloads
  • 416 NUnit 2.x Framework Driver
  • 417 Complete work on NUnit projects
  • 419 Create Windows Installer for NUnit.
  • 420 Create Settings file in proper location
  • 427 [TestFixture(Ignore=true)] should not be allowed.
  • 428 Add ExpectedExceptionAttribute to C# samples
  • 437 Errors in tests under Linux due to hard-coded paths.
  • 440 Automatic selection of Test Engine to use
  • 441 NUnit-Console should support –testlist option
  • 442 Add –stoponerror option back to nunit-console.
  • 450 Create master install that includes the framework, engine and console installs
  • 454 Rare registry configurations may cause NUnit to fail
  • 456 Fix memory leak in RuntimeFramework.
  • 459 Remove the Mixed Platforms build configuration.
  • 468 Change default domain usage to multiple.
  • 469 Truncate string arguments in test names in order to limit the length.
  • 472 Overflow exception and DivideByZero exception from the RangeAttribute
  • 477 Assert does not work with ArraySegment
  • 478 RepeatAttribute
  • 481 Testing multiple assemblies in nunitlite
  • 482 nunit-console has multiple errors related to -framework option
  • 483 Adds constraint for asserting that a dictionary contains a particular key
  • 484 Missing file in NUnit.Console nuget package
  • 485 Can’t run v2 tests with nunit-console 3.0
  • 487 NUnitLite can’t load assemblies by their file name
  • 488 Async setup and teardown still don’t work
  • 497 Framework installer shold register the portable framework
  • 504 Option –workers:0 is ignored
  • 508 Travis builds with failure in engine tests show as successful
  • 509 Under linux, not all mono profiles are listed as available
  • 512 Drop the .NET 3.5 build
  • 515 OSPlatform.IsMacOSX doesn’t work
  • 517 V2 FrameworkDriver does not make use of passed in TestEventListener
  • 523 Provide an option to disable shadowcopy in NUnit v3
  • 524 int and char do not compare correctly?
  • 528 V2 FrameworkDriver does not make use of passed in TestFilter
  • 530 Color display for Silverlight runner
  • 531 Display text output from tests in Silverlight runner
  • 534 Add classname and methodname to test result xml
  • 538 Potential bug using TestContext in constructors
  • 539 Truncation of string arguments
  • 541 Console help doesn’t indicate defaults
  • 544 AsyncTestMethodTests for 4.5 Framework fails frequently on Travis CI
  • 546 Enable Parallel in NUnitLite/CF (or more) builds
  • 551 TextRunner not passing the NumWorkers option to the ITestAssemblyRunner
  • 556 Executed tests should always return a non-zero duration
  • 559 Fix text of NuGet packages
  • 560 Fix PackageVersion property on wix install projects
  • 562 Program.cs in NUnitLite NuGet package is incorrect
  • 564 NUnitLite Nuget package is Beta 1a, Framework is Beta 1
  • 565 NUnitLite Nuget package adds Program.cs to a VB Project
  • 568 Isolate packaging from building
  • 570 ThrowsConstraint failure message should include stack trace of actual exception
  • 573 nunit-console hangs on Mac OS X after all tests have run
  • 575 Add support for ASP.NET 5 and the new Core CLR
  • 576 Throws.ArgumentNullException would be nice
  • 577 Documentation on some members of Throws falsely claims that they return TargetInvocationException constraints
  • 579 No documentation for recommended usage of TestCaseSourceAttribute
  • 580 TeamCity Service Message Uses Incorrect Test Name with NUnit2Driver
  • 582 Test Ids Are Not Unique
  • 583 TeamCity service messages to support parallel test execution
  • 584 Non-runnable assembly has incorrect ResultState
  • 585 RetryAttribute
  • 609 Add support for integration with TeamCity
  • 611 Remove unused –teamcity option from CF build of NUnitLite
  • 612 MaxTime doesn’t work when used for TestCase
  • 621 Core Engine
  • 622 nunit-console fails when use –output
  • 628 Modify IService interface and simplify ServiceContext
  • 631 Separate packaging for the compact framework
  • 642 Restructure MSBuild script
  • 646 ConfigurationManager.AppSettings Params Return Null under Beta 1
  • 649 Change how we zip packages
  • 654 ReflectionOnlyLoad and ReflectionOnlyLoadFrom
  • 656 Unused parameter in Console.WriteLine found
  • 664 Invalid “id” attribute in the report for case “test started”
  • 669 TeamCity service message should have assembly name as a part of test name.
  • 670 Failing Tests in TeamCity Build
  • 673 Ensure proper disposal of engine objects
  • 674 Engine does not release test assemblies
  • 679 Windows 10 Support
  • 682 Add Async Support to Portable Framework
  • 683 Make FrameworkController available in portable build
  • 685 In the some cases when tests cannot be started NUnit returns exit code “0”
  • 687 TestAgency does not launch agent process correctly if runtime type is not specified (i.e. v4.0)
  • 689 The TeamCity service message “testFinished” should have an integer value in the “duration” attribute
  • 692 PlatformAttribute_OperatingSystemBitNess fails when running in 32-bit process
  • 693 Generic Test<T> Method cannot determine type arguments for fixture when passed as IEnumerable<T>
  • 698 Require TestCaseSource and ValueSource named members to be static
  • 703 TeamCity non-equal flowid for ‘testStarted’ and ‘testFinished’ messages
  • 712 Extensions to RandomAttribute
  • 713 Include command information in XML
  • 715 Provide a data source attribute at TestFixture Level
  • 718 RangeConstraint gives error with from and two args of differing types
  • 719 We have no way to configure tests for several assemblies using NUnit project file and the common installation from msi file
  • 723 Does nunit.nuspec require dependency on Microsoft.Bcl.Async?
  • 724 Adds support for Nullable<bool> to Assert.IsTrue and Assert.IsFalse
  • 728 Missing Assert.That overload
  • 734 Console without parameters doesn’t show help
  • 735 Workers number in xml report file cannot be found
  • 741 Explicit Tests get run when using –exclude
  • 746 Framework should send events for all tests
  • 747 NUnit should apply attributes even if test is non-runnable
  • 749 Review Use of Mono.Addins for Engine Extensibility
  • 750 Include Explicit Tests in Test Results
  • 753 Feature request: Is.SupersetOf() assertion constraint
  • 755 TimeOut attribute doesn’t work with TestCaseSource Attribute
  • 757 Implement some way to wait for execution to complete in ITestEngineRunner
  • 760 Packaging targets do not run on Linux
  • 766 Added overloads for True()/False() accepting booleans
  • 778 Build and build.cmd scripts invoke nuget.exe improperly
  • 780 Teamcity fix
  • 782 No sources for 2.6.4
  • 783 Package separately for Silverlight
  • 784 Build Portable Framework on Linux
  • 790 Allow Extensions to provide data through an attribute
  • 794 Make it easier to debug tests as well as NUnit itself
  • 801 NUnit calls Dispose multiple times
  • 814 Support nullable types with TestCase
  • 818 Possible error in Merge Pull Request #797
  • 821 Wrapped method results in loss of result information
  • 822 Test for Debugger in NUnitTestAssemblyRunner probably should not be in CF build
  • 824 Remove unused System.Reflection using statements
  • 826 Randomizer uniqueness tests fail randomly!
  • 828 Merge pull request #827 (issue 826)
  • 830 Add ability to report test results synchronously to test runners
  • 833 Intermittent failure of WorkItemQueueTests.StopQueue_WithWorkers
  • 837 Enumerators not disposed when comparing IEnumerables
  • 840 Add missing copyright notices
  • 844 Pull Request #835 (Issue #814) does not build in CF
  • 847 Add new –process:inprocess and –inprocess options
  • 850 Test runner fails if test name contains invalid xml characters
  • 851 ‘Exclude’ console option is not working in NUnit Lite
  • 853 Cannot run NUnit Console from another directory
  • 859 NUnit-Console output - move Test Run Summary to end
  • 860 Use CDATA section for message, stack-trace and output elements of XML
  • 863 Eliminate core engine
  • 865 Intermittent failures of StopWatchTests
  • 867 Remove Warnings from Ignored tests
  • 868 Review skipped tests
  • 869 Tests that use directory separator char to determine platform misreport Linux on MaxOSX
  • 870 NUnit Console Runtime Environment misreports on MacOSX
  • 874 Add .NET Core Framework
  • 878 Cannot exclude MacOSX or XBox platforms when running on CF
  • 887 Move environment and settings elements to the assembly suite in the result file
  • 892 Fixed test runner returning early when executing more than one test run.
  • 894 Give nunit.engine and nunit.engine.api assemblies strong names
  • 896 NUnit 3.0 console runner not placing test result xml in –work directory
  • 899 Colors for ColorConsole on grey background are too light
  • 904 InternalPreserveStackTrace is not supported on all Portable platforms
  • 914 Unclear error message from console runner when assembly has no tests
  • 916 Console runner dies when test agent dies
  • 918 Console runner –where parameter is case sensitive
  • 920 Remove addins.engine.api.dll from NuGet package
  • 929 Rename nunit-console.exe
  • 931 Remove beta warnings from NuGet packages
  • 936 Explicit skipped tests not displayed
  • 939 Installer complains about .NET even if already installed
  • 940 Confirm or modify list of packages for release
  • 947 Breaking API change in ValueSourceAttribute
  • 949 Update copyright in NUnit Console
  • 954 NUnitLite XML output is not consistent with the engine’s
  • 955 NUnitLite does not display the where clause
  • 959 Restore filter options for NUnitLite portable build
  • 960 Intermittent failure of CategoryFilterTests
  • 967 Run Settings Report is not being displayed.

0.19.0.2 GitHub Issues from former nunit-console repository

  • 2 Failure in TestFixtureSetUp is not reported correctly
  • 5 CI Server for nunit-console
  • 6 System.NullReferenceException on start nunit-console-x86
  • 21 NUnitFrameworkDriverTests fail if not run from same directory
  • 24 ‘Debug’ value for /trace option is deprecated in 2.6.3
  • 38 Confusing Excluded categories output

0.19.0.3 Launchpad Bugs

  • 400502 NUnitEqualityComparer.StreamsE­qual fails for same stream
  • 400508 TestCaseSource attirbute is not working when Type is given
  • 400510 TestCaseData variable length ctor drops values
  • 417557 Add SetUICultureAttribute from NUnit 2.5.2
  • 417559 Add Ignore to TestFixture, TestCase and TestCaseData
  • 417560 Merge Assert.Throws and Assert.Catch changes from NUnit 2.5.2
  • 417564 TimeoutAttribute on Assembly
  • 419411 Fixture With No Tests Shows as Non-Runnable
  • 430100 Assert.Catch<T> should return T
  • 432566 NUnitLite shows empty string as argument
  • 432573 Mono test should be at runtime
  • 432805 Some Framework Tests don’t run on Linux
  • 440109 Full Framework does not support “Contains”
  • 459219 Changes to thread princpal cause failures under .NET 4.0
  • 459224 Culture test failure under .NET 4.0
  • 462019 Line endings needs to be better controlled in source
  • 462418 Assume.That() fails if I specify a message
  • 463470 We should encapsulate references to pre-2.0 collections
  • 483836 Allow non-public test fixtures consistently
  • 483845 TestCase expected return value cannot be null
  • 487878 Tests in generic class without proper TestFixture attribute should be invalid
  • 488002 Should not report tests in abstract class as invalid
  • 490679 Category in TestCaseData clashes with Category on ParameterizedMethodSuite
  • 498656 TestCase should show array values in GUI
  • 498690 Assert.That() doesn’t like properties with scoped setters
  • 501352 VS2010 projects have not been updated for new directory structure
  • 501784 Theory tests do not work correctly when using null parameters
  • 504018 Automatic Values For Theory Test Parameters Not Provided For bool And enum
  • 505899 ‘Description’ parameter in both TestAttribute and TestCaseAttribute is not allowed
  • 513989 Is.Empty should work for directories
  • 519912 Thread.CurrentPrincipal Set In TestFixtureSetUp Not Maintained Between Tests
  • 523335 TestFixtureTearDown in static class not executed
  • 531873 Feature: Extraction of unit tests from NUnit test assembly and calling appropriate one
  • 532488 constraints from ConstraintExpression/ConstraintBuilder are not reusable
  • 556971 Datapoint(s)Attribute should work on IEnumerable<T> as well as on Arrays
  • 561436 SetCulture broken with 2.5.4
  • 563532 DatapointsAttribute should be allowed on properties and methods
  • 590717 categorie contains dash or trail spaces is not selectable
  • 590970 static TestFixtureSetUp/TestFixtureTearDown methods in base classes are not run
  • 595683 NUnit console runner fails to load assemblies
  • 600627 Assertion message formatted poorly by PropertyConstraint
  • 601108 Duplicate test using abstract test fixtures
  • 601645 Parametered test should try to convert data type from source to parameter
  • 605432 ToString not working properly for some properties
  • 606548 Deprecate Directory Assert in 2.5 and remove it in 3.0
  • 608875 NUnit Equality Comparer incorrectly defines equality for Dictionary objects
  • 611325 Allow Teardown to detect if last test failed
  • 611938 Generic Test Instances disappear
  • 655882 Make CategoryAttribute inherited
  • 664081 Add Server2008 R2 and Windows 7 to PlatformAttribute
  • 671432 Upgrade NAnt to Latest Release
  • 676560 Assert.AreEqual does not support IEquatable<T>
  • 691129 Add Category parameter to TestFixture
  • 697069 Feature request: dynamic location for TestResult.xml
  • 708173 NUnit’s logic for comparing arrays - use Comparer<T[]> if it is provided
  • 709062 “System.ArgumentException : Cannot compare” when the element is a list
  • 712156 Tests cannot use AppDomain.SetPrincipalPolicy
  • 719184 Platformdependency in src/ClientUtilities/util/Services/DomainManager.cs:40
  • 719187 Using Path.GetTempPath() causes conflicts in shared temporary folders
  • 735851 Add detection of 3.0, 3.5 and 4.0 frameworks to PlatformAttribute
  • 736062 Deadlock when EventListener performs a Trace call + EventPump synchronisation
  • 756843 Failing assertion does not show non-linear tolerance mode
  • 766749 net-2.0-console-x86.exe.config should have a <startup /> element and also enable loadFromRemoteSources
  • 770471 Assert.IsEmpty does not support IEnumerable
  • 785460 Add Category parameter to TestCaseSourceAttribute
  • 787106 EqualConstraint provides inadequate failure information for IEnumerables
  • 792466 TestContext MethodName
  • 794115 HashSet incorrectly reported
  • 800089 Assert.Throws() hides details of inner AssertionException
  • 848713 Feature request: Add switch for console to break on any test case error
  • 878376 Add ‘Exactly(n)’ to the NUnit constraint syntax
  • 882137 When no tests are run, higher level suites display as Inconclusive
  • 882517 NUnit 2.5.10 doesn’t recognize TestFixture if there are only TestCaseSource inside
  • 885173 Tests are still executed after cancellation by user
  • 885277 Exception when project calls for a runtime using only 2 digits
  • 885604 Feature request: Explicit named parameter to TestCaseAttribute
  • 890129 DelayedConstraint doesn’t appear to poll properties of objects
  • 892844 Not using Mono 4.0 profile under Windows
  • 893919 DelayedConstraint fails polling properties on references which are initially null
  • 896973 Console output lines are run together under Linux
  • 897289 Is.Empty constraint has unclear failure message
  • 898192 Feature Request: Is.Negative, Is.Positive
  • 898256 IEnumerable<T> for Datapoints doesn’t work
  • 899178 Wrong failure message for parameterized tests that expect exceptions
  • 904841 After exiting for timeout the teardown method is not executed
  • 908829 TestCase attribute does not play well with variadic test functions
  • 910218 NUnit should add a trailing separator to the ApplicationBase
  • 920472 CollectionAssert.IsNotEmpty must dispose Enumerator
  • 922455 Add Support for Windows 8 and Windows 2012 Server to PlatformAttribute
  • 928246 Use assembly.Location instead of assembly.CodeBase
  • 958766 For development work under TeamCity, we need to support nunit2 formatted output under direct-runner
  • 1000181 Parameterized TestFixture with System.Type as constructor arguments fails
  • 1000213 Inconclusive message Not in report output
  • 1023084 Add Enum support to RandomAttribute
  • 1028188 Add Support for Silverlight
  • 1029785 Test loaded from remote folder failed to run with exception System.IODirectory
  • 1037144 Add MonoTouch support to PlatformAttribute
  • 1041365 Add MaxOsX and Xbox support to platform attribute
  • 1057981 C#5 async tests are not supported
  • 1060631 Add .NET 4.5 build
  • 1064014 Simple async tests should not return Task<T>
  • 1071164 Support async methods in usage scenarios of Throws constraints
  • 1071343 Runner.Load fails on CF if the test assembly contains a generic method
  • 1071861 Error in Path Constraints
  • 1072379 Report test execution time at a higher resolution
  • 1074568 Assert/Assume should support an async method for the ActualValueDelegate
  • 1082330 Better Exception if SetCulture attribute is applied multiple times
  • 1111834 Expose Random Object as part of the test context
  • 1111838 Include Random Seed in Test Report
  • 1172979 Add Category Support to nunitlite Runner
  • 1203361 Randomizer uniqueness tests sometimes fail
  • 1221712 When non-existing test method is specified in -test, result is still “Tests run: 1, Passed: 1”
  • 1223294 System.NullReferenceException thrown when ExpectedExceptionAttribute is used in a static class
  • 1225542 Standardize commandline options for test harness

0.19.1 Earlier Releases

  • Release Notes for <a href=“http://www.nunit.org/?p=releaseNotes&r=2.6.4”>NUnit 2.6 through 2.6.4</a>
  • Release Notes for <a href=“http://www.nunit.org/?p=releaseNotes&r=2.5.10”>NUnit 2.5 through 2.5.10</a>
  • Release Notes for <a href=“http://www.nunit.org/?p=releaseNotes&r=2.4.8”>NUnit 2.4 through 2.4.8</a>
  • Release Notes for <a href=“http://www.nunit.org/?p=releaseNotes&r=2.2.10”>NUnit 2.0 through 2.2.10</a>

This attribute is used inside a [[TestFixture|TestFixture-Attribute]] to provide a common set of functions that are performed after each test method.

TearDown methods may be either static or instance methods and you may define more than one of them in a fixture. Normally, multiple TearDown methods are only defined at different levels of an inheritance hierarchy, as explained below.

So long as any SetUp method runs without error, the TearDown method is guaranteed to run. It will not run if a SetUp method fails or throws an exception.

0.19.1.1 Example:

namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture]
  public class SuccessTests
  {
    [SetUp] public void Init()
    { /* ... */ }

    [TearDown] public void Cleanup()
    { /* ... */ }

    [Test] public void Add()
    { /* ... */ }
  }
}

0.19.1.2 Inheritance

The TearDown attribute is inherited from any base class. Therefore, if a base class has defined a TearDown method, that method will be called after each test method in the derived class.

You may define a TearDown method in the base class and another in the derived class. NUnit will call base class TearDown methods after those in the derived classes.

0.19.1.3 Notes:

  1. Although it is possible to define multiple TearDown methods in the same class, you should rarely do so. Unlike methods defined in separate classes in the inheritance hierarchy, the order in which they are executed is not guaranteed.

2.TearDown methods may be async if running under .NET 4.0 or higher.

0.19.1.4 See also…

  • [[SetUp Attribute]]
  • [[OneTimeSetUp Attribute]]
  • [[OneTimeTearDown Attribute]]
  • [[TestFixture Attribute]]

The Test attribute is one way of marking a method inside a TestFixture class as a test. It is normally used for simple (non-parameterized) tests but may also be applied to parameterized tests without causing any extra test cases to be generated. See [[Parameterized Tests]] for more info.

The test method may be either an instance or a static method.

Test methods targetting .Net 4.0 or higher may be marked as <b>async</b> and NUnit will wait for the method to complete before recording the result and moving on to the next test. Async test methods must return Task if no value is returned, or Task<T> if a value of type T is returned.

If the programmer marks a test method that does not have the correct signature it will be considered as not runnable and be indicated as such by the console or gui runner. In the Gui, such tests are marked in red.

If the test method returns a value, you must pass in the ExpectedResult named parameter to the Test attribute. This expected return value will be checked for equality with the return value of the test method.

The test Description may be specified as a named parameter. This is exactly equivalent to using the DescriptionAttribute on the test.

0.19.1.5 Examples:

namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture]
  public class SuccessTests
  {
    // A simple test
    [Test]
    public void Add()
    { /* ... */ }

    // A test with a descripton property
    [Test(Description="My really cool test")]
    public void Add()
    { /* ... */ }

    // Alternate way to specify description as a separate attribute
    [Test, Description("My really really cool test")]
    public void Add()
    { /* ... */ }

    // A simple async test
    [Test]
    public async Task AddAsync()
    { /* ... */ }
   
    // Test with an expected result
    [Test(ExpectedResult = 4)]
    public int TestAdd()
    {
        return 2 + 2;
    }
   
    // Async test with an expected result
    [Test(ExpectedResult = 4)]
    public async Task<int> TestAdd()
    {
        await ...
        return 2 + 2;
    }
  }
}

0.19.2 DRAFT

This spec summarizes the three levels of APIs provided by NUnit for discovering and running tests. Note that only one of them - the highest level - is supported for general usage. The others have very specific purposes and should only be used for those purposes.

The three APIs, from highest to lowest level, are:

  • Test Engine API - for use by any program that needs to discover and execute tests.
  • Engine Driver API - for use by framework drivers, intended to be loaded by the engine to enable communication with a particular framework.
  • Framework API - only used by NUnit’s own framework driver to communicate with the framework.

0.19.2.1 Test Engine API

The NUnit TestEngine is a separate component, new to NUnit 3.0, which knows how to discover and execute tests. It provides an API for both simple batch execution and more complex interaction as needed by typical Gui test runners. It also provides additional Engine services beyond what the framework provides.

This API is currently under development. See [Test Engine API] for more info.

0.19.2.2 Engine Driver API

The NUnit TestEngine uses drivers to communicate with test frameworks. Initially, the engine will come with a driver for NUnit 3.0, followed by an NUnit 2.x driver. It is possible to create a driver for running any sort of test framework, supporting any language at all. The driver API is what makes this possible.

The driver API is only intended to be implemented by drivers and is only used by the NUnit engine. This API is currently under development. See [Engine Driver API] for more info.

0.19.2.3 NUnit Framework API

This is a primitive API implemented by the nunit.framework and nunitlite assemblies. The NUnitFrameworkDriver in the engine uses this API. The API is a bit complicated to use. Since it needs to support multiple versions of the framework, it uses well-known framework class names, which are constructed via reflection. All results are returned as raw XML.

This API is not intended for any use except by NUnit itself. See [Framework API] for more info.

Test Filters represent a selection of tests to be displayed, run or loaded. When a filter needs to be passed to the framework, it is passed as a string containing an XML fragment. This page describes the elements present in the XML for a filter.

0.19.2.4 <filter>

This is the required top-level element for any filter. If it contains no other elements, it represents an empty filter. If it contains just one element, that element is used as the filter for all tests. If it contains multiple elements, it works like an <b><and></b> element.

Child elements allowed: <b><and></b>, <b><or></b>, <b><not></b>, <b><id></b>, <b><tests></b>, <b><cat></b>.

0.19.2.5 <and>

Represents an AndFilter. All contained filter elements must pass in order for this filter to pass.

Child elements allowed: <b><and></b>, <b><or></b>, <b><not></b>, <b><id></b>, <b><tests></b>, <b><cat></b>.

0.19.2.6 <or>

Represents an OrFilter. At least one contained filter element must pass in order for this filter to pass.

Child elements allowed: <b><and></b>, <b><or></b>, <b><not></b>, <b><id></b>, <b><tests></b>, <b><cat></b>.

0.19.2.7 <not>

Represents a NotFilter. The single contained filter element must fail in order for this filter to pass.

Child elements allowed: <b><and></b>, <b><or></b>, <b><not></b>, <b><id></b>, <b><tests></b>, <b><cat></b>.

0.19.2.8 <id>

Represents an IdFilter. The text of the element contains a single test id or multiple ids separated by commas. Since test ids are an internal construct, this filter is not useful externally. However, it is used by the NUnit Gui to efficiently select tests from the list of those loaded.

Child elements allowed: None.

0.19.2.9 <tests>

Represents a selection of test names. This element contains one or more <b><test></b> elements, each holding the full name of a test as it’s inner text.

Child elements allowed: None.

0.19.2.10 <cat>

Represents a CategoryFilter. The text of the element contains a single category or multiple categories separated by commas.

Child elements allowed: None.

0.20 <test-run>

The required root element for any NUnit 3.0 test result file. * Containing Elements: None * Contained Elements: <command-line>, <filter>, <test-suite> * Attributes: * id The unique ID of this test. For the test-run, it is hard-coded as “2”. * testcasecount The number of test cases contained in this test run. * result The basic result of the test. May be Passed, Failed, Inconclusive or Skipped. * total The total number of test cases executed in the run. This may be less than the testcasecont due to filtering of tests. * passed The number of test cases that passed. * failed The number of test cases that failed. * inconclusive The number of test cases that were inconclusive. * skipped The number of test cases that were skipped. * asserts The number of asserts executed in the test run. * engine-version The version of the NUnit test engine in use. * clr-version The runtime version under which the engine is running, taken from Environment.Version. * start-time The UTC time that the test run started. * end-time The UTC time that the test run ended. * duration The duration of the test run in seconds, expressed as a real number.

0.21 <command-line>

Holds a CDATA section containing the text of the command used to run the tests. * Containing Elements: <test-run> * Contained Elements: None * Attributes: None

0.22 <filter>

The XML representation of the filter used to execute tests. This element is also used as a fragment in passing the filter to a runner and by the NUnit 3.0 framework and driver.

0.23 <or>

Represents a composite filter that contains other filters. At least one of the contained filters must pass in order for this filter to pass.

0.24 <and>

Represents a composite filter that contains other filters. All of the contained filters must pass in order for this filter to pass.

0.25 <not>

Represents a composite filter that contains wraps a single base filters. The base filter must fail in order for this filter to pass.

0.26 <id>

Represents a filter that examines the test id, which is generated by NUnit.

0.27 <test>

  • Containing Elements: <filter>, <or>, <and>, <not>
  • Contained Elements: None
  • Attributes:
    • re Set to ‘1’ to indicate that a regular expression comparison is to be used.

0.28 <class>

  • Containing Elements: <filter>, <or>, <and>, <not>
  • Contained Elements: None
  • Attributes:
    • re Set to ‘1’ to indicate that a regular expression comparison is to be used.

0.29 <method>

  • Containing Elements: <filter>, <or>, <and>, <not>
  • Contained Elements: None
  • Attributes:
    • re Set to ‘1’ to indicate that a regular expression comparison is to be used.

0.30 <prop>

  • Containing Elements: <filter>, <or>, <and>, <not>
  • Contained Elements: None
  • Attributes:
    • name The name of the property to filter on.
    • re Set to ‘1’ to indicate that a regular expression comparison is to be used.

0.31 <cat>

0.32 <test-suite>

  • Containing Elements: <test-run>, <test-suite>
  • Contained Elements: <environment>, <settings>, <properties>, <reason>, <failure>, <output>, <test-suite>, <test-case>
  • Attributes
    • type The type of suite represented by this element. Currently supported types are Assembly, TestSuite, TestFixture,
    • id The unique id of this test. Coded as “mmm-nnn” where the part before the hyphen represents the assembly and the part after it represents a test in that assembly. Currently, mmm and nnn are ints, but that is merely an accident of the implementation and should not be relied on.
    • name The display name of the test as generated by NUnit.
    • fullname The full name of the test as generated by NUnit.
    • classname The full name of the class (fixture) representing this test. Only present if type is equal to “TestFixture”.
    • testcasecount The number of test cases contained, directly or indirectly, in this suite.
    • runstate An indicator of whether the suite is runnable. Value may be NotRunnable, Runnable, Explicit, Skipped or Ignored. NotRunnable means there is an error in how the test is expressed in code, for example, the signature may be wrong. Explicit, Skipped and Ignored are set by attributes on the test.
    • result The basic result of the test. May be Passed, Failed, Inconclusive or Skipped.
    • label Additional labeling information about the result. In principle, this may be any string value and is combined with the result to give a more precise reason for failure. Currently, NUnit may use the following labels in combination with the Failure result: Error, Cancelled or Invalid. It may use the following labels in combination with a Skipped result: Ignored or Explicit. Additional values may be added in future releases or supplied by extensions, so code that processes a result file should be prepared to deal with any label or none at all.
    • site Optional element indicating where a failure occured. Values are Test, SetUp, TearDown, Parent or Child. Default is Test and the attribute does not normally appear in that case.
    • start-time The UTC time that the suite started.
    • end-time The UTC time that the suite ended.
    • duration The duration of the suite in seconds, expressed as a real number.
    • total The total number of test cases executed under this suite.
    • passed The number of test cases that passed.
    • failed The number of test cases that failed.
    • inconclusive The number of test cases that were inconclusive.
    • skipped The number of test cases that were skipped.
    • asserts The number of asserts executed by the suite, including any nested suites or test cases. Since asserts may be executed in OneTimeSetUp and in ActionAttributes, this number can be greater than the total of the asserts for the test cases.

0.33 <environment>

Describes the environment in which the tests in a particular assembly are being run. * Containing Elements: <test-suite> (Assembly level only) * Contained Elements: None * Attributes: * framework-version The version of the nunit framework in use. * clr-version The runtime version under which the tests are running, taken from Environment.Version. * os-version A text string describing the operating system running the tests, taken from Environment.OsVersion. * platform The platform id, taken from Environment.OsVersion.Platform. * cwd The current working directory path, taken from Environment.CurrentDirectory. * machine-name The machine name, taken from Environment.MachineName. * user The user id, taken from Environment.UserName. * user-domain The user domain, taken from Environment.UserDomainName. * culture The current culture, taken from CultureInfo.CurrentCulture. * uiculture The current UI culture, taken from CultureInfo.CurrentUICulture. * os-architecture The architecture, taken from GetProcessorArchitecture().

0.34 <settings>

Settings used by the engine for executing an assembly. These are taken from the supplied settings in the TestPackage supplemented by default settings created by the engine itself. * Containing Elements: <test-suite> (Assembly level only) * Contained Elements: <setting> * Attributes: None

0.35 <setting>

A single setting * Containing Elements: <settings> * Contained Elements: None * Attributes: * name The name of the setting. * value The value assigned to the setting.

0.36 <test-case>

  • Containing Elements: <test-suite>
  • Contained Elements: <properties>, <reason>, <failure>, <output>
  • Attributes
    • id The unique id of this test. Coded as “mmm-nnn” where the part before the hyphen represents the assembly and the part after it represents a test in that assembly. Currently, mmm and nnn are ints, but that is merely an accident of the implementation and should not be relied on.
    • name The display name of the test as generated by NUnit or, in the case of some parameterized tests, specified by the user.
    • fullname The full name of the test as generated by NUnit.
    • methodname The name of the method representing the test case.
    • classname The full name of the class in which this test case appears.
    • runstate An indicator of whether the suite is runnable. Value may be NotRunnable, Runnable, Explicit, Skipped or Ignored. NotRunnable means there is an error in how the test is expressed in code, for example, the signature may be wrong. Explicit, Skipped and Ignored are set by attributes on the test.
    • seed The seed used to generate random arguments and other values for this test.
    • result The basic result of the test. May be Passed, Failed, Inconclusive or Skipped.
    • label Additional labeling information about the result. In principle, this may be any string value and is combined with the result to give a more precise reason for failure. Currently, NUnit may use the following labels in combination with the Failure result: Error, Cancelled or Invalid. It may use the following labels in combination with a Skipped result: Ignored or Explicit. Additional values may be added in future releases or supplied by extensions, so code that processes a result file should be prepared to deal with any label or none at all.
    • site Optional element indicating where a failure occured. Values are Test, SetUp, TearDown, Parent or Child. Default is Test and the attribute does not normally appear in that case.
    • start-time The UTC time that the test started.
    • end-time The UTC time that the test ended.
    • duration The duration of the test in seconds, expressed as a real number.
    • asserts The number of asserts executed by the test.

0.37 <properties>

Optional element containing any properties assigned to the test case or suite. * Containing Elements: <test-suite>, <test-case> * Contained Elements: <property> * Attributes None

0.38 <property>

A single property * Containing Elements: <properties> * Contained Elements: None * Attributes * name The name of the property * value The value of the property

0.39 <reason>

Optional element that may appear on tests or suites that were not executed. Contains a message giving the reason for skipping the test. * Containing Elements: <test-suite>, <test-case> * Contained Elements: <message> * Attributes None

0.40 <failure>

Optional element that appears on all tests or suites with a result of ‘Failed’. Contains the error message and optionally a stack trace. * Containing Elements: <test-suite>, <test-case> * Contained Elements: <message>, <stack-trace> * Attributes None

0.41 <message>

Optional element with a CDATA section containing a message relating to the test’s result. * Containing ELements: <failure>, <reason> * Contained Elements: None * **Attributes: None

0.42 <stack-trace>

Optional element with a CDATA section containing a stack-trace of the location where a test failed. * Containing Elements: <failure> * Contained Elements: None * **Attributes: None

0.43 <output>

Optional element that appears on tests or suites that produce text output. The output may be intercepted from writes to the console or captured directly when the test writes to the TestContext. It is contained in a CDATA section. * Containing Elements: <test-suite>, <test-case> * Contained Elements: None * **Attributes: None

The console command-line allows you to specify a filter, which will select which tests are executed. This is done using the –where option, followed by an expression in NUnit’s Test Selection Language (TSL), a simple domain-specific language designed for this purpose.

Some of the characters used in the expression, such as space, | or &, may have a special meaning when entered on the command-line. In such a case, you should place the expression in quotation marks.

  nunit3-console mytest.dll --where "cat == Urgent || Priority == High"

Note that TSL is handled by the NUnit engine but requires framework support to actually select the tests. The NUnit 3.0 framework supports it fully. See below for support limitations in NUnit V2 tests.

0.43.0.1 Simple Expressions

Simple Expressions are essentially comparisons, consisting of a key word or property name on the left-hand side, an operator and some constant value on the right-hand side. Here are some examples:

  cat == Data
  test =~ /TestCaseAttributeTest/
  method == SomeMethodName
  cat != Slow
  Priority == High

The following key words are recognized on the left-hand side of the comparison: * test - The full test name as assigned by NUnit, e.g. My.Name.Space.TestFixture.TestMethod(5) * name - The test name assigned by NUnit, e.g. TestMethod(5) * class - The full name of the class containing the test, e.g. My.Name.Space.TestFixture * method - The name of the method, e.g. TestMethod * cat - A category assigned to the test, e.g. SmokeTests

If the left-hand side of the comparison does not consist of a key word, it is treated as the name of a property on the test whose value is to be checked. See below for restrictions on use of properties.

The following operators are supported * == to test for equality - a single equal sign (=) may be used as well and has the same meaning * != to test for inequality * =~ to match a regular expression * !~ to not match a regular expression

The right-hand side of the comparison may be a sequence of non-blank, non-special characters or a quoted string. Quoted strings may be surrounded by single quotes ('), double quotes (") or slashes (/) and may contain any character except the quote character used to delimit them. If it is necessary to include the quote character in the string, it may be escaped using a backslash () as may the backslash itself should you need to include one. The following expressions all do the same thing:

  test =~ /TestCaseAttributeTest/
  test =~ "TestCaseAttributeTest"
  test =~ 'TestCaseAttributeTest'
  test =~ TestCaseAttributeTest
  test=~TestCaseAttributeTest

For matching regular expressions, NUnit users .NET’s Regex.IsMatch method. For detailed information on the syntax of regular expressions in .NET, see https://msdn.microsoft.com/en-us/library/az24scfc%28v=vs.110%29.aspx.

0.43.0.2 Filtering Based on Properties

Although the syntax will accept any property name - including names that don’t actually exist - filtering is only useful for existing, string-valued properties. The following properties are created by NUnit and have string values:

  • Author
  • Category
  • Description
  • SetCulture
  • SetUICulture
  • TestOf
  • IgnoreUntilDate

In general, these properties were not created with filtering in mind, but you can use them if it suits your needs. Using the Category property currently accomplishes the same thing as the cat keyword. You should be aware that the use of these properties by NUnit is considered an implementation detail and they may change in the future.

We envision that most filtering by property will be based on user-defined properties, created for this purpose by inheriting from [[Property Attribute]]. When defining a property, you should keep the limitation to string values in mind. For example, a PriorityAttribute taking values of “High”, “Medium” and “Low” could be used for filtering, while one that took the integers 1, 2 and 3 could not.

0.43.0.3 Filtering by Test Id

In addition to the left-hand-side items listed, NUnit supports filtering by the test id through the id keyword. The id may only be selected using the == operator and is intended only for use by programs that have explored the tests and cached the ids, not for general use by users. The reason for this restriction is that users have no way of predicting the id that will be assigned to a test. The id is not persistent across test runs and it’s format can differ between different framework drivers.

0.43.0.4 Compound Expressions

Simple expressions may be combined using logical and, logical or, parentheses or negation operators.

Logical and is expressed as &&, & or and. Logical or is expressed as ||, |, or or. The negation operator is ! and may only appear before a left parenthesis. The letter variants, and and or, are provided for use on the command-line in systems that give & and | a special meaning.

The following are valid compound expressions:

  test == "My.Namespace" and cat == Urgent
  test == "My.Namespace" and (cat == Urgent or Priority == High)
  test == "My.Namespace" and (cat == Urgent or Priority == High)
  method =~ /Source.*Test/ and class =~ "My.Namespace.ClassName"

0.43.0.5 Usage on the Command Line

Because TSL contains special characters and may contain blank spaces, you will usually want to put the expression in quotes on the command line. Consequently, any strings within the TSL expression will most likely need to use an alternate quote character. For example:

  nunit-console test.dll --where "method =~ /Source.*Test/ && class =~ 'My.Namespace.Classname'"

0.43.0.6 Support in NUnit V2

The driver for NUnit V2 supports a subset of TSL. Because the V2 NUnit framework only allowed filtering on test names and categories, you may only use the cat and test keywords in comparisons. In addition, the regular expression operators =~ and !~ are not supported.

If you use any of the unsupported keywords or operators with V2 tests, an error message is displayed and the tests are not run.

TestCaseAttribute serves the dual purpose of marking a method with parameters as a test method and providing inline data to be used when invoking that method. Here is an example of a test being run three times, with three different sets of data:

[TestCase(12,3,4)]
[TestCase(12,2,6)]
[TestCase(12,4,3)]
public void DivideTest(int n, int d, int q)
{
  Assert.AreEqual( q, n / d );
}

Note: Because arguments to .NET attributes are limited in terms of the Types that may be used, NUnit will make some attempt to convert the supplied values using Convert.ChangeType() before supplying it to the test.

TestCaseAttribute may appear one or more times on a test method, which may also carry other attributes providing test data. The method may optionally be marked with the [[Test Attribute]] as well.

By using the named parameter ExpectedResult this test set may be simplified further:

[TestCase(12,3, ExpectedResult=4)]
[TestCase(12,2, ExpectedResult=6)]
[TestCase(12,4, ExpectedResult=3)]
public int DivideTest(int n, int d)
{
  return( n / d );
}

In the above example, NUnit checks that the return value of the method is equal to the expected result provided on the attribute.

TestCaseAttribute supports a number of additional named parameters:

  • Author sets the author of the test.

  • Category provides a comma-delimited list of categories for this test.

  • Description sets the description property of the test.

  • ExcludePlatform specifies a comma-deliminted list of platforms on which the test should not run.

  • ExpectedResult sets the expected result to be returned from the method, which must have a compatible return type.

  • Explicit is set to true in order to make the individual test case Explicit. Use Reason to explain why.

  • Ignore* causes the test case to be ignored and specifies the reason.

  • IgnoreReason causes this test case to be ignored and specifies the reason.

  • IncludePlatform specifies a comma-delimited list of platforms on which the test should run.

  • Reason specifies the reason for not running this test case. Use in conjunction with <b>Explicit</b>.

  • TestName provides a name for the test. If not specified, a name is generated based on the method name and the arguments provided.

  • TestOf specifies the Type that this test is testing

0.43.0.7 Order of Execution

Individual test cases are executed in the order in which NUnit discovers them. This order does not necessarily follow the lexical order of the attributes and will often vary between different compilers or different versions of the CLR.

As a result, when <b>TestCaseAttribute</b> appears multiple times on a method or when other data-providing attributes are used in combination with <b>TestCaseAttribute</b>, the order of the test cases is undefined.

Although any object implementing <b>ITestCaseData</b> may be used to provide extended test case information, NUnit provides the <b>TestCaseData</b> class for this purpose. The following example returns <b>TestCaseData</b> instances from a data source in a separately defined class.

[TestFixture]
public class MyTests
{
    [Test,TestCaseSource(typeof(MyDataClass),"TestCases")]
    public int DivideTest(int n, int d)
    {
        return n/d;
    }
}

public class MyDataClass
{
    public static IEnumerable TestCases
    {
        get
        {
            yield return new TestCaseData( 12, 3 ).Returns( 4 );
            yield return new TestCaseData( 12, 2 ).Returns( 6 );
            yield return new TestCaseData( 12, 4 ).Returns( 3 );
        }
    }  
}

This example uses the fluent interface supported by <b>TestCaseData</b> to make the program more readable.

TestCaseData supports the following properties and methods, which may be appended to an instance in any order.

  • Explicit() or Explicit(string) causes the test case to be marked explicit, optionally specifing the reason for doing so.

  • Ignore(string) causes the test case to be ignored and specifies the reason, which is required.

  • Returns specifies the expected result to be returned from the method, which must have a compatible return type.

  • SetCategory(string) applies a category to the test.

  • SetDescription(string) sets the description property of the test.

  • SetName(string) provides a name for the test. If not specified, a name is generated based on the method name and the arguments provided.

  • SetProperty(string, string), SetProperty(string, int) and SetProperty(string, double) apply a named property and value to the test.

TestCaseSourceAttribute is used on a parameterized test method to identify the source from which the required arguments will be provided. The attribute additionally identifies the method as a test method. The data is kept separate from the test itself and may be used by multiple test methods. See [[Parameterized Tests]] for a general introduction to tests with arguments.

0.43.0.8 Usage

Consider a test of the divide operation, taking three arguments: the numerator, the denominator and the expected result. We can specify the test and it’s data using one of the forms of TestCaseSourceAttribute:

0.43.0.8.0.1 Form 1 - [TestCaseSource(string sourceName)]
public class MyTestClass
{
    [TestCaseSource("DivideCases")]
    public void DivideTest(int n, int d, int q)
    {
        Assert.AreEqual( q, n / d );
    }

    static object[] DivideCases = {
        new object[] { 12, 3, 4 },
        new object[] { 12, 2, 6 },
        new object[] { 12, 4, 3 }
    };
}

The single attribute argument in this form is a string representing the name of the source used to provide test cases. It has the following characteristics:

  • It may be a field, property or method in the test class.

  • It must be static. This is a change from NUnit 2.x.

  • It must return an IEnumerable or a type that implements IEnumerable. For fields an array is generally used. For properties and methods, you may return an array or implement your own iterator.

  • The individual items returned by the enumerator must be compatible with the signature of the method on which the attribute appears. See the Test Case Construction section below for details.

0.43.0.8.0.2 Form 2 - [TestCaseSource(Type sourceType, string sourceName)]
public class MyTestClass
{
    [TestCaseSource(typeof(AnotherClass), "DivideCases")]
    public void DivideTest(int n, int d, int q)
    {
        Assert.AreEqual( q, n / d );
    }
}

class AnotherClass
{
    static object[] DivideCases = {
        new object[] { 12, 3, 4 },
        new object[] { 12, 2, 6 },
        new object[] { 12, 4, 3 }
    };
}

The first argument of the attribute in this form is a Type representing the class that will provide the test cases.

The second argument is a string representing the name of the source used to provide test cases. It has the following characteristics:

  • It may be a field, property or method in the test class.

  • It must be static. This is a change from NUnit 2.x.

  • It must return an IEnumerable or a type that implements IEnumerable. For fields an array is generally used. For properties and methods, you may return an array or implement your own iterator.

  • The individual items returned by the enumerator must be compatible with the signature of the method on which the attribute appears. See the Test Case Construction section below for details.

0.43.0.8.0.3 Form 3 - [TestCaseSource(Type sourceType)]
public class MyTestClass
{
    [TestCaseSource(typeof(DivideCases))]
    public void DivideTest(int n, int d, int q)
    {
        Assert.AreEqual( q, n / d );
    }
}

class DivideCases : IEnumerable
{
    public IEnumerator GetEnumerator()
    {
        yield return new object[] { 12, 3, 4 };
        yield return new object[] { 12, 2, 6 };
        yield return new object[] { 12, 4, 3 };
    }
}

The Type argument in this form represents the class that provides test cases. It must have a default constructor and implement <b>IEnumerable</b>. The enumerator should return test case data compatible with the signature of the test on which the attribute appears. See the Test Case Construction section below for details.

0.43.0.9 Named Parameters

TestCaseSourceAttribute supports one named parameter:

  • Category is used to assign one or more categories to every test case returned from this source.

0.43.0.10 Test Case Construction

In constructing tests, NUnit uses each item returned by the enumerator as follows:

  1. If it is an object implementing NUnit.Framework.ITestCaseData, its properties are used to provide the test case. NUnit provides the [[TestCaseData]] type for this purpose.

  2. If the test has a single argument and the returned value matches the type of that argument it is used directly.

This can eliminate a bit of extra typing by the programmer, as in this example:

```C# static int[] EvenNumbers = new int[] { 2, 4, 6, 8 };

[Test, TestCaseSource(“EvenNumbers”)] public void TestMethod(int num) { Assert.IsTrue( num % 2 == 0 ); } ```

  1. If it is an <b>object[]</b>, its members are used to provide the arguments for the method. This is the approach taken in the three examples above.

  2. If it is an array of some other type, NUnit can use it provided that the arguments to the method are all of that type. For example, the above examples could be modified to make the three nested arrays of type int[].

  3. If anything else is returned, it is used directly as the sole argument to the method. Because every returned value is used, NUnit is able to give an error message in cases where the method requires a different number of arguments or an argument of a different type.

0.43.0.10.1 Notes:
  1. It is recommended that the SourceType not be the same as the test fixture class. It may be a nested class, however, and probably should be if the data is only used within that fixture.

  2. A generic IEnumerable and IEnumerator may be used but NUnit will actually deal with the underlying IEnumerator in the current release.

  3. The GetEnumerator method may use yield statements or simply return the enumerator for an array or other collection held by the class.

0.43.0.11 Order of Execution

Individual test cases are executed in the order in which NUnit discovers them. This order does <b>not</b> follow the lexical order of the attributes and will often vary between different compilers or different versions of the CLR.

As a result, when <b>TestCaseSourceAttribute</b> appears multiple times on a method or when other data-providing attributes are used in combination with <b>TestCaseSourceAttribute</b>, the order of the test cases is undefined.

However, when a single <b>TestCaseSourceAttribute</b> is used by itself, the order of the tests follows exactly the order in which the test cases are returned from the source.

0.43.0.12 Object Construction

NUnit locates the test cases at the time the tests are loaded. It creates instances of each class used with the third form of the attribute and builds a list of tests to be executed. Each data source class is only created once at this time and is destroyed after all tests are loaded. By design, no communication is possible between the load and execution phases except through the tests that are created.

Each NUnit test runs in an execution context, which includes information about the environment as well as the test itself. The TestContext class allows tests to access certain information about the execution context.

0.43.1 Static Properties

0.43.1.1 CurrentContext

Gets the context of the currently executing test. This context is created separately for each test before it begins execution. See below for properties of the current context.

0.43.1.2 Out

Gets a TextWriter used for sending output to the current test result.

0.43.1.3 Error

Gets a TextWriter used for sending error output intended for immediate display.

0.43.1.4 Progress

Gets a TextWriter used for sending normal (non-error) output intended for immediate display.

0.43.1.5 TestParameters

Test parameters may be supplied to a run in various ways, depending on the runner used. For example, the console runner provides a command-line argument and v3.4 of the NUnit 3 VS Adapter will supports specifying them in a .runsettings file. The static TestParameters property returns an object representing those passed-in parameters.

The TestParameters object supports the following properties:

  • Count - The number of parameters.
  • Names - A collection of the names of the supplied parameters.
  • this[string name] - The value of a parameter. In Vb, use Item.

The TestParameters object supports the following methods:

  • Exists(string name) - Returns true if a parameter of that name exists.
  • Get(string name) - Returns the same value as the indexer.
  • Get<T>(string name, T defaultValue) - Returns the value of the parameter converted from a string to type T or the specified default if the paramter doesn’t exist. Throws an exception if conversion fails.

Note that all parameter values are strings. You may convert them to other Types using the generic Get method listed above or using your own code. An exception may be thrown if the supplied value cannot be converted correctly.

0.43.2 Static Methods

0.43.2.1 Write

Writes text to the current test result.

    Write(bool value)
    Write(char value)
    Write(char[] value)
    Write(double value)
    Write(Int32 value)
    Write(Int64 value)
    Write(decimal value)
    Write(object value)
    Write(Single value)
    Write(string value)
    Write(UInt32 value)
    Write(UInt64 value)
    Write(string format, object arg1)
    Write(string format, object arg1, object arg2)
    Write(string format, object arg1, object arg2, object arg3)
    Write(string format, params object[] args)

0.43.2.2 WriteLine

Writes text to the current test result, followed by a newline.

    WriteLine()
    WriteLine(bool value)
    WriteLine(char value)
    WriteLine(char[] value)
    WriteLine(double value)
    WriteLine(Int32 value)
    WriteLine(Int64 value)
    WriteLine(decimal value)
    WriteLine(object value)
    WriteLine(Single value)
    WriteLine(string value)
    WriteLine(UInt32 value)
    WriteLine(UInt64 value)
    WriteLine(string format, object arg1)
    WriteLine(string format, object arg1, object arg2)
    WriteLine(string format, object arg1, object arg2, object arg3)
    WriteLine(string format, params object[] args)

0.43.2.3 AddFormatter (3.2+)

Adds a formatter for values based on some criterion, such as the Type of the value. The provided formatter will be used when an expected or actual value needs to be displayed as part of a message from a constraint.

    AddFormatter(ValueFormatter formatter);
    AddFormatter(ValueFormatterFactory formatterFactory);

Both ValueFormatter and ValueFormatterFactory are delegates. ValueFormatter takes a single object as an argument and returns it’s string representation. The AddFormatter overload that takes a ValueFormatter is intended for use in most cases that arise.

0.43.3 Properties of the CurrentContext

0.43.3.1 Test

Gets a representation of the current test, with the following properties:

  • ID - The unique Id of the test
  • Name - The name of the test, whether set by the user or generated automatically
  • FullName - The fully qualified name of the test
  • MethodName - The name of the method representing the test, if any
  • Properties - An IPropertyBag of the test properties

0.43.3.2 Result

Gets a representation of the test result, with the following properties:

  • Outcome - A ResultState representing the outcome of the test. ResultState has the following properties:
  • Status - A TestStatus with four possible values:
    • Inconclusive
    • Skipped
    • Passed
    • Failed
  • Label - An optional string value, which can provide sub-categories for each Status. See below for a list of common outcomes supported internally by NUnit.
  • Site - A FailureSite value, indicating the stage of execution in which the test generated its result. Possible values are
    • Test
    • SetUp
    • TearDown
    • Parent
    • Child

Although the outcome of the test may be accessed during setup or test execution, it only has a useful value in the teardown stage.

0.43.3.2.1 Common Outcomes

The following is a list of outcomes currently produced by NUnit. Others may be added in the future. * Success: the test passed. (Status=Passed) * Inconclusive: the test was inconclusive. (Status=Inconclusive) * Failure: a test assertion failed. (Status=Failed, Label=empty) * Error: an unexpected exception occurred. (Status=Failed, Label=Error) * NotRunnable: the test was invalid and could not be run. (Status=Failed, Label=Invalid) * Cancelled: the user cancelled while this test was running. (Status=Failed, Label=Cancelled) * Ignored: the test was ignored. (Status=Skipped, Label=Ignored) * Explicit: the test was not run because it is marked Explicit. (Status=Skipped, Label=Explicit) * Skipped: the test was skipped for some other reason. (Status=Skipped, Label=empty)

0.43.3.3 TestDirectory

Gets the full path of the directory containing the current test assembly. Not available in the Silverlight or Portable builds.

0.43.3.4 WorkDirectory

Gets the full path of the directory to be used for output from this test run. The XML result file and any redirected output files are located under this directory. This is normally the directory that was current when execution of NUnit began but may be changed by use of the –work option of nunit-console.

0.43.3.5 Random

Returns a Randomizer object, which may be used in the test code to generate random values. These values are repeatable on reruns of the tests so long as (a) the test assembly is not changed and (b) the same seed is used. The initial random seed used in any test run may be found in the XML result file and may be provided to a subsequent run on the command line.

See [[Randomizer Methods]] for details about each available random data type.

This is the attribute that marks a class that contains tests and, optionally, setup or teardown methods.

Most restrictions on a class that is used as a test fixture have now been eliminated. A test fixture class:

  • May be public, protected, private or internal.
  • May be a static class.
  • May be generic, so long as any type parameters are provided or can be inferred from the actual arguments.
  • May not be abstract - although the attribute may be applied to an abstract class intended to serve as a base class for test fixtures.
  • If no arguments are provided with the TestFixtureAttribute, the class must have a default constructor.
  • If arguments are provided, they must match one of the constructors.

If any of these restrictions are violated, the class is not runnable as a test and will display as an error.

It is advisable that the constructor not have any side effects, since NUnit may construct the object multiple times in the course of a session.

Beginning with NUnit 2.5, the <b>TestFixture</b> attribute is optional for non-parameterized, non-generic fixtures. So long as the class contains at least one method marked with the <b>Test</b>, <b>TestCase</b> or <b>TestCaseSource</b> attribute, it will be treated as a test fixture.

0.43.3.6 Example:

namespace NUnit.Tests
{
  using System;
  using NUnit.Framework;

  [TestFixture]
  public class SuccessTests
  {
    // ...
  }
}

0.43.3.7 Inheritance

The <b>TestFixtureAttribute</b> may be applied to a base class and is inherited by any derived classes. This includes any abstract base class, so the well-known Abstract Fixture pattern may be implemented if desired.

In order to facilitate use of generic and/or parameterized classes, where the derived class may require a different number of arguments (or type arguments) from the base class, superfluous <b>TestFixture</b> attributes are ignored, using the following rules:

  1. If all TestFixture attributes provide constructor or type arguments, then all of them are used.
  2. If some of the attributes provide arguments and others do not, then only those with arguments are used and those without arguments are ignored.
  3. If none of the attributes provide arguments, one of them is selected for use by NUnit. It is not possible to predict which will be used, so this situation should generally be avoided.

This permits code like the following, which would cause an error if the attribute on the base class were not ignored.

[TestFixture]
public class AbstractFixtureBase
{
    ...
}

[TestFixture(typeof(string))]
public class DerivedFixture<T> : AbstractFixtureBase
{
    ...
}

0.43.3.8 Parameterized Test Fixtures

Test fixtures may take constructor arguments. Argument values are specified as arguments to the <b>TestFixture</b> attribute. NUnit will construct a separate instance of the fixture for each set of arguments.

Individual fixture instances in a set of parameterized fixtures may be ignored. Set the <b>Ignore</b> named parameter of the reason for ignoring the instance.

Individual fixture instances may be given categories as well. Set the <b>Category</b> named parameter of the attribute to the name of the category or to a comma-separated list of categories.

0.43.3.9 Example

The following test fixture would be instantiated by NUnit three times, passing in each set of arguments to the appropriate constructor. Note that there are three different constructors, matching the data types provided as arguments.

[TestFixture("hello", "hello", "goodbye")]
[TestFixture("zip", "zip")]
[TestFixture(42, 42, 99)]
public class ParameterizedTestFixture
{
    private string eq1;
    private string eq2;
    private string neq;
    
    public ParameterizedTestFixture(string eq1, string eq2, string neq)
    {
        this.eq1 = eq1;
        this.eq2 = eq2;
        this.neq = neq;
    }

    public ParameterizedTestFixture(string eq1, string eq2)
        : this(eq1, eq2, null) { }

    public ParameterizedTestFixture(int eq1, int eq2, int neq)
    {
        this.eq1 = eq1.ToString();
        this.eq2 = eq2.ToString();
        this.neq = neq.ToString();
    }

    [Test]
    public void TestEquality()
    {
        Assert.AreEqual(eq1, eq2);
        if (eq1 != null && eq2 != null)
            Assert.AreEqual(eq1.GetHashCode(), eq2.GetHashCode());
    }

    [Test]
    public void TestInequality()
    {
        Assert.AreNotEqual(eq1, neq);
        if (eq1 != null && neq != null)
            Assert.AreNotEqual(eq1.GetHashCode(), neq.GetHashCode());
    }
}

0.43.3.10 Generic Test Fixtures

You may also use a generic class as a test fixture. In order for NUnit to instantiate the fixture, you must either specify the types to be used as arguments to <b>TestFixtureAttribute</b> or use the named parameter <b>TypeArgs=</b> to specify them. NUnit will construct a separate instance of the fixture for each <b>TestFixtureAttribute</b> you provide.

0.43.3.11 Example

The following test fixture would be instantiated by NUnit twice, once using an ArrayList and once using a List<int>.

[TestFixture(typeof(ArrayList))]
[TestFixture(typeof(List<int>))]
public class IList_Tests<TList> where TList : IList, new()
{
  private IList list;

  [SetUp]
  public void CreateList()
  {
    this.list = new TList();
  }

  [Test]
  public void CanAddToList()
  {
    list.Add(1); list.Add(2); list.Add(3);
    Assert.AreEqual(3, list.Count);
  }
}

0.43.3.12 Generic Test Fixtures with Parameters

If a Generic fixture, uses constructor arguments, there are three approaches to telling NUnit which arguments are type parameters and which are normal constructor parameters.

  1. Specify both sets of parameters as arguments to the <b>TestFixtureAttribute</b>. Leading <b>System.Type</b> arguments are used as type parameters, while any remaining arguments are used to construct the instance. In the following example, this leads to some obvious duplication…

```C# [TestFixture(typeof(double), typeof(int), 100.0, 42)][TestFixture(typeof(int) typeof(double), 42, 100.0)] public class SpecifyBothSetsOfArgs<T1, T2> { T1 t1; T2 t2;

   public SpecifyBothSetsOfArgs(T1 t1, T2 t2)
   {
       this.t1 = t1;
       this.t2 = t2;
   }

   [TestCase(5, 7)]
   public void TestMyArgTypes(T1 t1, T2 t2)
   {
       Assert.That(t1, Is.TypeOf<T1>());
       Assert.That(t2, Is.TypeOf<T2>());
   }

} ```

  1. Specify normal parameters as arguments to <b>TestFixtureAttribute</b> and use the named parameter <b>TypeArgs=</b> to specify the type arguments. Again, for this example, the type info is duplicated, but it is at least more cleanly separated from the normal arguments…

```C# [TestFixture(100.0, 42, TypeArgs=new Type[] {typeof(double), typeof(int) } )][TestFixture(42, 100.0, TypeArgs=new Type[] {typeof(int), typeof(double) } )] public class SpecifyTypeArgsSeparately<T1, T2> { T1 t1; T2 t2;

   public SpecifyTypeArgsSeparately(T1 t1, T2 t2)
   {
       this.t1 = t1;
       this.t2 = t2;
   }

   [TestCase(5, 7)]
   public void TestMyArgTypes(T1 t1, T2 t2)
   {
       Assert.That(t1, Is.TypeOf<T1>());
       Assert.That(t2, Is.TypeOf<T2>());
   }

} ```

  1. In some cases, when the constructor makes use of all the type parameters NUnit may simply be able to deduce them from the arguments provided. That’s the case here and the following is the preferred way to write this example…

```C# [TestFixture(100.0, 42)][TestFixture(42, 100.0)] public class DeduceTypeArgsFromArgs<T1, T2> { T1 t1; T2 t2;

   public DeduceTypeArgsFromArgs(T1 t1, T2 t2)
   {
       this.t1 = t1;
       this.t2 = t2;
   }

   [TestCase(5, 7)]
   public void TestMyArgTypes(T1 t1, T2 t2)
   {
       Assert.That(t1, Is.TypeOf<T1>());
       Assert.That(t2, Is.TypeOf<T2>());
   }

} ```

Although any object implementing <b>IFixtureData</b> may be used to provide specific instance information for a parameterized fixture, NUnit provides the <b>TestFixtureData</b> class for this purpose.

The following example varies the example shown under [[TestFixture Attribute]] by using a TestFixtureSourceAttribute with a data source in a separately defined class.

[TestFixtureSource(typeof(MyFixtureData), "FixtureParms")]
public class ParameterizedTestFixture
{
    private string eq1;
    private string eq2;
    private string neq;

    public ParameterizedTestFixture(string eq1, string eq2, string neq)
    {
        this.eq1 = eq1;
        this.eq2 = eq2;
        this.neq = neq;
    }

    public ParameterizedTestFixture(string eq1, string eq2)
        : this(eq1, eq2, null) { }

    public ParameterizedTestFixture(int eq1, int eq2, int neq)
    {
        this.eq1 = eq1.ToString();
        this.eq2 = eq2.ToString();
        this.neq = neq.ToString();
    }

    [Test]
    public void TestEquality()
    {
        Assert.AreEqual(eq1, eq2);
        if (eq1 != null && eq2 != null)
            Assert.AreEqual(eq1.GetHashCode(), eq2.GetHashCode());
    }

    [Test]
    public void TestInequality()
    {
        Assert.AreNotEqual(eq1, neq);
        if (eq1 != null && neq != null)
            Assert.AreNotEqual(eq1.GetHashCode(), neq.GetHashCode());
    }
}


public class MyFixtureData
{
    public static IEnumerable FixtureParms
    {
        get
        {
            yield return new TestFixtureData( "hello", "hello", "goodbye" );
            yield return new TestFixtureData( "zip", "zip" );
            yield return new TestFixtureData( 42, 42, 99 );
        }
    }  
}

The TextFixtureSetUp attribute has been deprecated, use [[OneTimeSetUp Attribute]] instead.

TestFixtureSourceAttribute is used on a parameterized fixture to identify the source from which the required constructor arguments will be provided. The data is kept separate from the fixture itself and may be used by multiple fixtures. See [[Parameterized Tests]] for a general introduction to tests with arguments.

0.43.3.13 Usage

Consider a test fixture class taking two parameters in its constructor, a string and an int. We can specify the test and it’s data using one of the forms of TestFixtureSourceAttribute:

0.43.3.13.0.1 Form 1 - [TestFixtureSource(string sourceName)]
[TestFixtureSource("FixtureArgs")]
public class MyTestClass
{
    public MyTestClass(string word, int num) { ... }

    ...

    static object [] FixtureArgs = {
        new object[] { "Question", 1 },
        new object[] { "Answer", 42 }
    };
}

The single attribute argument in this form is a string representing the name of the source used to provide arguments for constructing the TestFixture. It has the following characteristics:

  • It may be a field, property or method in the test class.

  • It must be static.

  • It must return an IEnumerable or a type that implements IEnumerable. For fields an array is generally used. For properties and methods, you may return an array or implement your own iterator.

  • The individual items returned by the enumerator must either be object arrays or implement ITestFixtureData. Arguments must be consistent with the fixture constructor.

0.43.3.13.0.2 Form 2 - [TestFixtureSource(Type sourceType, string sourceName)]
[TestFixtureSource(typeof(AnotherClass), "FixtureArgs")]
public class MyTestClass
{
    public MyTestClass(string word, int num) { ... }

    ...
}

class AnotherClass
{
    static object [] FixtureArgs = {
        new object[] { "Question", 1 },
        new object[] { "Answer", 42 }
    };
}

The first argument of the attribute in this form is a Type representing the class that will provide the test fixture data.

The second argument is a string representing the name of the source used to provide test fixtures. It has the following characteristics:

  • It may be a field, property or method in the test class.

  • It must be static.

  • It must return an IEnumerable or a type that implements IEnumerable. For fields an array is generally used. For properties and methods, you may return an array or implement your own iterator.

  • The individual items returned by the enumerator must either be object arrays or implement ITestFixtureData. Arguments must be consistent with the fixture constructor.

0.43.3.13.0.3 Form 3 - [TestFixtureSource(Type sourceType)]
[TestFixtureSource(typeof(FixtureArgs))]
public class MyTestClass
{
    public MyTestClass(string word, int num) { ... }

    ...
}

class FixtureArgs: IEnumerable
{
    public IEnumerator GetEnumerator()
    {
        yield return new object[] { "Question", 1 };
        yield return new object[] { "Answer", 42 };
    }
}

The Type argument in this form represents the class that provides test cases. It must have a default constructor and implement <b>IEnumerable</b>.

The individual items returned by the enumerator must either be object arrays or implement ITestFixtureData. Arguments must be consistent with the fixture constructor.

0.43.3.14 Named Parameters

TestCaseSourceAttribute supports one named parameter:

  • Category is used to assign one or more categories to every test case returned from this source.

0.43.3.15 Test Case Construction

In constructing tests, NUnit uses each item returned by the enumerator as follows:

  1. If it is an object implementing ITestFixtureData, its properties are used to provide the test case. NUnit provides the [[TestFixtureData]] type for this purpose.

  2. If it is an <b>object[]</b>, its members are used to provide the arguments for the method. This is the approach taken in the three examples above.

0.43.3.15.1 Notes:
  1. It is recommended that the SourceType not be the same as the test fixture class. It may be a nested class, however, and probably should be if the data is only used within that fixture.

  2. A generic IEnumerable and IEnumerator may be used but NUnit will actually deal with the underlying IEnumerator in the current release.

  3. The GetEnumerator method may use yield statements or simply return the enumerator for an array or other collection held by the class.

0.43.3.16 Order of Execution

Individual test cases are executed in the order in which NUnit discovers them. This order does <b>not</b> follow the lexical order of the attributes and will often vary between different compilers or different versions of the CLR.

As a result, when <b>TestCaseSourceAttribute</b> appears multiple times on a method or when other data-providing attributes are used in combination with <b>TestCaseSourceAttribute</b>, the order of the test cases is undefined.

However, when a single <b>TestCaseSourceAttribute</b> is used by itself, the order of the tests follows exactly the order in which the test cases are returned from the source.

0.43.3.17 Object Construction

NUnit locates the test cases at the time the tests are loaded. It creates instances of each class used with the third form of the attribute and builds a list of tests to be executed. Each data source class is only created once at this time and is destroyed after all tests are loaded. By design, no communication is possible between the load and execution phases except through the tests that are created.

The TextFixtureTearDown attribute has been deprecated, use [[OneTimeTearDown Attribute]] instead.

The TestOf attribute adds information about the class that is being tested. It can be applied to test fixtures and to tests.

The constructor takes the string name or the type of the class being tested. TestOf can also be specified on a TestFixture or Test attribute.

[TestFixture]
[TestOf(typeof(MyClass)]
public class MyTests
{
 [Test]
 public void Test1() { /* ... */ }

 [Test]
 [TestOf("MySubClass")]
 public void Test2() { /* ... */ }
}

[TestFixture(TestOf = typeof(MyClass))]
public class MyOtherTests
{
 [Test]
 public void Test1() { /* ... */ }

 [Test(TestOf = typeof(MySubClass))]
 public void Test2() { /* ... */ }
}

Note: you can currently only have one TestOf attribute per fixture or test. A Theory is a special type of test, used to verify a general statement about the system under development. Normal tests are <em>example-based</em>. That is, the developer supplies one or more examples of inputs and expected outputs either within the code of the test or - in the case of [[Parameterized Tests]] - as arguments to the test method. A theory, on the other hand, makes a general statement that all of its assertions will pass for all arguments satisfying certain assumptions.

Theories are implemented in NUnit as non-generic methods within a <b>TestFixture</b>, which are annotated with the <b>TheoryAttribute</b>. Theory methods must always have arguments and therefore appears quite similar to [[Parameterized Tests]] at first glance. However, a Theory incorporates additional data sources for its arguments and allows special processing for assumptions about that data. The key difference, though, is that theories make general statements and are more than just a set of examples.

0.43.3.18 Data for Theories

The primary source of data for a <b>Theory</b> is the [[Datapoint Attribute]] or [[DatapointSource Attribute]]. NUnit will use any class members of the required types, which are annotated with one of these attributes, to provide data for each parameter of the Theory. NUnit assembles the values for individual arguments combinatorially to provide test cases for the theory.

In addition to the Datapoint and Datapoints attributes, it is possible to use any of the approaches for supplying data that are recognized on normal parameterized tests. We suggest that this capability not be overused, since it runs counter to the distinction between a test based on examples and a theory. However, it may be useful in order to guarantee that a specific test case is included.

For <b>boolean</b> and <b>enum</b> arguments, NUnit can supply the data without any action by the user. All possible values are supplied to the argument. This feature is disabled if the user supplies any values for the argument.

Note: Because NUnit searches for datapoints based on the type of the argument, generic methods may not currently be used as theories. This limitation may be removed in a future release. See below for a workaround using a generic fixture.

0.43.3.19 Assumptions

The theory itself is responsible for ensuring that all data supplied meets its assumptions. It does this by use of the <b>Assume.That(…)</b> construct, which works just like <b>Assert.That(…)</b> but does not cause a failure. If the assumption is not satisfied for a particular test case, that case returns an Inconclusive result, rather than a Success or Failure.

The overall result of executing a Theory over a set of test cases is determined as follows:

  • If the assumptions are violated for <b>all</b> test cases, then the Theory itself is marked as a failure.

  • If any Assertion fails, the Theory itself fails.

  • If at least <b>some</b> cases pass the stated assumptions, and there are <b>no</b> assertion failures or exceptions, then the Theory passes.

Since the user does not generally care about inconclusive cases under a theory, they are not normally displayed in the Gui. For situations where they are needed - such as debugging - the context menu for the theory provides an option to display them.

0.43.3.20 Example:

In the following example, the Theory SquareRootDefinition verifies that the implementation of square root satisfies the following definition:

Given a non-negative number, the square root of that number is always non-negative and, when multiplied by itself, gives the original number.

public class SqrtTests
{
    [DatapointSource]
    public double[] values = new double[] { 0.0, 1.0, -1.0, 42.0 };

    [Theory]
    public void SquareRootDefinition(double num)
    {
        Assume.That(num >= 0.0);

        double sqrt = Math.Sqrt(num);

        Assert.That(sqrt >= 0.0);
        Assert.That(sqrt * sqrt, Is.EqualTo(num).Within(0.000001));
    }
}

0.43.3.21 Theories in Generic Fixtures

In a generic fixture with Type parameter T individual methods using T as a parameter type or not generic, since T has been resolved to an actual Type in instantiating the fixture instance. You may use such methods as theories and any data of the appropriate type will be used.

[TestFixture(typeof(int))]
[TestFixture(typeof(double))]
public class TheorySampleTestsGeneric<T>
{
    [Datapoint]
    public double[] ArrayDouble1 = { 1.2, 3.4 };
    [Datapoint]
    public double[] ArrayDouble2 = { 5.6, 7.8 };
    [Datapoint]
    public int[] ArrayInt = { 0, 1, 2, 3 };

    [Theory]
    public void TestGenericForArbitraryArray(T[] array)
    {
        Assert.That(array.Length, Is.EqualTo(4));
    }
}

0.43.3.22 See also…

  • [[Datapoint Attribute]]
  • [[DatapointSource Attribute]]
  • [[Parameterized Tests]]

ThrowsConstraint is used to test that some code, represented as a delegate, throws a particular exception. It may be used alone, to merely test the type of constraint, or with an additional constraint to be applied to the exception specified as an argument.

The related [[ThrowsNothingConstraint]] simply asserts that the delegate does not throw an exception.

0.43.3.23 Constructors

ThrowsConstraint(Type expectedType)
ThrowsConstraint<T>()
ThrowsConstraint(Type expectedType, Constraint constraint)
ThrowsConstraint<T>(Constraint constraint)

0.43.3.24 Syntax

Throws.Exception
Throws.TargetInvocationException
Throws.ArgumentException
Throws.ArgumentNullException
Throws.InvalidOperationException
Throws.TypeOf(Type expectedType)
Throws.TypeOf<T>()
Throws.InstanceOf(Type expectedType)
Throws.InstanceOf<T>()
Throws.InnerException

0.43.4 Examples of Use

Assert.That( SomeMethod, Throws.TypeOf<ArgumentException>());
Assert.That( SomeMethod, Throws.Exception.TypeOf<ArgumentException>());
Assert.That( SomeMethod, Throws.TypeOf<ArgumentException>()
                    .With.Property("Parameter").EqualTo("myParam"));
Assert.That( SomeMethod, Throws.ArgumentException );
Assert.That( SomeMethod, Throws.TargetInvocationException
                    .With.InnerException.TypeOf<ArgumentException>());
Assert.That(() => { throw new ArgumentException(); }, Throws.ArgumentException);

<h4>Notes</h4> <ol> <li> <b>Throws.Exception</b> may be followed by further constraints, which are applied to the exception itself as shown in the last two examples above. It may also be used alone to verify that some exception has been thrown, without regard to type. This is not a recommended practice since you should normally know what exception you are expecting. </li> <li> <b>Throws.TypeOf</b> and <b>Throws.InstanceOf</b> are provided as a shorter syntax for this common test. They work exactly like the corresponding forms following <b>Throws.Exception</b>. </li> <li> <b>Throws.TargetInvocationException</b>, <b>Throws.ArgumentException</b> and <b>Throws.InvalidOperationException</b> provide a shortened form for some common exceptions. </li> <li> Used alone, <b>Throws.InnerException</b> simply tests the InnerException value of the thrown exception. More commonly, it will be used in combination with a test for the type of the outer exception as shown in the examples above. </li> </ol> ThrowsNothingConstraint asserts that the delegate passed as its argument does not throw an exception.

0.43.4.1 Constructor

ThrowsNothingConstraint()

0.43.4.2 Syntax

Throws.Nothing

0.43.4.3 Example of Use

Assert.That(() => SomeMethod(actual), Throws.Nothing);

Normally, NUnit simply runs tests and waits for them to terminate - the test may is allowed to run indefinitely. For certain kinds of tests, however, it may be desired to specify a timeout value.

The <b>TimeoutAttribute</b> is used to specify a timeout value in milliseconds for a test case. If the test case runs longer than the time specified it is immediately cancelled and reported as a failure, with a message indicating that the timeout was exceeded.

The specified timeout value covers the test setup and teardown as well as the test method itself. Before and after actions may also be included, depending on where they were specified. Since the timeout may occur during any of these execution phases, no guarantees can be made as to what will be run and any of these phases of execution may be incomplete. Specifically, once a test has timed out, no further attempt is made to execute its teardown methods.

The attribute may also be specified on a fixture or assembly, in which case it indicates the default timeout for any subordinate test cases. When using the console runner, it is also possible to specify a default timeout on the command-line.

0.43.4.4 Example

[Test, Timeout(2000)]
public void PotentiallyLongRunningTest()
{
    ...
}

0.43.4.5 See Also…

  • [[Maxtime Attribute]]

TrueConstraint tests that a value is true.

<h4>Constructor</h4>

TrueConstraint()

<h4>Syntax</h4>

Is.True

<h4>Example of Use</h4>

Assert.That( condition, Is.True );

UniqueItemsConstraint tests that an array, collection or other IEnumerable is composed of unique items with no duplicates.

<h4>Constructor</h4>

UniqueItemsConstraint()

<h4>Syntax</h4>

Is.Unique

<h4>Example of Use</h4>

Assert.That( collection, Is.Unique );

<p>This document assumes you are upgrading to NUnit 3.0 from NUnit 2.6 or later. While it’s possible to upgrade from an earlier version, you will need to take into account many additional changes in NUnit, which are not described here. If this is your situation, be sure to check the release notes for earlier versions of NUnit as well.</p>

<h3>Review Breaking Changes</h3>

<p>The [[Breaking Changes]] page lists missing and changed functionality in NUnit 3.0. You should review this page to see which items are likely to affect your own tests.</p>

<h3>Update Your Code</h3>

<p>In many cases, the items being removed have been deprecated for some time and an alternate approach may already be available in your current release of NUnit. If that is the case, it will probably save you time and effort if you modify the code in your current environment before switching to NUnit 3.0.</p>

<p>For example, NUnit 3.0 no longer supports <b>ExpectedExceptionAttribute</b>. However, preferred alternatives <b>Assert.Throws</b> and the <b>ThrowsConstraint</b> have been available for several years. If you remove the attribute from your tests and use one of the alternatives instead, you can verify that they work in your present environment and they will continue to work after conversion.</b>

<h3>Switch to NUnit 3.0</h3>

<p>Remove references to your old version of NUnit and replace them with references to NUnit 3.0. In the case of NUnitLite executable tests, you will need to reference both the nunit.framework and nunitlite assemblies. Compile your code. It’s possible that you will find compiler errors due to breaking changes in NUnit 3.0, which you missed in the prior step. Make the necessary changes until it all works.</p>

<h3>Make the Tests Pass</h3>

<p>Hopefully, you aren’t converting tests that were not passing when you started! If all goes well, they will continue to pass under NUnit 3.0. If not, investigate each case and make necessary changes. If something isn’t working as advertised, please let us know.

0.43.4.6 Usage Notes

  • [[Runtime Selection]]
  • [[Platform Selection]]
  • [[Assembly Isolation]]
  • [[Configuration Files]]
  • [[XML Formats]]
  • [[Visual Studio Support]]
  • [[SetUp and TearDown]]
  • [[Parameterized Tests]]
  • [[Addin Replacement in the Framework]]
  • [[Counting Tests]]
  • [[Parallel Test Execution]]

ValueSourceAttribute is used on individual parameters of a test method to identify a named source for the argument values to be supplied. The attribute has two public constructors.

ValueSourceAttribute(Type sourceType, string sourceName);
ValueSourceAttribute(string sourceName);

If <b>sourceType</b> is specified, it represents the class that provides the data. It must have a default constructor.

If <b>sourceType</b> is not specified, the class containing the test method is used.

The <b>sourceName</b>, represents the name of the source that will provide the arguments. It should have the following characteristics: * It may be a field, a non-indexed property or a method taking no arguments. * It must be a static member. * It must return an IEnumerable or a type that implements IEnumerable. * The individual items returned from the enumerator must be compatible with the type of the parameter on which the attribute appears.

0.43.4.7 Order of Execution

Individual test cases are executed in the order in which NUnit discovers them. This order does <b>not</b> follow the lexical order of the attributes and will often vary between different compilers or different versions of the CLR.

As a result, when <b>ValueSourceAttribute</b> appears multiple times on a parameter or when other data-providing attributes are used in combination with <b>ValueSourceAttribute</b>, the order of the arguments is undefined.

However, when a single <b>ValueSourceAttribute</b> is used by itself, the order of the arguments follows exactly the order in which the data is returned from the source.

0.43.4.8 Note on Object Construction

NUnit locates the test cases at the time the tests are loaded, creates instances of each class with non-static sources and builds a list of tests to be executed. Each source object is only created once at this time and is destroyed after all tests are loaded.

If the data source is in the test fixture itself, the object is created using the appropriate constructor for the fixture parameters provided on the <b>TestFixtureAttribute</b>, or the default constructor if no parameters were specified. Since this object is destroyed before the tests are run, no communication is possible between these two phases - or between different runs - except through the parameters themselves.

The <b>ValuesAttribute</b> is used to specify a set of values to be provided for an individual parameter of a parameterized test method. Since NUnit combines the data provided for each parameter into a set of test cases, data must be provided for all parameters if it is provided for any of them.

By default, NUnit creates test cases from all possible combinations of the data values provided on parameters - the combinatorial approach. This default may be modified by use of specific attributes on the test method itself.

0.43.4.9 Example

[Test]
public void MyTest([Values(1,2,3)] int x, [Values("A","B")] string s)
{
    ...
}

The above test will be executed six times, as follows:

MyTest(1, "A")
MyTest(1, "B")
MyTest(2, "A")
MyTest(2, "B")
MyTest(3, "A")
MyTest(3, "B")

0.43.4.10 Values with Enum or Boolean

The values attribute works in a special way with Enums and Boolean parameters.

When used without any arguments, the <b>[Values]</b> attribute on an enum parameter will automatically include all possible values of the enumeration.

[Test]
public void MyEnumTest([Values]MyEnumType myEnumArgument)
{
    //...
}

There is the same support for Boolean values. Add the <b>[Values]</b> attribute to a bool and the method will be run with true and false.

[Test]
public void MyBoolTest([Values]bool value)
{
    //...
}

0.43.4.11 See also…

  • [[Range Attribute]]
  • [[Random Attribute]]
  • [[Sequential Attribute]]
  • [[Combinatorial Attribute]]
  • [[Pairwise Attribute]]

<h2>DRAFT - Out of Date</h2>

NUnit supports the Visual Studio project and solution format, which are also used by a number of other IDEs.

0.43.4.12 Running From Within Visual Studio

The most convenient way to do this is to set up a custom tool entry specifying the path to NUnit as the command. For a VS2003 C# project, you can use $(TargetPath) for the arguments and $(TargetDir) for the initial directory.

With Visual Studio VS2005 this becomes a bit harder, because that release changed the meaning of the ‘Target’ macros so they now point to the intermediate ‘obj’ directories rather than the final output in one of the ‘bin’ directories. Here are some alternatives that work in both versions:

<ul> <li><b>(ProjectDir)(ProjectFileName)</b> to open the VS Project rather than the assembly. If you use this approach, be sure to rename your config file accordingly and put it in the same directory as the VS project file. <li><b>(ProjectDir)bin/Debug/(TargetName)$(TargetExt)</b> to run the assembly directly. Note that this requires hard-coding part of the path, including the configuration. </ul>

If you would like to debug your tests, use the Visual Studio Debug | Processes… menu item to attach to NUnit after starting it and set breakpoints in your test code as desired before running the tests.

0.43.4.13 Opening Visual Studio Projects

When Visual Studio support is enabled, the File Open dialog displays the following supported Visual Studio project types: C#, VB.Net, J# and C++. The project file is read and the configurations and output assembly locations are identified. Since the project files do not contain information about the most recently opened configuration, the output assembly for the first configuration found (usually Debug) is loaded in the GUI. The tree shows the project as the toplevel node with the assembly shown as its descendant.

When tests are run for a Visual studio project, they run just as if the output assembly had been loaded with one exception. The default location for the config file is the directory containing the project file and it’s default name is the same as the project file with an extension of .config. For example, the following command would load the tests in the nunit.tests assembly using the configuration file nunit.tests.dll.config located in the same directory as the dll. <pre class=“programtext”> nunit.exe nunit.tests.dll</pre> On the other hand, the following command would load the tests using the configuration file nunit.tests.config located in the same directory as the csproj file. <pre class=“programtext”> nunit.exe nunit.tests.csproj</pre> The same consideration applies to running tests using the console runner.

0.43.4.14 Opening Visual Studio Solutions

When Visual Studio support is enabled, solution files may be opened as well. All the output assemblies from contained projects of the types supported will be loaded in the tree. In the case where all contained projects are located in the subdirectories beneath the solution, it will be possible to load and run tests using this method directly.

When a solution contains projects located elsewhere in the file system, it may not be possible to run the tests - although the solution will generally load without problem. In this case, the Project Editor should be use to modify and save the NUnit test project so that there is all referenced assemblies are located in or beneath the application base directory.

The NUnit Test Engine uses a plugin architecture that allows users and third parties to add new functionality to the engine. The extensibility model defines a number of Extension Points to which Extensions may be added. This page gives general information that applies to all types of extensions you may want to write. The individual pages for each type of extension give specific details. For a detailed description of the engine extensibility architecture, see [[Engine Extensibility]].

0.44 Extension Attribute

Every extension is implemented by a class with specific characteristics: * Has a default constructor so that NUnit can create an instance. * Implements some interface that varies according to the particular extension. * Is marked with the ExtensionAttribute so that NUnit can recognize it as an extension.

The code for a typical extension might look like this.

    [Extension]
    public class MyExtension : ISomeInterface // Depending on the extension point
    {
        // Your code here
    }

The attribute has four named properties, all optional:

  • Path This is a string that uniquely identifies the extension point to which the extension applies. It is only rarely needed, since NUnit can usually deduce the type of extension based on what interface is implemented by the extension class.

  • Description An optional description of what the extension does.

  • Enabled A boolean flag indicating whether the extension is enabled. This defaults to true. The setting is used by advanced extensions with functionality that is turned on and off depending on user input.

  • EngineVersion The minimum engine version supported by the extension. Although optional, you should use this property if your extension will not work with all versions of the engine. If you don’t use it and your extension requires engine services that are not present, then it might throw an exception or cause other errors.

Note: Only engine versions 3.4 or later check this property. The only way to avoid errors in the case of lower engine versions is to not install such extensions.

0.45 ExtensionPropertyAttribute

Using only the ExtensionAttribute, NUnit would have to create instances of every extension in order to ask questions like “What file extensions do you support.” This would mean loading many potentially unneeded assemblies.

The ExtensionPropertyAttribute avoids the problem. NUnit’s own extension for loading NUnit projects is a good example:

    [Extension]
    [ExtensionProperty("FileExtension", ".nunit")]
    public class NUnitProjectLoader : IProjectLoader
    {
        ...
    }

By use of the ExtensionPropertyAttribute NUnit is able to postpone loading the extension until the user actually uses a file of type .nunit. If the extension is never needed, then it won’t be loaded at all. For information about what properties are used by each extension point, see the individual pages for each type of extension.

Note: Extensions are usually created each in their own assembly for efficiency. It’s possible to have several related extensions in the same assembly, but they will all be loaded into memory as soon as one is used.

0.46 Kinds of Extensions

As of version 3.4, the NUnit Engine supports four types of extensions. The individual pages for each type give specific details on implementing each of them.

  • [[Project Loaders]]
  • [[Result Writers]]
  • [[Framework Drivers]]
  • [[Event Listeners]]

0.47 Installing Extensions

Once an extension is written and compiled, it has to be placed somewhere such that NUnit will find it.

This section is under construction. For now see https://github.com/nunit/docs/wiki/Engine-Extensibility#locating-addins ###DRAFT Use of XML in NUnit 2.x is somewhat limited and is only used in external files. NUnit 3.0 uses XML in several of its interfaces for communicating between its three architectural layers.

The samples given here represent the current state of the application and are subject to change. Watch this page for new information as development continues.

0.47.0.1 NUnit Projects

An NUnit project is stored as a file with the extension .nunit and describes one or more test assemblies to be executed, together with certain parameters used in running them. No schema is used for this file.

  • [[Sample NUnit Project|http://nunit.org/files/nunit_project_25.txt]]

Currently, the format is the same as for NUnit 2.x but it is likely to change as the project proceeds. Note that NUnitLite does not use or recognize NUnit projects, but only assemblies.

For details of the file contents, see [[NUnit Project XML Format]]

0.47.0.2 NUnit Settings

The NUnitSettings30.xml file holds default settings for NUnit 3.0. No schema is used for this file. The format is the same as for 2.x but new settings may be added and the names of certain keys are likely to change.

  • [[Sample NUnit Settings File|http://nunit.org/files/sample_nunitsettings_file.txt]]
  • List of [[NUnit Setting Names]]

0.47.0.3 Test Results

The results of an test run are saved in a file with the default name of TestResult.xml. The schema of this file is being modified substantially for NUnit 3.0.

  • [[Sample|http://nunit.org/files/testresult_30.txt]]
  • Schema not yet available

For details of the file layout see [[Test Result XML Format]].

0.47.0.4 V2 Test Results

Optionally, NUnit 3.0 is able to save results in the NUnit 2.x format for use with CI servers that do not yet understand the new format.

  • [[Sample|http://nunit.org/files/testresult_25.txt]]
  • [[Schema|http://nunit.org/files/testresult_schema_25.txt]]

0.47.0.5 Test Representation

When using the -explore option, a list of tests without results is returned. The format used is simply a subset of the test result format, without the result information.

See [[Test Result XML Format]] for details.

0.47.0.6 Test Filters

Information about which tests to run is provided to the framework using a test filter, represented as an XML fragment. See [Test Filters] for details.

0.47.0.7 Test Packages

0.47.0.8 Progress Reports

As a test run progresses, individual test run results are sent as xml fragments from the result file under construction. A runner can, in fact, construct a copy of the result file incrementally using these fragments if desired.

0.47.0.9 VisualState

The Gui runner uses a file with the suffix .VisualState.xml to save the current visual state of a project so that it may be restored upon re-opening. This file is private to the Gui but is included here for completeness. The current format is identical to that used in NUnit 2.x.

  • [[Sample Visual State File|http://nunit.org/files/sample_visual_state_25.txt]]

XmlSerializableConstraint tests whether an object is serializable in XML format.

<h4>Constructor</h4>

XmlSerializableConstraint()

<h4>Syntax</h4>

Is.XmlSerializable

<h4>Examples of Use</h4>

Assert.That( someObject, Is.XmlSerializable ));

0.47.0.10 See also…

  • [[BinarySerializableConstraint]] Copyright (c) 2015 NUnit Software ###User Docs

  • [[NUnit|NUnit-Documentation]]
  • [[Release Notes]]<br/>
  • [[License]]<br/>
  • Getting Started
    • [[Installation]]
    • [[Upgrading]]
    • [[Samples]]
    • [[Breaking Changes]]
  • Writing Tests
  • Running Tests
    • [Console Runner]
    • [[Console Command Line]]
    • [[NUnit Test Projects]]
    • [[NUnitLite Runner]]
  • Extending NUnit
    • [[Framework Extensibility]]
    • [[Writing Engine Extensions]]
  • Technical Notes
  • [[NUnit Gui]]
  • [[Project Editor]]
  • Visual Studio
  • [[Test Adapter|Visual-Studio-Test-Adapter]]
  • [[Test Generator|Visual-Studio-Test-Generator]]
  • [[User FAQ]]

0.47.1 Developer Docs

  • [[NUnit Vision]]
  • [[NUnit Roadmap]]
  • [[Team Practices]]
  • [[Specifications]]
  • [[Developer FAQ]]

0.48 Configuration Editor

The Configuration Editor is displayed using the Project | Configuration | Edit… menu item and supports the following operations:

<img align=“right” src=“project-editor/images/configEditor.jpg”>

0.48.1 Remove

Remove the selected configuration. If it was the active config, then the next one in the list is made active.

0.48.2 Rename

Rename the selected configuration.

0.48.3 Add…

Add a new configuration. The Add Configuration dialog allows specifying an existing configuration to use as a template.

0.48.4 Make Active

Makes the selected configuration active.

0.48.5 Close

Exits the configuration editor ## Project Editor

The NUnit Project editor is a graphical interface for editing [[NUnit Test Projects]]. The editor provides two main views of the project: a property-oriented view, and an xml view, which allows direct editing of the .nunit file.

0.49 Property View

This view consists of a common area and two tabs, as seen in the image below.

Screenshot of Property View
Screenshot of Property View

0.49.1 Common Area

The common area of the Project Editor contains information pertaining to the project as a whole. Information that applies to a particular configuration is displayed in the General and Assemblies tabs.

0.49.1.1 Project Path

This label shows the full path to the project file. In the case of a wrapper project, the path is set to the same directory as the assembly that was initially opened.

0.49.1.2 Application Base

This TextBox allows the user to change the project AppBase, which defaults to the directory of the project file. The button to the right of the TextBox allows the user to browse and select a directory.

0.49.1.3 Process Model

This dropdown list allows you to specify how operating system processes are used in loading and running the tests in this project. Four settings are defined:

  • The Default setting refers to the option selected by the user on the Assembly Isolation page of the NUnit Settings Dialog.
  • Single means that tests are run in a test domain in the same process as NUnit. This is the way previous versions of NUnit ran tests.
  • Separate means that all the tests are run in a separate process that NUnit creates.
  • Multiple means that NUnit will create a separate process for each test assembly in the project and run its tests there.

0.49.1.4 Domain Usage

This dropdown list allows you to specify how tests are loaded into AppDomains by NUnit. Three settings are defined:

  • The Default setting refers to the option selected by the user on the Assembly Isolation page of the NUnit Settings Dialog.
  • Single means that all tests will run in a single test domain created by NUnit.
  • Multiple means that each test assembly is loaded into a separate AppDomain. This setting is not available when Multiple processes are selected in the Process Model dropdown.

0.49.1.5 Configuration

This dropdown list allows you to select the particular configuration within a project that is displayed in the bottom part of the dialog.

0.49.1.6 Edit Configs…

This button opens the [Configuration Editor], which allows you to add, delete or rename configs and set the active configuration.

0.49.2 General Tab

The General tab allows setting a number of options pertaining to the selected configuration, all of which will be stored in the NUnit project file as attributes of the <config> xml node.

0.49.2.1 Runtime

This dropdown allows you to select a particular runtime framework to be used for loading and running tests under the current configuration. Currently, only Microsoft .NET and Mono are supported. If Any is selected, the tests will be run under the same runtime that NUnit itself is currently using.

0.49.2.2 Version

This ComboBox allows you to select the particular version of the runtime framework to be used for loading and running tests under the current configuration. The dropdown list contains entries for

  • Default
  • 1.0
  • 1.1
  • 2.0
  • 4.0

If you select “Default” the assemblies in the project are examined to determine the version that is required. See [[Runtime Selection]] for more information on how NUnit selects the version to be used.

In special cases, you may wish to enter a version number that is not listed in the list box. You may specify the version using two, three or four components. The version you provide will be saved as you enter it. Leaving the text box blank is equivalent to selecting “Default.”

Note: Running tests under a different runtime or version from the one that NUnit is currently using will force them to run in a separate process.

Note: To conform with normal usage, specifying Mono as the runtime with “1.0” as the version results in use of the Mono 1.0 profile, equating to version 1.1.4322.

0.49.2.3 ApplicationBase

The ApplicationBase defaults to the directory containing the project file.

0.49.2.4 Configuration File Name

The configuration file defaults to the name of the test project with the extension changed from .nunit to .config. The user may substitute another name.

0.49.2.5 PrivateBinPath

By default, the PrivateBinPath is generated from the assembly locations specified on the Assemblies Tab. For those applications requiring a different level of control, it may be specified manually or using this editor or placed in the configuration file.

0.49.3 Assemblies Tab

The assemblies tab contains the list of assemblies that form part of this test project.

Note: Although the dialog shows the location of assemblies as absolute paths, they are always persisted in the NUnit project file as paths relative to the application base. This allows moving projects as a whole to a different directory location.

Screenshot of Assemblies Tab
Screenshot of Assemblies Tab

0.49.3.1 Add

Opens a dialog allowing adding an assembly to this configuration. If Visual Studio support is enabled, you may also select and add a VS project.

0.49.3.2 Remove

After confirmation, removes the selected assembly from this configuration.

0.49.3.3 Assembly Path

This text box displays the full path to the selected assembly. You may edit the contents to change the path to the assembly.

0.50 XML View

This view simply displays the XML from the project file, as seen here. You may edit the XML directly.

Screenshot of XML View
Screenshot of XML View

Note: The XML editor is somewhat primitive. Errors in XML formatting are caught and an error message displayed. However, the values of attributes are not validated as they are in the property-based view and it is possible to create a project file, which NUnit is unable to load. Improvements, including intellisense and better error handling, are planned in future versions of the Project Editor. Copyright (c) 2016 NUnit Software ###User Docs

  • [[NUnit|NUnit-Documentation]]
  • [[NUnit Gui]]
  • Project Editor
  • [[Main Dialog|Project-Editor]]
  • [Configuration Editor]
  • Visual Studio
  • [[Test Adapter|Visual-Studio-Test-Adapter]]
  • [[Test Generator|Visual-Studio-Test-Generator]]
  • [[User FAQ]]

0.50.1 Developer Docs

  • [[NUnit Vision]]
  • [[NUnit Roadmap]]
  • [[Team Practices]]
  • [[Specifications]]
  • [[Developer FAQ]]

This is an extension for Visual Studio 2015 that extends the test functionality to allow you to create unit tests and IntelliTests. This extension extends the built in test generator functionality allowing developers to generate tests using NUnit 2.6.x or NUnit 3.0.

Please note that IntelliTest is only available in Visual Studio Enterprise edition. Other versions of Visual Studio only have the Create Unit Tests menu option.

0.50.2 Download

You can download this extension in Visual Studio using Tools | Extensions and Updates… and searching for Test Generator NUnit Extension. You can also download from the Visual Studio Gallery or from the GitHub Releases Page.

0.50.3 How to Use

For more information on using IntelliTest and this extension, please see the Microsoft documentation.

0.50.3.1 Right-Click to Create Tests

Right-Click to Create Tests
Right-Click to Create Tests

0.50.3.2 Select NUnit from the Test Framework dropdown

Selecting NUnit
Selecting NUnit
                      Copyright (c) 2016 NUnit Software

0.50.4 User Docs

  • [[NUnit|NUnit-Documentation]]
  • [[NUnit Gui]]
  • [[Project Editor]]
  • Visual Studio
  • [[Test Adapter|Visual-Studio-Test-Adapter]]
  • Test Generator
  • [[User FAQ]]

0.50.5 Developer Docs

  • [[NUnit Vision]]
  • [[NUnit Roadmap]]
  • [[Team Practices]]
  • [[Specifications]]
  • [[Developer FAQ]]

0.50.6 NUnit Documentation Wiki

This wiki contains the documentation for all active NUnit projects as well as developer documentation for those working on NUnit or wishing to do so. Some sections are not yet complete.

0.50.6.1 User Documentation

  • [[NUnit|NUnit-Documentation]] covers NUnit itself, including the framework, engine and console runner.
  • [[NUnit Gui]] covers the new NUnit 3.0 GUI.
  • [[Project Editor]] covers the Windows program for creating and editing .nunit project files.
  • [[Test Adapter|Visual-Studio-Test-Adapter]] covers the NUnit Visual Studio test adapters for both NUnit V2 and V3.
  • [[Test Generator|Visual-Studio-Test-Generator]] covers the NUnit Visual Studio test generator for both NUnit V2 and V3.

0.50.6.2 Developer Documentation

  • [[NUnit Vision]] covers our vision for NUnit and its future.
  • [[NUnit Roadmap]] shows our plans for coming releases.
  • [[Team Practices]] describes how we work on NUnit and the standards we follow.
  • [[Specifications]] are descriptions of features we plan to add.
  • [[Developer FAQ]] contains basic info for those who want to work on NUnit.

We use the GitHub issue tracker to keep track of our ongoing work. We supplement GitHub’s native issue handling by using ZenHub, which provides a number of additional features.

0.50.7 Workflow

We use a ZenHub Board with a standard set of piplelines to track the flow of work in each repo. Our standard set of pipleline columns are slightly different from the ZenHub default:

  • New Issues All issues start as new. We like to review them as quickly as possible. Some issues can be immediately closed while others need to be categorized for further work. We usually assign at least an is and a priority label (see below) to every issue and then move it to either the Discussion or the Backlog column.

  • Discussion These are items that require some discussion, either about whether we want to do them or how they should be implemented. Some items here may require confirmation or design as well.

  • Backlog This is our product backlog consisting of all the issues we want to get done now or in the future. In the past, we have often put things we didn’t really want to do in our Backlog, assigning them a low priority. While this avoids having to say no to the requestor, it doesn’t do them any real service. Putting an issue into the Backlog should mean that it will eventually make it to a release.

  • To Do These are things we are ready to get working on right now and represent a suggestion to developers as to what to choose to work on next. Most but not all of them will be scheduled for the next milestone. This is a good place to keep issues that are blocking other work, so as to encourage developers to get the issue done.

Note: Since the number of people actively working on a given project varies, we can’t establish a fixed number of items to keep in this column. The project lead should try to keep it to a reasonable size, generally no more than twice the number of active developers. If it seems to be growing too rapidly, some of the issues should be moved to the backlog.

  • In Progress These are issues that somebody is already working on. That person should be shown as assigned to the issue and would normally be the one to move the issue into this column.

  • Done When all work is done and the person doing it feels it is ready to be merged, the issue should be moved to this column. It remains there while comments are discussed and small changes are made. If the comments will lead to significant rework, then move the issue back to In Progress.

  • Closed Issues go here automatically when they are closed.

0.50.8 Issue Assignment

Normally, committers self-assign items they want to work on. Please don’t assign something to yourself and then let it sit. As a matter of courtesy, nobody else will start working on something that is assigned to you, so the result is that the work won’t get done.

GitHub won’t let non-committers assign issues to themselves (or anyone) so if a contributor wants to work on an issue, they should post a comment to that effect. One of the committers will make the assignment. In many cases, the assignment will be shared between the contributor and a committer who is providing mentoring for them.

If an issue that is assigned to you then you should do what needs to be done. For example, if it’s in the discussion column, then it’s up to you to make sure the discussion happens and progresses to a conclusion. If it’s marked confirm then you should confirm it. If it’s marked design then you should do the design and get it reviewed. And, of course, if it’s ready for implementation you should do that!

0.50.9 Issue Labels

We try to use a standard set of labels across all the NUnit repositories. In some cases, an individual project may not yet have been converted to use these labels, but we expect to do so soon.

That said, don’t stress about whether something is a bug or an enhancement, normal versus low priority, etc. Just pick one. If things change later, the label can be changed as well. This is only intended to help us organize a relatively large number of issues, not to give us extra work.

0.50.9.1 What it is

Labels starting with is: indicate the nature of the issue. Only one should be used, based on the judgement of the committer who assigns the label. If there is no is: label, then we presumably don’t know what the item is and should not be working on it! * is:bug Something that isn’t working as designed. * is:docs Solely pertaining to the documentation or sample code. * is:enhancement An addition or improvement to an existing feature. * is:feature An entirely new feature. * is:idea An idea about something we might do. We discuss these until they are either dropped or turned into a feature or enhancement we can work on. * is:question Just a question - we discourage these as issues but they do happen. * is:build Something to do with how we build the software, scripts, etc. * is:refactor What it says: refactoring that is needed.

0.50.9.2 Priority

Labels starting with pri: indicate the priority of an issue. Pick just one, please. Priority may, of course, change over time, as items become more or less important to us. If no priority is assigned, we shouldn’t be working on it. * pri:critical Should only apply to bugs, which need to be fixed immediately, dropping everything else. At times, we will even speed up the release cycle due to a critical bug. * pri:high High priority - implement as soon as possible. * pri:normal Standard priority - implement when we can. * pri:low Low priority - implement later or not at all.

0.50.9.3 Close Reason

Labels starting with closed: indicate the status of the bug at closing and should only appear on closed bugs. Please remember to apply one of these when closing a bug as it makes it easier to review the list of closed bugs without opening each one to see what the disposition was. * closed:done The work called for is done, i.e. the bug is fixed or the feature/enhancement is implemented. * closed:duplicate The issue is a duplicate of one that we are working. A comment should indicate the issue number. * closed:notabug The issue (generally a bug) is not valid or the feature already exists. There should be an explanatory comment. * closed:norepro While the issue (generally a bug) may exist on the user’s system, we have tried and are unable to reproduce it. If somebody later figures out a repro, the issue can be reopened. * closed:wontfix The issue is possibly valid but we don’t intend to implement it. It may be out of scope for the project or inconsistent with the values and priorities of the project. There should be an explanatory comment.

0.50.9.4 Other Labels

  • confirm Somebody should verify that the issue actually exists and then remove the label. In some cases, a bug may have been reported against an older version of NUnit and needs to be checked out using the current code.
  • blocked The issue cannot be worked on until something else happens, external to the project. There should be a comment on the issue indicating what that something is.
  • design Some design decisions need to be made before this can really be worked on. Sometimes this label may be applied before anything happens and other times the work may have started but reached a point where design decisions need to be made involving others in the team.
  • easyfix Indicates an issue that might be a good place for a new contributor to start. Whoever adds the label should couple it with a comment suggesting what code to look at and a general approach to working the issue.

0.50.10 Epics

Where appropriate, we make use of the ZenHub Epic feature to create issues that include a number of sub-issues. For example, when we split the original nunit repository into separate framework and engine repositories, we created an epic that included a number of separate issues that had to be completed in order to accomplish the split smoothly.

0.50.11 Milestones

We use GitHub Milestones to represent future releases. Generally milestones are created by the project or team leader, who also decides what issues to place into the milestone. Our practice is to only schedule a limited number of key features in advance for each milestone, rather than trying to “fill” the milestone with the amount of work we think can be completed. Other work is only added to the next milestone after it has been completed.

We do it this way because the amount of time developers have to spend on our projects can vary considerably over the course of time. We never know how much work will be done in a given period. Since we can’t predict both the timing and the content of each release, we have chosen to hold the time constant and vary the content.

The NUnit Console/Engine currently ships with the following extensions,

  • NUnit.Extension.NUnitProjectLoader
  • NUnit.Extension.VSProjectLoader
  • NUnit.Extension.NUnitV2ResultWriter
  • NUnit.Extension.NUnitV2Driver
  • NUnit.Extension.TeamCityEventListener

All but the TeamCityEventListener are built and shipped by the NUnit team. These extensions must be built and released before building and releasing the Console/Engine, but only if they are changed and a release is planned. For the 3.5 release, all extensions will be built and released with the console. Future releases of each extension will be on an as-needed basis and the version numbers of the extensions and the console/engine will diverge over time.

0.50.11.1 Create a Release Branch

All work on releases should be done on a branch.

  1. Fetch and pull latest from master
  2. Create a branch in the form release-3.5
  3. As you make the changes below, push the branch to GitHub and create a Pull Request to allow other team members to review your changes.
  4. Do not merge this branch/PR, we will create a separate PR to merge the changes back into master.

0.50.11.2 Make Sure it Works!

  1. Close all instances of Visual Studio or other IDE to ensure that no changes are left unsaved.

  2. Do a clean build and run all the tests on Windows. You may use the command below or three separate commands if preferred. If you encounter errors at any stage, you’re not actually ready to release!

    build.cmd -Target=Clean build.cmd -Target=Test

  3. Repeat the build on a Linux system, if available. If this is not possible, be sure to scrutinize the results from the Travis CI build carefully. On Linux, you may use the command

    ./build -Target=Test

  4. Make sure that the most recent commits of master passed all tests in the CI builds. Check the builds on both Travis and Appveyor.

0.50.11.3 Review Milestone Status

  1. Check the milestone for the current release to see that there are no open issues. Any issues that are not going to be in this release should be moved to a future milestone. This may be the time to create the next milestone.

  2. Make sure that completed issues are marked with the appropriate ‘closed’ label depending on disposition. The release notes will end up reflecting issues marked closed:done.

  3. Check all future open milestones for completed issues. Anything that is completed will be included in this release so change its milestone to the current release.

0.50.11.4 Check Versioning

AssemblyVersion and AssemblyFileVersion are set in src\extension\Properties\AssemblyInfo.cs and should match the version in build.cake. These values are normally incremented after a release, but should be checked.

The copyright year in all the source files is only updated as they are changed, but the copyright in the [assembly: AssemblyCopyright("...")] should be updated to the year of the release.

If necessary, update the year in the general copyright notice LICENSE.txt. The .nuspec files in solution root contains a copyright line, which should also be updated.

Notices at the top of each source code file are only updated when copyrightable changes are made to the file, not at the time of release.

0.50.11.6 Push All Changes

Make sure the release branch and any changes are pushed to GitHub, reviewed in a PR and all CI servers are passing.

0.50.11.7 Creating the Release

  1. Clear the package directory to avoid confusion:

    erase package\*

This is not absolutely required, but will be helpful if you have other release packages present in the directory.

  1. You should be working on the release branch. Do a pull to make sure you have everything up to date. If changes of any significance were merged, you should test again before creating the release.

  2. Ensure that the release build is up to date. If you have any doubt whether the latest code changes have actually been built, do a clean build. If the build is up to date you may skip this step.

    build -Target Build

  3. Create the packages by running:

    build -Target Package

  4. Verify that the correct package has been created in the package sub-directory.

  • NUnit..Extension.{NAME}.{VERSION}.nupkg

0.51 Testing the Release

Open the NuGet package from the package sub-directory using NuGet Package Explorer. Inspect the package metadata for errors and check that the tools folder contains the extension dll and any dependent assemblies.

Next install the extension into a project with the NUnit.ConsoleRunner package and make sure you can run tests. If you are releasing the V2 driver, run NUnit 2 tests. If you are releasing the V2 Result Writer, write out V2 results.

0.52 Archiving the Release

Packages are archived on nunit.org in the downloads directory. Create a new subfolder under downloads/extensions/v3/EXTENSION_NAME for the release. Upload all the package files into that directory.

0.53 Publishing the Release

0.53.0.1 Github

  1. Log onto Github and go to the extension repository.

  2. Select Releases and then click on the “Draft a new release” button.

  3. Enter a tag to be used for the release in the format v3.x.y Select your release branch as the target for the tag.

  4. Enter a title for the release, like NUnit Project Loader Extension 3.5 release. If you type ‘N’ you’ll get some hints.

  5. Add a description of the release. It will be easier if you have prepared this in advance and can just paste it in.

  6. If this is an Alpha or Beta release, check the box that indicates a pre-release.

  7. Upload the nupkg package you created.

  8. Click the “Publish release” button to publish the release on Github.

0.53.0.2 NuGet

  1. Sign on to Nuget.org.

  2. Select Upload package.

  3. Browse to the location of the nupkg you created and upload it.

  4. Verify that the info is correct and click the “Submit” button.

0.53.0.3 Merge into Master

  1. Close your release Pull Request
  2. Create a new branch off of your release branch
  3. Increment the version in the various locations. For example 3.5 -> 3.6. See the Update Package Versions section above.
  4. Push your changes to GitHub
  5. Create a pull request from your branch making sure it is based off master

0.53.0.4 Notify Users

Send notifications to the mailing list and twitter if the changes to the extension warrant it.

0.53.0.5 Close the Milestone

The milestone representing this release should be closed at this time. TODO: This documentation was copied from the original packaging info and needs to be updated for packaging just the Console and Engine.

This note describes how to create release packages for the NUnit console runner and test engine. Currently, all the builds and packaging must be done on a single Windows machine. This is likely to change in the future as we add more platforms.

0.54 Software Prerequisites

Various software combinations and environments may be used to build the NUnit 3 console runner and engine. Our standard environment is Visual Studio 2015 Community Edition but it may be built with Vs 2012, 2013 or 2015 as well as MonoDevelop.

0.55 Preparing for Release

0.55.0.1 Create a Release Branch

All work on releases should be done on a branch.

  1. Fetch and pull latest from master
  2. Create a branch in the form release/3.2.0
  3. As you make the changes below, push the branch to GitHub and create a Pull Request to allow other team members to review your changes.
  4. Do not merge this branch/PR, we will create a separate PR to merge the changes back into master.

0.55.0.2 Make Sure it Works!

  1. Close all instances of Visual Studio or other IDE to ensure that no changes are left unsaved.

  2. Do a clean build and run all the tests on Windows. You may use the command below or three separate commands if preferred. If you encounter errors at any stage, you’re not actually ready to release!

    build.cmd -Target=Clean build.cmd -Target=Test

  3. Repeat the build on a Linux system, if available. If this is not possible, be sure to scrutinize the results from the Travis CI build carefully. On Linux, you may use the command

    ./build -Target=Test

  4. Make sure that the most recent commits of master passed all tests in the CI builds. Check the builds on both Travis and Appveyor. Check on TeamCity once we get that build working again.

0.55.0.3 Review Milestone Status

  1. Check the milestone for the current release to see that there are no open issues. Any issues that are not going to be in this release should be moved to a future milestone. This may be the time to create the next milestone.

  2. Make sure that completed issues are marked with the appropriate ‘closed’ label depending on disposition. The release notes will end up reflecting issues marked closed:done.

  3. Check all future open milestones for completed issues. Anything that is completed will be included in this release so change its milestone to the current release.

0.55.0.4 Check Assembly Versioning

AssemblyVersion and AssemblyFileVersion are set separately for the framework, engine, engine api and console runner. Each is kept in a separate file and they may be updated separately. Using the 3.4.1 release as an example, version information would be set as follows:

  Component             | File to Update      | AssemblyVersion | AssemblyFileVersion 
  --------------------- | ------------------- | --------------- | -------------------
  Engine                | EngineVersion.cs    |     3.4.1.0     |      3.4.1.0
  Engine API            | EngineApiVersion.cs |     3.0.0.0     |      3.4.1.0
  Console               | ConsoleVersion.cs   |     3.4.1.0     |      3.4.1.0
0.55.0.4.1 Notes:
  1. The Engine API AssemblyVersion is fixed and will not be changed unless it becomes necessary to modify the API in a non-additive manner.

  2. These values will normally already be correct for the release, since they should have been set immediately following the prior release.

The copyright year in all the source files is only updated as they are changed, but the copyright in the [assembly: AssemblyCopyright("...")] and the copyright text displayed by nunit3-console and nunitlite should be updated to the year of the release. Search for AssemblyCopyright in the solution and update it where needed, then check Program.cs in nunit3-console for default values used when no attribute is found.

If necessary, update the year in the general copyright notice LICENSE.txt. Note that these copyright notices refer to each of the packages in their entirety. Each of the .nuspec files in the nuget subdirectory contains a copyright line, which should also be updated.

Notices at the top of each source code file are only updated when copyrightable changes are made to the file, not at the time of release.

0.55.0.6 Update Package Versions

The package version is updated in the build.cake file. The following lines appear near the beginning of the file. Update the versions and modifiers if necessary. Normally, they will already have been set correctly.

var version="3.4.1";
var modifier=""

The version variables are three-part version numbers that follow the basic principles of semantic versioning. Since we publish a number of nuget packages, we use the nuget implementation of semantic versioning.

For NUnit, the major version is updated only rarely. Normal releases will update the minor version and set the third component to zero. The third component is incremented when “hot fixes” are made to a production release or for builds created for a special purpose.

For pre-release versions, a non-empty modifier is specified. This is a suffix added to the version. Our standard suffixes are currently -alpha-n, -beta-n and -rc-n The build script adds an additional suffix of -dbg to any packages created using a Debug build.

NOTE: The first alpha, beta or rc release may omit the -n. In that case, any following alpha, beta or rc should use -2.

0.55.0.7 Update CHANGES File

The CHANGES.txt file in the project root contains all relevant changes for each release. It contains the same information as the release notes in the project documentation, in text format. Because the CHANGES file includes the date of the release, you must know when the release is coming out in order to edit it. Otherwise, it will be necessary to make a final change to the file at the point of making the release.

Create new sections in the CHANGES file to match those for prior releases. To ensure that all changes are included, review closed issues in the current and any future milestones. If an issue for a previous milestone was actually completed and closed, move it to the current milestone, since that’s where it is being released. Include all issues resolved as closed:done in the issues section of the file. Significant feature additions and changes should be documented, even if they are also listed with issue numbers. Reviewing commits and merged pull requests may help in catching additional changes.

You should commit the CHANGES file separately from the version number changes since that commit will be merged back into master while the version changes will not.

0.55.0.8 Update the Documentation

The Release Notes section of the documentation wiki should match the content of the CHANGES.txt file except for any format differences.

NOTE: Now that the documentation is being kept in a github wiki, it may be possible to use the github markdown text directly in this file for future releases.

For any significant changes to how NUnit is used or what it does, the appropriate pages of the documentation should be updated or new pages created. If you do this in advance of the release (which is actually a good idea) you should do it in a way that the new documentation is not visible until the release is actually made.

0.55.0.9 Push All Changes

If you made any changes to the files in git as part of the preceding steps. Make sure you have pushed them and they have been reviewed in the PR.

0.56 Creating the Release

  1. Clear the package directory to avoid confusion:

    erase package\*

This is not absolutely required, but will be helpful if you have other release packages present in the directory.

  1. You should be working on the release branch. Do a pull to make sure you have everything up to date. If changes of any significance were merged, you should test again before creating the release.

  2. Create the packages by running:

    build -Target Package

  3. Verify that the correct packages have been created in the package sub-directory:

  • NUnit.Console.VERSION.nupkg
  • NUnit.ConsoleRunner.VERSION.nupkg
  • NUnit.Engine.VERSION.nupkg
  • NUnit.Engine.Api.VERSION.nupkg
  • NUnit.Engine.Tool.VERSION.nupkg (Do not release)
  • NUnit.Runners.VERSION.nupkg

0.57 Testing the Release

The degree to which each package needs testing may vary depending on what has been changed. Usually, you should install all the nuget packages into a test project, verifying that the

0.58 Archiving the Release

Packages are archived on nunit.org in the downloads directory. Create a new subfolder under downloads/nunit/v3 for the release. Upload all the package files into that directory.

0.59 Publishing the Release

0.59.0.1 Github

  1. Log onto Github and go to the main nunit repository at https://github.com/nunit.nunit.

  2. Select Releases and then click on the “Draft a new release” button.

  3. Enter a tag to be used for the release. Currently our tags are simply the version of the release, like 3.0.0-alpha-5. If you start typing with ‘3’ you’ll get a list of earlier release tags so you can see the format. Select your release branch as the target for the tag.

  4. Enter a title for the release, like NUnit 3.0 RC release. If you type ‘N’ you’ll get some hints.

  5. Add a description of the release. It will be easier if you have prepared this in advance and can just paste it in.

  6. If this is an Alpha or Beta release, check the box that indicates a pre-release.

  7. Upload the packages you created earlier either directly from the package directory or from their archive location. Note that we upload all the packages, including those that are also published on NuGet.

  8. Click the “Publish release” button to publish the release on Github.

0.59.0.2 NuGet

  1. Sign on to Nuget.org.

  2. Select Upload package.

  3. Browse to the location of the NUnit.VERSION.nupkg you created and upload it.

  4. Verify that the info is correct and click the “Submit” button.

  5. Repeat steps 2-4 for NUnitLite.VERSION.nupkg, NUnit.Console.VERSION.nupkg, NUnitCF.VERSION.nupkg and NUnitLiteCF.VERSION.nupkg, NUnit.SL50.VERSION.nupkg and NUnitLite.SL50.nupkg. Do not upload NUnit.Engine.Tool.VERSION.nupkg.

0.59.0.3 Merge into Master

  1. Close your release Pull Request
  2. Create a new branch off of your release branch
  3. Increment the version in the various locations from the even release to the SemVer odd dev release number. For example 3.2 -> 3.3. See the Update Package Versions section above.
  4. Push your changes to GitHub
  5. Create a pull request from your branch making sure it is based off master

0.59.0.4 Website

If changes to the website have been accumulated in a branch, now is the time to merge it and upload the pages to the site.

0.59.0.5 Notify Users

Send notifications to the mailing list and twitter.

0.59.0.6 Close the Milestone

The milestone representing this release should be closed at this time.

This note describes how to create release packages for the NUnit Framework. Currently, all the builds and packaging must be done on a single Windows machine. This is likely to change in the future as we add more platforms.

Note: These instructions assume that you are creating releases for Silverlight and the Compact Framework at the same time as the general release. If that’s not the case, you may skip the steps marked CF or SL. To release either build separately, perform those steps in sequence.

0.60 Software Prerequisites

Various software combinations and environments may be used to build NUnit components. The following software is what we have used and tested for building everything and creating the release packages. We’ll add options to the list as we find them.

  1. Visual Studio 2012, 2013 or 2015 with the Nuget Package manager.
  2. Visual Studio 2008 (for the compact framework build).

0.61 Preparing for Release

0.61.0.1 Create a Release Branch

All work on releases should be done on a branch.

  1. Fetch and pull latest from master
  2. Create a branch in the form release/3.5.0
  3. As you make the changes below, push the branch to GitHub and create a Pull Request to allow other team members to review your changes.
  4. Do not merge this branch/PR, we will create a separate PR to merge the changes back into master.

0.61.0.2 Make Sure it Works!

  1. Close all instances of Visual Studio or other IDE to ensure that no changes are left unsaved.

  2. Do a clean build and run all the tests on Windows. You may use the command below or three separate commands if preferred. If you encounter errors at any stage, you’re not actually ready to release!

    build.cmd -Target=Clean build.cmd -Target=Test

SL Nothing further - Silverlight is included in the overall build and test targets on Windows.

CF is included in the overall build and test targets on Windows. The build output will appear in the bin\Release\netcf-3.5 sub-directory, alongside the other framework builds. Run the tests using a device or an emulator.

  1. Repeat the build on a Linux system, if available. If this is not possible, be sure to scrutinize the results from the Travis CI build carefully. On Linux, you may use the command

    ./build -Target=Test

  2. Make sure that the most recent commits of master passed all tests in the CI builds. Check the builds on both Travis and Appveyor.

0.61.0.3 Review Milestone Status

  1. Check the milestone for the current release to see that there are no open issues. Any issues that are not going to be in this release should be moved to a future milestone. This may be the time to create the next milestone.

  2. Make sure that completed issues are marked with the appropriate ‘closed’ label depending on disposition. The release notes will end up reflecting issues marked closed:done.

  3. Check all future open milestones for completed issues. Anything that is completed will be included in this release so change its milestone to the current release.

0.61.0.4 Check Assembly Versioning

AssemblyVersion and AssemblyFileVersion are set separately for the framework, engine, engine api and console runner. Each is kept in a separate file and they may be updated separately. Using the 3.4.1 release as an example, version information was set as follows:

  Component             | File to Update      | AssemblyVersion | AssemblyFileVersion 
  --------------------- | ------------------- | --------------- | -------------------
  Framework             | FrameworkVersion.cs |     3.5.0.0     |      3.5.0.0

The copyright year in all the source files is only updated as they are changed, but the copyright in the [assembly: AssemblyCopyright("...")] and the copyright text displayed by nunitlite should be updated to the year of the release. Search for AssemblyCopyright in the solution and update it where needed, then check Program.cs in TextUI.cs in nunitlite-runner for default values used when no attribute is found.

If necessary, update the year in the general copyright notices LICENSE.txt and License.rtf. Note that these copyright notices refer to each of the packages in their entirety. Each of the .nuspec files in the nuget subdirectory contains a copyright line, which should also be updated.

Notices at the top of each source code file are only updated when copyrightable changes are made to the file, not at the time of release.

0.61.0.6 Update Package Versions

The package version is updated in the build.cake file. The following lines appear near the beginning of the file. Update the versions and modifiers as necessary.

var version="3.5.0";
var modifier=""

The version variables are three-part version numbers that follow the basic principles of semantic versioning. Since we publish a number of nuget packages, we use the nuget implementation of semantic versioning.

For NUnit, the major version is updated only rarely. Normal releases will update the minor version and set the third component to zero. The third component is incremented when “hot fixes” are made to a production release or for builds created for a special purpose.

For pre-release versions, a non-empty modifier is specified. This is a suffix added to the version. Our standard suffixes are currently -alpha-n, -beta-n and -rc-n The build script adds an additional suffix of -dbg to any packages created using a Debug build.

NOTE: The first alpha, beta or rc release may omit the -n. In that case, any following alpha, beta or rc should use -2.

0.61.0.7 Update CHANGES File

The CHANGES.txt file in the project root contains all relevant changes for each release. It contains the same information as the release notes in the project documentation, in text format. Because the CHANGES file includes the date of the release, you must know when the release is coming out in order to edit it. Otherwise, it will be necessary to make a final change to the file at the point of making the release.

Create new sections in the CHANGES file to match those for prior releases. To ensure that all changes are included, review closed issues in the current and any future milestones. If an issue for a previous milestone was actually completed and closed, move it to the current milestone, since that’s where it is being released. Include all issues resolved as closed:done in the issues section of the file. Significant feature additions and changes should be documented, even if they are also listed with issue numbers. Reviewing commits and merged pull requests may help in catching additional changes.

You should commit the CHANGES file separately from the version number changes since that commit will be merged back into master while the version changes will not.

0.61.0.8 Update the Documentation

The Release Notes section of the documentation wiki should match the content of the CHANGES.txt file except for any format differences.

NOTE: Now that the documentation is being kept in a github wiki, it may be possible to use the github markdown text directly in this file for future releases.

For any significant changes to how NUnit is used or what it does, the appropriate pages of the documentation should be updated or new pages created. If you do this in advance of the release (which is actually a good idea) you should do it in a way that the new documentation is not visible until the release is actually made.

0.61.0.9 Push All Changes

If you made any changes to the files in git as part of the preceding steps. Make sure you have pushed them and they have been reviewed in the PR.

0.62 Creating the Release

  1. Clear the package directory to avoid confusion:

    erase package\*

This is not absolutely required, but will be helpful if you have other release packages present in the directory.

  1. You should be working on the release branch. Do a pull to make sure you have everything up to date. If changes of any significance were merged, you should test again before creating the release.

  2. Ensure that the release build is up to date. If you have any doubt whether the latest code changes have actually been built, do a clean build. If the build is up to date you may skip this step.

    build -Target Build

  3. Create the image directory

    build -Target CreateImage

You do this to ensure that the latest build is used for packaging. If the images directory does not already contain a subdirectory named for this release (package version and suffix) you may skip this step as a new one will be created automatically.

  1. Create the packages by running:

    build -Target Package

    CF build -Target PackageCF

  2. Verify that the correct packages have been created in the package sub-directory:

  • NUnit-VERSION.zip
  • NUnitCF-VERSION.zip
  • NUnitSL-VERSION.zip
  • NUnit.VERSION.nupkg
  • NUnitCF.VERSION.nupkg
  • NUnit.SL50.VERSION.nupkg
  • NUnitLite.VERSION.nupkg
  • NUnitLiteCF.VERSION.nupkg
  • NUnitLite.SL50.VERSION.nupkg

0.63 Testing the Release

The degree to which each package needs testing may vary depending on what has been changed. Here is a minimal set of tests for each package.

  • Binary Zip Unzip the file, change into the bin directory and use an install of the console to run all the tests.
nunit3-console.exe net-2.0\nunit.framework.tests.dll
nunit3-console.exe net-3.5\nunit.framework.tests.dll
nunit3-console.exe net-4.0\nunit.framework.tests.dll
nunit3-console.exe net-4.5\nunit.framework.tests.dll
portable\nunitlite-runner.exe portable\nunit.framework.tests.dll
portable\nunitlite.tests.exe
  • Silverlight Zip Unzip the file, change into the bin
"C:\Program Files\Microsoft Silverlight\sllauncher.exe" /emulate:nunit.framework.tests.xap /origin:http://localhost
  • CF Zip Unzip the file, change into the bin-3.5 directory and run the tests. The following command runs the tests using your desktop runtime. TODO: Develop an approach to testing on devices.
nunit.framework.tests.exe
  • NuGet Packages Create a test project. Install the packages and verify that they apply to the project correctly. Run tests.

It is also a good idea to install over the previous version to ensure that all files that need upgrades are upgraded. This will catch changes in the Engine.API.

0.64 Archiving the Release

Packages are archived on nunit.org in the downloads directory. Create a new subfolder under downloads/nunit/v3 for the release. Upload all the package files into that directory.

Note: We need to develop a plan for additionally archiving the image directory. For the moment, the binary zip package provides a backup but it’s possible that the two may diverge in the future.

0.65 Publishing the Release

0.65.0.1 Github

  1. Log onto Github and go to the main nunit repository at https://github.com/nunit.nunit.

  2. Select Releases and then click on the “Draft a new release” button.

  3. Enter a tag to be used for the release. Currently our tags are simply the version of the release, like 3.0.0-alpha-5. If you start typing with ‘3’ you’ll get a list of earlier release tags so you can see the format. Select your release branch as the target for the tag.

  4. Enter a title for the release, like NUnit 3.0 RC release. If you type ‘N’ you’ll get some hints.

  5. Add a description of the release. It will be easier if you have prepared this in advance and can just paste it in.

  6. If this is an Alpha or Beta release, check the box that indicates a pre-release.

  7. Upload the packages you created earlier either directly from the package directory or from their archive location. Note that we upload all the packages, including those that are also published on NuGet.

  8. Click the “Publish release” button to publish the release on Github.

0.65.0.2 NuGet

  1. Sign on to Nuget.org.

  2. Select Upload package.

  3. Browse to the location of the NUnit.VERSION.nupkg you created and upload it.

  4. Verify that the info is correct and click the “Submit” button.

  5. Repeat steps 2-4 for NUnitLite.VERSION.nupkg, NUnit.Console.VERSION.nupkg, NUnitCF.VERSION.nupkg and NUnitLiteCF.VERSION.nupkg, NUnit.SL50.VERSION.nupkg and NUnitLite.SL50.nupkg. Do not upload NUnit.Engine.Tool.VERSION.nupkg.

0.65.0.3 Merge into Master

  1. Close your release Pull Request
  2. Create a new branch off of your release branch
  3. Increment the version in the various locations. For example 3.5 -> 3.6. See the Update Package Versions section above.
  4. Push your changes to GitHub
  5. Create a pull request from your branch making sure it is based off master

0.65.0.4 Website

If changes to the website have been accumulated in a branch, now is the time to merge it and upload the pages to the site.

0.65.0.5 Notify Users

Send notifications to the mailing list and twitter.

0.65.0.6 Close the Milestone

The milestone representing this release should be closed at this time.

Before packaging the installer, you must first package and release the Console and Engine. See [[Packaging the Console and Engine]]

0.66 Prepare the Release

  1. Get latest from master
  2. Update CHANGES.TXT. Set the date of the release, and list any packages which have been upgraded since the last release.
  3. Check the version and displayVersion in build.cake. They should have been updated at the last release.
  4. Package the release, .\build.ps1 or .\build.cmd
  5. Check the distribution directory for NUnit.{VERSION}.msi and NUnit.{VERSION}.zip

0.67 Test the Release

0.67.1 Test the Installer

  1. Install NUnit.{VERSION}.msi
  2. Ensure it installs correctly
  3. Check that the extensions included in build.cake are installed
  4. Run unit tests using the install
  5. Ensure the extensions work by running NUnit 2 tests and by creating NUnit 2 test results
  6. Check the version in the nunit3-console.exe output headers when running tests.

0.67.2 Test the ZIP File

  1. Unzip NUnit.{VERSION}.zip
  2. Check that the extensions included in build.cake are installed
  3. Run unit tests using the install
  4. Ensure the extensions work by running NUnit 2 tests and by creating NUnit 2 test results
  5. Check the version in the nunit3-console.exe output headers when running tests.

0.68 Archiving the Release

Packages are archived on nunit.org in the downloads directory. Add the MSI and ZIP to the existing downloads/nunit/v3 for the Console/Engine release.

0.69 Publishing the Release

  1. Log onto Github and go to the main nunit-console repository at https://github.com/nunit/nunit-console.
  2. Go to the releases tab and edit the existing Console release
  3. Add the MSI and ZIP files to the release
  4. Update the website if required

0.70 Tag the Release

  1. Still on master, tag the release with the version, git tag v3.5
  2. Push the tags to GitHub, git push --tags

0.71 Increment the Version

  1. Create a release branch git checkout -b release-3.5
  2. Increment the version and displayVersion in build.cake.
  3. Check-in your changes and push to GitHub
  4. Create a Pull Request, have it reviewed, merged and delete the branch
  5. Close the milestone if one exists

Note: The release branch in this project is not like the release branches in other projects, we don’t save it. If we need to go back and do hotfixes, we will branch off the tag that was created for the release. We do this because there are usually no changes required to this repository for a release except incrementing the version post release. Any larger changes to this repository should be done and tested prior to a release using the normal Pull Request workflow. ###User Docs

  • [[NUnit|NUnit-Documentation]]
  • [[NUnit Gui]]
  • [[Project Editor]]
  • Visual Studio
  • [[Test Adapter|Visual-Studio-Test-Adapter]]
  • [[Test Generator|Visual-Studio-Test-Generator]]
  • [[User FAQ]]

0.71.1 Developer Docs

  • [[NUnit Vision]]
  • [[NUnit Roadmap]]
  • [[Team Practices]]
  • [[Specifications]]
  • [[Developer FAQ]]
© https://gittobook.org, 2017. Unauthorized use and/or duplication of this material without express and written permission from this author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to this site, with appropriate and specific direction to the original content.
Table