PK ûC Ci‹N«¬ «¬ ( nose2-new-doc-theme-latest/such_dsl.html
Note
New in version 0.4
Such is a DSL for writing tests with expensive, nested fixtures – which typically means functional tests. It requires the layers plugin (see Organizing Test Fixtures into Layers).
Unlike some python testing DSLs, such is just plain old python.
import unittest
from nose2.tools import such
class SomeLayer(object):
@classmethod
def setUp(cls):
it.somelayer = True
@classmethod
def tearDown(cls):
del it.somelayer
#
# Such tests start with a declaration about the system under test
# and will typically bind the test declaration to a variable with
# a name that makes nice sentences, like 'this' or 'it'.
#
with such.A('system with complex setup') as it:
#
# Each layer of tests can define setup and teardown methods.
# setup and teardown methods defined here run around the entire
# group of tests, not each individual test.
#
@it.has_setup
def setup():
it.things = [1]
@it.has_teardown
def teardown():
it.things = []
#
# The 'should' decorator is used to mark tests.
#
@it.should('do something')
def test():
assert it.things
#
# Tests can use all of the normal unittest TestCase assert
# methods by calling them on the test declaration.
#
it.assertEqual(len(it.things), 1)
#
# The 'having' context manager is used to introduce a new layer,
# one that depends on the layer(s) above it. Tests in this
# new layer inherit all of the fixtures of the layer above.
#
with it.having('an expensive fixture'):
@it.has_setup
def setup():
it.things.append(2)
#
# Tests that take an argument will be passed the
# unittest.TestCase instance that is generated to wrap
# them. Tests can call any and all TestCase methods on this
# instance.
#
@it.should('do more things')
def test(case):
case.assertEqual(it.things[-1], 2)
#
# Layers can be nested to any depth.
#
with it.having('another precondtion'):
@it.has_setup
def setup():
it.things.append(3)
@it.has_teardown
def teardown():
it.things.pop()
@it.should('do that not this')
def test(case):
it.things.append(4)
#
# Tests can add their own cleanup functions.
#
case.addCleanup(it.things.pop)
case.assertEqual(it.things[-1], 4, it.things)
@it.should('do this not that')
def test(case):
case.assertEqual(it.things[-1], 3, it.things[:])
#
# A layer may have any number of sub-layers.
#
with it.having('a different precondition'):
#
# A layer defined with ``having`` can make use of
# layers defined elsewhere. An external layer
# pulled in with ``it.uses`` becomes a parent
# of the current layer (though it doesn't actually
# get injected into the layer's MRO).
#
it.uses(SomeLayer)
@it.has_setup
def setup():
it.things.append(99)
@it.has_teardown
def teardown():
it.things.pop()
#
# Layers can define setup and teardown methods that wrap
# each test case, as well, corresponding to TestCase.setUp
# and TestCase.tearDown.
#
@it.has_test_setup
def test_setup(case):
it.is_funny = True
case.is_funny = True
@it.has_test_teardown
def test_teardown(case):
delattr(it, 'is_funny')
delattr(case, 'is_funny')
@it.should('do something else')
def test(case):
assert it.things[-1] == 99
assert it.is_funny
assert case.is_funny
@it.should('have another test')
def test(case):
assert it.is_funny
assert case.is_funny
@it.should('have access to an external fixture')
def test(case):
assert it.somelayer
with it.having('a case inside the external fixture'):
@it.should('still have access to that fixture')
def test(case):
assert it.somelayer
#
# To convert the layer definitions into test cases, you have to call
# `createTests` and pass in the module globals, so that the test cases
# and layer objects can be inserted into the module.
#
it.createTests(globals())
#
# Such tests and normal tests can coexist in the same modules.
#
class NormalTest(unittest.TestCase):
def test(self):
pass
The tests it defines are unittest tests, and can be used with nose2 with just the layers plugin. You also have the option of activating a reporting plugin (nose2.plugins.layers.LayerReporter) to provide a more discursive brand of output:
test (test_such.NormalTest) ... ok
A system with complex setup
should do something ... ok
having an expensive fixture
should do more things ... ok
having another precondtion
should do that not this ... ok
should do this not that ... ok
having a different precondition
should do something else ... ok
should have another test ... ok
----------------------------------------------------------------------
Ran 7 tests in 0.002s
OK
Such uses the things in python that are most like anonymous code blocks to allow you to construct tests with meaningful names and deeply-nested fixtures. Compared to DSLs in languages that do allow blocks, it is a little bit more verbose – the block-like decorators that mark fixture methods and test cases need to decorate something, so each fixture and test case has to have a function definition. You can use the same function name over and over here, or give each function a meaningful name.
The set of tests begins with a description of the system under test as a whole, marked with the A context manager:
from nose2.tools import such
with such.A('system described here') as it:
# ...
Groups of tests are marked by the having context manager:
with it.having('a description of a group'):
# ...
Within a test group (including the top-level group), fixtures are marked with decorators:
@it.has_setup
def setup():
# ...
@it.has_test_setup
def setup_each_test_case():
# ...
And tests are likewise marked with the should decorator:
@it.should('exhibit the behavior described here')
def test(case):
# ...
Test cases may optionally take one argument. If they do, they will be passed the unittest.TestCase instance generated for the test. They can use this TestCase instance to execute assert methods, among other things. Test functions can also call assert methods on the top-level scenario instance, if they don’t take the case argument:
@it.should("be able to use the scenario's assert methods")
def test():
it.assertEqual(something, 'a value')
@it.should("optionally take an argument")
def test(case):
case.assertEqual(case.attribute, 'some value')
Finally, to actually generate tests, you must call createTests on the top-level scenario instance:
it.createTests(globals())
This call generates the unittest.TestCase instances for all of the tests, and the layer classes that hold the fixtures defined in the test groups. See Organizing Test Fixtures into Layers for more about test layers.
Since order is often significant in functional tests, such DSL tests always execute in the order in which they are defined in the module. Parent groups run before child groups, and sibling groups and sibling tests within a group execute in the order in which they are defined.
Otherwise, tests written in the such DSL are collected and run just like any other tests, with one exception: their names. The name of a such test case is the name of its immediately surrounding group, plus the description of the test, prepended with test ####:, where ‘####’ is the test’s (0-indexed) position within its group. To run a case individually, you must pass in this full name – usually you’ll have to quote it. For example, to run the case should do more things defined above (assuming the layers plugin is activated by a config file, and the test module is in the normal path of test collection), you would run nose2 like this:
nose2 "test_such.having an expensive fixture.test 0000: should do more things"
That is, for the a generated test case, the group description is the class name, and the test case description is the test case name. As you can see if you run an individual test with the layer reporter active, all of the group fixtures execute in proper order when a test is run individually:
$ nose2 "test_such.having an expensive fixture.test 0000: should do more things"
A system with complex setup
having an expensive fixture
should do more things ... ok
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
Test scenario context manager.
Returns a nose2.tools.such.Scenario instance, which by convention is bound to it:
with such.A('test scenario') as it:
# tests and fixtures
A test scenario.
A test scenario defines a set of fixtures and tests that depend on those fixtures.
Generate test cases for this scenario.
Warning
You must call this, passing in globals(), to generate tests from the scenario. If you don’t call createTests, no tests will be created.
it.createTests(globals())
Add a setup method to this group.
The setup method will run once, before any of the tests in the containing group.
A group may define any number of setup functions. They will execute in the order in which they are defined.
@it.has_setup
def setup():
# ...
Add a teardown method to this group.
The teardown method will run once, after all of the tests in the containing group.
A group may define any number of teardown functions. They will execute in the order in which they are defined.
@it.has_teardown
def teardown():
# ...
Add a test case setup method to this group.
The setup method will run before each of the tests in the containing group.
A group may define any number of test case setup functions. They will execute in the order in which they are defined.
Test setup functions may optionally take one argument. If they do, they will be passed the unittest.TestCase instance generated for the test.
@it.has_test_setup
def setup(case):
# ...
Add a test case teardown method to this group.
The teardown method will run before each of the tests in the containing group.
A group may define any number of test case teardown functions. They will execute in the order in which they are defined.
Test teardown functions may optionally take one argument. If they do, they will be passed the unittest.TestCase instance generated for the test.
@it.has_test_teardown
def teardown(case):
# ...
Define a new group under the current group.
Fixtures and tests defined within the block will belong to the new group.
with it.having('a description of this group'):
# ...
Define a test case.
Each function marked with this decorator becomes a test case in the current group.
The decorator takes one optional argument, the description of the test case: what it should do. If this argument is not provided, the docstring of the decorated function will be used as the test case description.
Test functions may optionally take one argument. If they do, they will be passed the unittest.TestCase instance generated for the test. They can use this TestCase instance to execute assert methods, among other things.
@it.should('do this')
def dothis(case):
# ....
@it.should
def dothat():
"do that also"
# ....
Most configuration of nose2 is done via config files. These are standard, .ini-style config files, with sections marked off by brackets (“[unittest]”) and key = value pairs within those sections.
Two command line options, -c and --no-user-config may be used to determine which config files are loaded.
Config files to load. Default behavior is to look for unittest.cfg and nose2.cfg in the start directory, as well as any user config files (unless --no-user-config is selected).
Do not load user config files. If not specified, in addition to the standard config files and any specified with -c, nose2 will look for .unittest.cfg and .nose2.cfg in the user’s $HOME directory.
The [unittest] section of nose2 config files is used to configure nose2 itself. The following options are available to configure test discovery:
This option configures the default directory to start discovery. The default value is ”.” (the current directory where nose2 is executed). This directory is where nose2 will start looking for tests.
This option configures nose2 to add the named directories to sys.path and the discovery path. Use this if your project has code in a location other than the top level of the project, or the directories lib or src. The value here may be a list: put each directory on its own line in the config file.
This option configures how nose detects test modules. It is a file glob.
This option configures how nose detects test functions and methods. The prefix set here will be matched (via simple string matching) against the start of the name of each method in test cases and each function in test modules.
Examples:
[unittest]
start-dir = tests
code-directories = source
more_source
test-file-pattern = *_test.py
test-method-prefix = t
To avoid loading any plugins, use the --no-plugins option. Beware, though: nose2 does all test discovery and loading via plugins, so unless you are patching in a custom test loader and runner, when run with --no-plugins, nose2 will do nothing.
Do not load any plugins. This kills the nose2.
To specify plugins to load beyond the builtin plugins automatically loaded, add a plugins entry under the [unittest] section in a config file.
List of plugins to load. Put one plugin module on each line.
To exclude some plugins that would otherwise be loaded, add an exclude-plugins entry under the [unittest] section in a config file.
List of plugins to exclude. Put one plugin module on each line.
Note
It bears repeating that in both plugins and exclude-plugins entries, you specify the plugin module, not the plugin class.
Examples:
[unittest]
plugins = myproject.plugins.frobulate
otherproject.contrib.plugins.derper
exclude-plugins = nose2.plugins.loader.functions
nose2.plugins.outcomes
Most plugins specify a config file section that may be used to configure the plugin. If nothing else, any plugin that specifies a config file section can be set to automatically register by including always-on = True in its config:
[my-plugin]
always-on = True
Plugins may accept any number of other config values, which may be booleans, strings, integers or lists. A polite plugin will document these options somewhere. Plugins that want to make use of nose2’s Sphinx extension as detailed in Documenting plugins must extract all of their config values in their __init__ methods.
You can use nose2.main in the same way that unittest.main (and unittest2.main) have historically worked: to run the tests in a single module. Just put a block like the following at the end of the module:
if __name__ == '__main__':
import nose2
nose2.main()
Then run the module directly – In other words, do not run the nose2 script.
You can take more control over the test runner by foregoing the nose2 script and rolling your own. To do that, you just need to write a script that calls nose2.discover, for instance:
if __name__ == '__main__':
import nose2
nose2.discover()
You can pass several keyword arguments to nose2.discover, all of which are detailed in the documentation for nose2.main.PluggableTestProgram.
To add plugin modules to the list of those automatically loaded, you can pass a list of module names to add (the plugins) argument or exclude (excludedPlugins). You can also subclass nose2.main.PluggableTestProgram and set the class-level defaultPlugins and excludePlugins attributes to alter plugin loading.
None of which will help if you need to register a plugin instance that you’ve loaded yourself. For that, use the extraHooks keyword argument to nose2.discover. Here, you pass in a list of 2-tuples, each of which contains a hook name and a plugin instance to register for that hook. This allows you to register plugins that need runtime configuration that is not easily passed in through normal channels – and also to register objects that are not nose2 plugins as hook targets. Here’s a trivial example:
if __name__ == '__main__':
import nose2
class Hello(object):
def startTestRun(self, event):
print("hello!")
nose2.discover(extraHooks=[('startTestRun', Hello())])
This can come in handy when integrating with other systems that expect you to provide a test runner that they execute, rather than executing tests yourself (django, for instance).
nose2 is the next generation of nicer testing for Python, based on the plugins branch of unittest2. nose2 aims to improve on nose by:
- providing a better plugin api
- being easier for users to configure
- simplifying internal interfaces and processes
- supporting Python 2 and 3 from the same codebase, without translation
- encourging greater community involvment in its development
In service of some those goals, some features of nose will not be supported in nose2. See differences for a thorough rundown.
In time – once unittest2 supports plugins – nose2 should be able to become just a collection of plugins and configuration defaults. For now, it provides a plugin api similar to the one in the unittest2 plugins branch, and overrides various unittest2 objects.
You are witnesses at the new birth of nose, mark 2. Hope you enjoy our new direction!
|
|
|
|
The recommended way to install nose2 is with pip
pip install nose2
You can also install from source by downloading the source distribution from pypi, un-taring it, and running python setup.py install in the source directory. Note that if you install this way, and do not have distribute or setuptools installed, you must install nose2’s dependencies manually.
To run tests in a project, use the nose2 script that is installed with nose2:
nose2
This will find and run tests in all packages in the current working directory, and any sub-directories of the current working directory whose names start with ‘test’.
To find tests, nose2 looks for modules whose names start with ‘test’. In those modules, nose2 will load tests from all unittest.TestCase subclasses, as well as functions whose names start with ‘test’.
The nose2 script supports a number of command-line options, as well as extensive configuration via config files. For more information see Using nose2 and Configuring nose2.
In the simplest case, go to the directory that includes your project source and run nose2 there:
nose2
This will discover tests in packages and test directories under that directory, load them, and run them, then output something like:
.............................................................................
----------------------------------------------------------------------
Ran 77 tests in 1.897s
OK
“Test directories” means any directories whose names start with “test”. Within test directories and within any Python packages found in the starting directory and any source directories in the starting directory, nose2 will discover test modules and load tests from them. “Test modules” means any modules whose names start with “test”.
Within test modules, nose2 will load tests from unittest.TestCase subclasses, and from test functions (functions whose names begin with “test”).
To change the place discovery starts, or to change the top-level importable directory of the project, use the -s and -t options.
Directory to start discovery. Defaults to the current working directory. This directory is where nose2 will start looking for tests.
Top-level directory of the project. Defaults to the starting directory. This is the directory containing importable modules and packages, and is always prepended to sys.path before test discovery begins.
Pass test names to nose2 on the command line to run individual test modules, classes, or tests.
A test name consists of a python object part and, for generator or parameterized tests, an argument part. The python object part is a dotted name, such as pkg1.tests.test_things.SomeTests.test_ok. The argument part is separated from the python object part by a colon (”:”) and specifies the index of the generated test to select, starting from 1. For example, pkg1.test.test_things.test_params_func:1 would select the first test generated from the parameterized test test_params_func.
Plugins may provide other means of test selection.
nose2 supports distribute/setuptools’ python setup.py test standard for running tests. To use nose2 to run your package’s tests, add the following to your setup.py:
setup(...
test_suite='nose2.collector.collector',
...
)
(Not literally. Don’t put the ‘...’ parts in.)
Two warnings about running tests this way.
One: because the setuptools test command is limited, nose2 returns a “test suite” that actually takes over the test running process completely, bypassing the test result and test runner that call it. This may be incompatible with some packages.
Two: because the command line arguments to the test command may not match up properly with nose2’s arguments, the nose2 instance started by the collector does not accept any command line arguments. This means that it always runs all tests, and that you cannot configure plugins on the command line when running tests this way. As a workaround, when running under the test command, nose2 will read configuration from setup.cfg if it is present, in addition to unittest.cfg and nose2.cfg. This enables you to put configuration specific to the setuptools test command in setup.cfg – for instance to activate plugins that you would otherwise activate via the command line.
Run:
nose2 -h
to get help for nose2 itself and all loaded plugins.
usage: nose2 [-s START_DIR] [-t TOP_LEVEL_DIRECTORY] [--config [CONFIG]]
[--no-user-config] [--no-plugins] [--verbose] [--quiet] [-B] [-D]
[--collect-only] [--log-capture] [-P] [-h]
[testNames [testNames ...]]
positional arguments:
testNames
optional arguments:
-s START_DIR, --start-dir START_DIR
Directory to start discovery ('.' default)
-t TOP_LEVEL_DIRECTORY, --top-level-directory TOP_LEVEL_DIRECTORY, --project-directory TOP_LEVEL_DIRECTORY
Top level directory of project (defaults to start dir)
--config [CONFIG], -c [CONFIG]
Config files to load, if they exist. ('unittest.cfg'
and 'nose2.cfg' in start directory default)
--no-user-config Do not load user config files
--no-plugins Do not load any plugins. Warning: nose2 does not do
anything if no plugins are loaded
--verbose, -v
--quiet
-h, --help Show this help message and exit
plugin arguments:
Command-line arguments added by plugins:
-B, --output-buffer Enable output buffer
-D, --debugger Enter pdb on test fail or error
--collect-only Collect and output test names, do not run any tests
--log-capture Enable log capture
-P, --print-hooks Print names of hooks in order of execution
These plugins are loaded by default. To exclude one of these plugins from loading, add the plugin’s module name to the exclude-plugins list in a config file’s [unittest] section, or pass the plugin module with the --exclude-plugin argument on the command line. You can also pass plugin module names to exclude to a nose2.main.PluggableTestProgram using the excludePlugins keyword argument.
These plugins are available as part of the nose2 package but are not loaded by default. To load one of these plugins, add the plugin module name to the plugins list in a config file’s [unittest] section, or pass the plugin module with the --plugin argument on the command line. You can also pass plugin module names to a nose2.main.PluggableTestProgram using the plugins keyword argument.
If you are a plugin author, please add your plugin to the list on the nose2 wiki. If you are looking for more plugins, check that list!