Custom Tests

Search…

Custom Tests

Custom tests allow you to use any installed framework. If you find yourself using a custom test frequently, let us know! We can add a template for it, or maybe even an entire new type of test.
Adding a Custom Test
To add a custom test, navigate to Checks in the left sidebar and use the New Check + button. Next, select the Custom Test option in the drop-down and see the settings below:
Settings
Select Template
Select a predefined check template to save yourself some time.
Description
A summary of the custom test. Shown to users if the task is live and checks are shared (which is the default).
Command
The command to run the custom test with (e.g. python3 {{test}}). Supports templates.
Test Contents
The actual code of the test.
Failed Pattern
A regular expression that will match the test's output when it fails.
Filename Override
Set a static filename to name the test as. If not provided, one will be generated using the format nt-test-XXXXXXXX.
For best results, start yours with nt-test- to prevent it from showing in the list of files. Alternatively, you can configure the task to run in a different directory or add the test name to the hidden files glob.
Build First
If enabled, the build script (if provided) will be executed before the test is executed.
Require Success
If enabled, the test will fail unless it returns a 0 exit status.
​

Command Tests

Full GUI Tests

Last modified 1yr ago

Copy link

Contents