Adding Test Cases

Test cases handle input and output checks. They can simulate a user for a command line interface (CLI) program. A test case will provide a program with input, and monitor the output for a pre-set response, or pattern. The output can be tested using regex expressions.

A test case's options are described in detail below.

Description

The test description shown to the user.

Input / STDIN

The input strings to stream into the program. Pressing Enter saves a line of input and is equivalent to pressing Enter when running the project.

Expected Output

The output strings expected from the program. Press Enter to save each one. The test case will be marked complete and / or graded based on the presence of these output strings.

Not Expected Output

Any strings that should not show up in the program's output. If any of these appear, the test case will be marked as failing. Useful for preventing cheating.

Entry Point

Override the value of the {{entry}} template with a custom value.

Shared

If the results from the test case should be shared with the user.

Ordered

If the expected output must appear in order.

Case Sensitive

If the expected output is case sensitive. If disabled, all output will be checked as lowercase.

Strict

If disabled, all output characters except for .,A-Z,a-z,0-9 and whitespace (including newlines) will be removed and repeating whitespace will be reduced to a single space / line.

Advanced Settings

Advanced settings can be accessed by clicking the Show Advanced Settings link at the bottom of the test case.

Regex

If enabled, text entered in "Expected Output" will be treated as regular expressions.

Timeout

Number of seconds to run the test case for before it times out.

Example

A failed regex test case

Here we have one test case that has been marked complete and one regex test case that has been marked incomplete. There are 3 fields in the test case check.

  1. Input - What the test provided as input. If blank, as in this case, the test did not provide any input and is only monitoring the output of the program. This can be done for programs that are not given input or when the input is read in from a file.

  2. Output - This field contains the output generated by the program.

  3. Results - This field contains the pattern or patterns that the test is actively searching for. If red, the pattern was not found in the “Output” section. If green, the pattern was successful matched. Sometimes, there are small discrepancies, like punctuation, that may go unnoticed by users when reviewing results that gives the appearance that tests are failing or passing incorrectly.

Complete test case

Here is a second example of a test case, without the use of a regex expression.