Next Tech content can be tested in hundreds of different ways. Tasks allow you to verify that a user has completed a specific task, assign a grade, and quantify the skills learned.
A user can access tasks and checks via the Tasks and Checks options in the left sidebar, as shown in the video below:
Checks are turned into code (or written as them) and run to validate a task was completed.
They are only run if they've been added to a task, as described below.
There are many different types of checks, as described in detail in the Tasks and Checks section.
Tasks are groups of checks that can be tied to instructions or assigned a graded weight for assessment. The checks tied to a task are run in order, so if you for example wanted to run a program supposed to write to a file and then validate the file contents, you'd add a check case to the task, then a code (file) pattern. Tasks are run asynchronously.
The options for a task are described below.
A summary of the task, for example "Program should produce the correct output given valid input." This is shown to the user on the grading screen (if assessment is enabled).
This is the text shown to the user as a message when they unsuccessfully attempt to complete the instruction. It supports both Markdown and templates.
This is the text shown to the user as a message when they successfully complete the instruction. It supports both Markdown and templates.
This setting can be disabled to completely hide this task from all users. It's good to do this if you're going to be making changes to a live content (or better yet, use Maintenance Mode).
If enabled, users will be able to see the checks and their results when attempting this task.
If enabled, the checks will be run in the same directory. It's faster to leave this enabled, but you will want to disable it in some situations, such as when working with a compiled language that will generate extra files when the code is run.
Here you can toggle on checks you'd like to be part of this task. Keep in mind that checks will be run in the order they are toggled on in.
Interface-wide task settings can be found by clicking on the Interface option in the left sidebar and scrolling down to the "Task Settings" section.
If you'd like to hide the Run Checks button from your users, you can disable this option. If enabled, the Run Checks button will only be shown if there are shared tests.
For the most part, testing across languages is the same. Certain exceptions exist like, HTML & CSS will not have unit tests. Other languages, like Python and Java and C#, will not have web page tests and browser simulations. Python has unit tests written in a framework called "unittest" and Java has unit tests written in a framework called "JUnit". The concepts are similar but the syntax will vary based on the language and the framework used for that language. If there is a framework or set of frameworks you think should be included on a stack for testing, please feel free to send out support
Each task is tied to grading weight. A percentage of weight is curved depending on how many graded checks pass in a task (partial credit). Some tasks are ungraded; furthermore, some checks are ungraded. Some ungraded checks are necessary in order for other checks to work properly, this may include creating dummy files to read into a program.
"Create a program that asks the user for something, does anything, and prints something else." Here we have 3 objectives (a.k.a. Tasks):
Ask the user for something
Print something else
To make sure that a student implements these tasks properly, each task might have 2 checks:
Task 1 : Ask the user for something
Test Case 1 - Runs the program simulating a user, and provides "something" as input, monitors the output for a specific pattern.
Test Case 2 - Runs the program simulating a user and provides "something drastically different" as input, monitors the output for a specific pattern.
Why are there two test cases?
Sometimes a program can behave differently depending on what it is given. It may appear to work properly when in fact, it does not with certain inputs. Test cases can also check to make sure that "Unexpected Output" is not present. So Test Case 1 can monitor the output to make sure that _Test Case 2'_s information is not present and vice versa. This helps prevent the "gaming" of the system where a user intentionally fails the test to see what they need to print to the console.
Task 2 : Do anything
Unit Test 1 - A unit test is a common and versatile form of testing functions and methods inside objects. A unit test will make sure that a function returns or behaves properly given various sets of parameters or conditions.
Task 3 : Print something else
Test Case 1 - Runs the program simulating a user, and provides something as input, monitors the output for a "Something else".
Code Pattern - Searches the code for a specific regex (regular expression) pattern.