Tasks and Checks Basics

Next Tech content can be tested in hundreds of different ways. Tasks allow you to verify that a user has completed a specific task, assign a grade, and quantify the skills learned.

User Display

A user can access tasks and checks via the Tasks and Checks options in the left sidebar, as shown in the video below:

Tasks and Checks

Checks Summary

Checks are turned into code (or written as them) and run to validate a lesson task was completed.

They are only run if they've been added to a task, as described below.

There are many different types of checks, as described in detail in the Tasks and Checks section.

Task Settings

Tasks are groups of checks that can be tied to instructions or assigned a graded weight for assessment. The checks tied to a task are run in order, so if you for example wanted to run a program supposed to write to a file and then validate the file contents, you'd add a check case to the task, then a code (file) pattern. Tasks are run asynchronously.

The options for a task are described below.

Description

A summary of the task, for example "Program should produce the correct output given valid input." This is shown to the user on the grading screen (if assessment is enabled).

Failed Feedback

This is the text shown to the user as a message when they unsuccessfully attempt to complete the instruction. It supports both Markdown and templates.

Passed Feedback

This is the text shown to the user as a message when they successfully complete the instruction. It supports both Markdown and templates.

Same Directory

If disabled, a copy of /home/nt-user/workspace will be created for the code to run in. It's faster to leave this enabled, but you should disable it for checks that create files (like unit tests).

Checks Table

Here you can toggle on checks you'd like to be part of this task. Keep in mind that checks will be run in the order they are toggled on in.

Advanced Settings

Advanced settings can be found by clicking on the Settings option in the left sidebar and scrolling down to the "Advanced Settings" section.

Show Run Checks Button

If you'd like to hide the Run Checks button from your users, you can disable this option. If enabled, the Run Checks button will only be shown if there are shared tests.

Require Test File

If enabled, the student must have the file that they want to test open in the editor before clicking the Run Checks button.

How does testing differ across languages?

For the most part, testing across languages is the same. Certain exceptions exist like, HTML & CSS will not have unit tests. Other languages, like Python and Java and C#, will not have web page tests and browser simulations. Python has unit tests written in a framework called “unittest” and Java has unit tests written in a framework called “JUnit”. The concepts are similar but the syntax will vary based on the language and the framework used for that language. If there is a framework or set of frameworks you think should be included on a stack for testing, please feel free to send out support

How does grading work?

Each task is tied to grading weight. A percentage of weight is curved depending on how many graded checks pass in a task (partial credit). Some tasks are ungraded; furthermore, some checks are ungraded. Some ungraded checks are necessary in order for other checks to work properly, this may include creating dummy files to read into a program.

Workflow Example

“Create a program that asks the user for something, does anything, and prints something else.” Here we have 3 objectives (aka Tasks):

  1. Ask the user for something

  2. Do anything

  3. Print something else

To make sure that a student implements these tasks properly each task might have 2 checks

Task 1 : Ask the user for something

  1. Test Case 1 - Runs the program simulating a user, and provides “something” as input, monitors the output for a specific pattern.

  2. Test Case 2 - Runs the program simulating a user and provides “something drastically different” as input, monitors the output for a specific pattern.

Why are there two test cases?

Sometimes a program can behave differently depending on what it is given. It may appear to work properly when in fact, it does not with certain inputs. Test cases can also check to make sure that “Unexpected Output” is not present. So Test Case 1 can monitor the output to make sure that Test Case 2 ‘s information is not present and vice versa. This helps prevent the “gaming” of the system where a user intentionally fails the test to see what they need to print to the console.

Task 2 : Do anything

  1. Unit Test 1 - A unit test is a common and versatile form of testing functions and methods inside objects. A unit test will make sure that a function returns or behaves properly given various sets of parameters or conditions.

Task 3 : Print something else

  1. Test Case 1 - Runs the program simulating a user, and provides something as input, monitors the output for a “Something else”.

  2. Code Pattern - Searches the code for a specific regex (regular expression) pattern.