Other Jupyter Notebook Tests

Search…
This guide uses an older testing methodology that can be a bit cumbersome to set up. This guide explains how to test Jupyter notebook code using an improved approach. If testing the code is what you are after, be sure to check it out first!
Jupyter Notebook tests allow you to test code and output from Jupyter Notebook cells. This comprises of two parts:
1.Setting up the Jupyter Notebook.
2.Creating a custom test for Jupyter Notebooks. This test must be used in conjunction with other tests.
The steps are described in detail below.
Set Up the Jupyter Notebook
First, in the Jupyter Notebook, edit the metadata for the cell you'd like to test. You can get to this by clicking View > Cell Toolbar > Edit Metadata.
Editing the Jupyter Notebook cell's metadata
You will need to add a unique identifier to the cell's metadata, for example, by generating a GUID using this site. You should add this value to the eid keyword, as shown below:
Adding an `eid` value to the Jupyter Notebook cell's metadata
Next, add the autosave magic command to the top cell of the notebook, as follows:
The `autosave` command
This step is necessary because notebooks must be saved before the test can be run. Once this is completed, hide the cells' metadata, save the Jupyter Notebook, and save the lesson files.
Create a Custom Test
Create a custom task with the following specifications:
Field
Content
Command
bash {{test}}
Test Contents
wget "https://storage.googleapis.com/codevolve-staging-misc/ipynb_export.py?bust=$RANDOM" -O .ipynb_export.py
python3 .ipynb_export.py 'name-of-notebook.ipynb'
Failed Pattern
NONE
Filename Override
.converter.sh
Require Success
Enable
For example:
Example of a Jupyter Notebook custom test
When this custom test is run, it will save the cell contents and output as .enb_<guid>.<extension> (see next section for more details).
For example, if the unique identifier you added to the cell's metadata is d5137d, and you would like to use a code pattern test on the code within the cell, you would direct the code pattern test to search within .*d5137d.py:
This Code Pattern Test looks at the file created by the Jupyter Notebook Custom Test
You will need one custom test per Jupyter Notebook you'd like to test and it should be enabled before any other tests, like so:
Enable Jupyter Notebook Custom Test before other tests
Files Created by Jupyter Notebook Custom Tests
The following explains what files are created by the Jupyter Notebook Custom Tests and use cases for each:
Cell Input
Cell input will be saved as .py files. These can then be checked using unit tests and code patterns.
Cell Output
Cell output will be saved as .txt or .stdout files, depending on whether a print statement was used. These can be used with code patterns and unit tests.
Errors
Errors are saved as .er files. This is useful for testing whether an error was raised correctly.
Images
Images created will be saved to .png files, which can be used with image tests. Any text associated with the images will be saved as .src files. You can these with code patterns.
Tip
If you are unsure of whether or not a user will use a print statement, and you would like to test the cell's output, you can create a test like the following:
Using a Unit Test on either a `txt` or a `stdout` file
If you get stuck, don't hesitate to reach out for help!