Other Jupyter Notebook Tests

This guide uses an older testing methodology that can be a bit cumbersome to set up. This guide explains how to test Jupyter notebook code using an improved approach. If testing the code is what you are after, be sure to check it out first!

Jupyter Notebook tests allow you to test code and output from Jupyter Notebook cells. This comprises of two parts:

  1. Setting up the Jupyter Notebook.

  2. Creating a custom test for Jupyter Notebooks. This test must be used in conjunction with other tests.

The steps are described in detail below.

Set Up the Jupyter Notebook

First, in the Jupyter Notebook, edit the metadata for the cell you'd like to test. You can get to this by clicking View > Cell Toolbar > Edit Metadata.

Editing the Jupyter Notebook cell's metadata

You will need to add a unique identifier to the cell's metadata, for example, by generating a GUID using this site. You should add this value to the eid keyword, as shown below:

Adding an `eid` value to the Jupyter Notebook cell's metadata

Next, add the autosave magic command to the top cell of the notebook, as follows:

The `autosave` command

This step is necessary because notebooks must be saved before the test can be run. Once this is completed, hide the cells' metadata, save the Jupyter Notebook, and save the lesson files.

Create a Custom Test

Create a custom task with the following specifications:




bash {{test}}

Test Contents

wget "https://storage.googleapis.com/codevolve-staging-misc/ipynb_export.py?bust=$RANDOM" -O .ipynb_export.py

python3 .ipynb_export.py 'name-of-notebook.ipynb'

Failed Pattern


Filename Override


Require Success


For example:

Example of a Jupyter Notebook custom test

When this custom test is run, it will save the cell contents and output as .enb_<guid>.<extension> (see next section for more details).

For example, if the unique identifier you added to the cell's metadata is d5137d, and you would like to use a code pattern test on the code within the cell, you would direct the code pattern test to search within .*d5137d.py:

This Code Pattern Test looks at the file created by the Jupyter Notebook Custom Test

You will need one custom test per Jupyter Notebook you'd like to test and it should be enabled before any other tests, like so:

Enable Jupyter Notebook Custom Test before other tests

Files Created by Jupyter Notebook Custom Tests

The following explains what files are created by the Jupyter Notebook Custom Tests and use cases for each:

Cell Input

Cell input will be saved as .py files. These can then be checked using unit tests and code patterns.

Cell Output

Cell output will be saved as .txt or .stdout files, depending on whether a print statement was used. These can be used with code patterns and unit tests.


Errors are saved as .er files. This is useful for testing whether an error was raised correctly.


Images created will be saved to .png files, which can be used with image tests. Any text associated with the images will be saved as .src files. You can these with code patterns.


If you are unsure of whether or not a user will use a print statement, and you would like to test the cell's output, you can create a test like the following:

Using a Unit Test on either a `txt` or a `stdout` file

If you get stuck, don't hesitate to reach out for help!