Loading Unit Test Cases Dynamically in a Notebook

testing
unittest
reflection
Dynamically load Python test cases from a notebook module. Explore the benefits and the pitfalls of this approach in your testing strategy.
Modified

06/08/2025

Beginner Tip

This tip is part of the Decembricks 2024 series to teach a new tip everyday of december.

Summary

  • Dynamic Test Case Discovery: Automatically loads test cases from the current notebook module.
  • Reflection Magic: Uses reflection to introspect classes, searching for unittest.TestCase subclasses.
  • Test Suite Management: Organizes tests into a suite for easier management.

Introduction

Have you ever had to define a unit test suite and got fed up with having to create a list of the unit test classes to include into the suite for running? Well this tip is for you, it looks up the objects defined in the current namespace that are of type UnitTest case.

Code

Load All Test Classes from the Current Notebook

Update August 2025

This tip was recently updated due to work going on with the databricks platform it seems the sys.module[name] call returns the different module details now. So the below code has been modified to resolve that by using the iPython Users Namespace.

import unittest

def load_test_classes_from_notebook():
    try:
        ns = get_ipython().user_ns #UserNameSpace
    except NameError:
        ns = globals()

    suite = unittest.TestSuite()

    # Limit to classes that look like *your* tests
    nb_name = ns.get('__name__', '__main__')
    for name, obj in list(ns.items()):
        if isinstance(obj, type) and issubclass(obj, unittest.TestCase) and obj is not unittest.TestCase:
            if getattr(obj, '__module__', None) in (nb_name, '__main__'): # Maybe Overkill todo this filtering 
                suite.addTests(unittest.TestLoader().loadTestsFromTestCase(obj))
    return suite

    
suite = load_test_classes_from_notebook()
runner = unittest.TextTestRunner(verbosity=2)
runner.run(suite)

Example Code & Tests

Example Methods to Test

These are some sample functions in python that we will use to illustrate how unit tests work

def add(a, b):
    """Perform addition of two numeric values."""
    return a + b

def multiply(a, b):  
    """Perform multiplication of two numeric values."""
    return a * b

Example Unit Test Cases

These are sample Unit Test Cases

import unittest

class TestGenericMathOperations(unittest.TestCase):
    def test_addition(self):
        # Test with numeric values
        self.assertEqual(add(2, 3), 5, "Addition of 2 and 3 should equal 5")
        self.assertEqual(add(-1, 5), 4, "Addition of -1 and 5 should equal 4") 
        self.assertAlmostEqual(add(0.1, 0.2), 0.3, places=1, msg="Addition of 0.1 and 0.2 should approximate 0.3")
        
        # Test with an invalid case
        with self.assertRaises(TypeError, msg="Adding a number and a string should raise a TypeError"):
            add(5, "hello")
            
    def test_multiplication(self):
        # Test with numeric values  
        self.assertEqual(multiply(2, 3), 6, "Multiplication of 2 and 3 should equal 6")
        self.assertEqual(multiply(-2, 4), -8, "Multiplication of -2 and 4 should equal -8")
        self.assertAlmostEqual(multiply(0.5, 0.4), 0.2, places=1, msg="Multiplying 0.5 and 0.4 should approximate 0.2")
        
        # Test with an invalid case
        with self.assertRaises(TypeError, msg="Multiplying a number and a string should raise a TypeError"):
            multiply(3, "world")

Details

In the spirit of enabling flexible test discovery, this snippet takes advantage of reflection to dynamically load all test cases in the users ipython notebook namespace. Let’s dive into the finer details:

  1. TestSuite Setup: We initialize an empty TestSuite to collect individual test cases.

  2. Current Namespace Context: Accessing get_ipython().user_ns grants a reference to the current namespace, a key tool for reflection.

  3. Reflection and Filtering:

    • The dir function retrieves all objects in the module.
    • getattr allows accessing the object itself by name.
    • We then ensure the object is a subclass of unittest.TestCase but not the base class itself.

  4. Loading Tests: unittest.TestLoader().loadTestsFromTestCase loads tests from each discovered class, and the suite aggregates them.

Benefits

  • Automation and Flexibility: Automatically loading tests simplifies running new tests without manual suite updates.
  • Dynamic Testing: Great for scenarios where test classes change frequently or are dynamically generated.

Cautions

  • Over-Discovery: This method could inadvertently load experimental or partial test classes, leading to unintended test runs.
  • Opaque Testing: The reliance on reflection makes the test suite implicit and less readable. Reviewers may struggle to understand the source of loaded tests.
  • Performance: The reflective lookup over the entire module can be inefficient, especially in larger projects.
  • pytest: Is more likely to align with your requirements and use concepts like fixtures to support you.
Back to top