Unit Testing in Maya (Part 2)

  • Edited: 6/18/2016: Updated RollbackImporter to support relative imports and “from package import module” syntax.

In my last post, I went over writing scripts that can be used to write unit tests for Maya tools and then run those tests from the commandline outside of Maya. In this post, I’ll go over running those tests interactively in Maya. Being able to run tests interactively in Maya will greatly speed up debug times when tracking down issues because you’ll be able to see what is going on in your tests. To make the process more user-friendly, I’ll also add a UI to help execute the tests. Again, all of this code is available on my Github.

cmt_testrunner

The interface is written in PySide and uses a QTreeView to display a derived QAbstractItemModel holding the discovered test data. Since the UI is pretty standard PySide code, I will only go over the code related to actually running the tests. The UI provides three different methods of running tests as a convenience to the user: run all tests, run selected tests, and run failed tests. To run all the tests, we can use the get_tests function mentioned in the previous post.

def run_all_tests(self):
    """Callback method to run all the tests found in MAYA_MODULE_PATH."""
    self.reset_rollback_importer()
    test_suite = unittest.TestSuite()
    mayaunittest.get_tests(test_suite=test_suite)
    self.output_console.clear()
    self.model.run_tests(self.stream, test_suite)

The first line resets what I’ll call the rollback importer. It is used to reload code updates without having to rely on sprinkling reload calls throughout your code which is typically bad form. I’ll go over the rollback importer shortly. Next, we populate a TestSuite with all tests found in the paths of MAYA_MODULE_PATH by calling our get_tests function. Then we clear our output console and run the tests.

The rollback importer helps to create a better testing workflow by getting the latest version of the code without having to reload anything. It allows for a iterative update/test workflow without having to restart Maya or insert several reload statements in your code. I learned about this technique from PyUnit.

import __builtin__
class RollbackImporter(object):
    """Used to remove imported modules from the module list.

    This allows tests to be rerun after code updates without doing any reloads.
    From: http://pyunit.sourceforge.net/notes/reloading.html

    Usage:
    def run_tests(self):
        if self.rollback_importer:
            self.rollback_importer.uninstall()
        self.rollback_importer = RollbackImporter()
        self.load_and_execute_tests()
    """
    def __init__(self):
        """Creates an instance and installs as the global importer."""
        self.previous_modules = set(sys.modules.keys())

    def uninstall(self):
        for modname in sys.modules.keys():
            if modname not in self.previous_modules:
                # Force reload when modname next imported
                del(sys.modules[modname])

The rollback importer temporarily replaces the Python import function to store all the modules imported from that point forward which it can then use to remove the loaded modules from the sys.modules dictionary. As a result, the module will be import fresh each time it is imported. Users just have to run the rollback importer (which runs by opening the UI) before importing any other modules.

The PySide data model class contains its own run_tests method to run all the specified tests.

def run_tests(self, stream, test_suite):
    """Runs the given TestSuite.

    @param stream: A stream object with write functionality to capture the test output.
    @param test_suite: The TestSuite to run.
    """
    runner = unittest.TextTestRunner(stream=stream, verbosity=2, resultclass=mayaunittest.TestResult)
    runner.failfast = False
    runner.buffer = mayaunittest.Settings.buffer_output
    result = runner.run(test_suite)

    self._set_test_result_data(result.failures, TestStatus.fail)
    self._set_test_result_data(result.errors, TestStatus.error)
    self._set_test_result_data(result.skipped, TestStatus.skipped)

    for test in result.successes:
        node = self.node_lookup[str(test)]
        index = self.get_index_of_node(node)
        self.setData(index, 'Test Passed', QtCore.Qt.ToolTipRole)
        self.setData(index, TestStatus.success, QtCore.Qt.DecorationRole)

def _set_test_result_data(self, test_list, status):
    """Store the test result data in model.

    @param test_list: A list of tuples of test results.
    @param status: A TestStatus value."""
    for test, reason in test_list:
        node = self.node_lookup[str(test)]
        index = self.get_index_of_node(node)
        self.setData(index, reason, QtCore.Qt.ToolTipRole)
        self.setData(index, status, QtCore.Qt.DecorationRole)

The code is similar to the run_tests function in previous post except this time we are capturing the result to display in the UI. You’ll see some code related to looking up the QModelIndex of an item in the data model given a test object. This is used to aid in updating the UI with the proper result per test. The stream that is passed in is used to capture the output of the tests and display them in the output console (a QTestEdit) with color output.

class TestCaptureStream(object):
    """Allows the output of the tests to be displayed in a QTextEdit."""
    success_color = QtGui.QColor(92, 184, 92)
    fail_color = QtGui.QColor(240, 173, 78)
    error_color = QtGui.QColor(217, 83, 79)
    skip_color = QtGui.QColor(88, 165, 204)
    normal_color = QtGui.QColor(200, 200, 200)

    def __init__(self, text_edit):
        self.text_edit = text_edit

    def write(self, text):
        """Write text into the QTextEdit."""
        # Color the output
        if text.startswith('ok'):
            self.text_edit.setTextColor(TestCaptureStream.success_color)
        elif text.startswith('FAIL'):
            self.text_edit.setTextColor(TestCaptureStream.fail_color)
        elif text.startswith('ERROR'):
            self.text_edit.setTextColor(TestCaptureStream.error_color)
        elif text.startswith('skipped'):
            self.text_edit.setTextColor(TestCaptureStream.skip_color)

        self.text_edit.insertPlainText(text)
        self.text_edit.setTextColor(TestCaptureStream.normal_color)

    def flush(self):
        pass

To run a given test interactively, we will want to be able to disable the functionality of creating a new scene after each test. That way we can run a test and see actually see what it did in Maya.

class MayaTestRunnerDialog(MayaQWidgetBaseMixin, QtGui.QMainWindow):
    def __init__(self, *args, **kwargs):
        super(MayaTestRunnerDialog, self).__init__(*args, **kwargs)
        # snip...
        action = menu.addAction('New Scene Between Test')
        action.setToolTip('Creates a new scene file after each test.')
        action.setCheckable(True)
        action.setChecked(mayaunittest.Settings.file_new)
        action.toggled.connect(mayaunittest.set_file_new)


# In mayaunittest.py
def set_file_new(value):
    """Set whether a new file should be created after each test.

    @param value: True or False
    """
    Settings.file_new = value

Running the selected tests is similar to running all the tests except we can now use the test parameter of the get_tests function to only load specific tests.

def run_selected_tests(self):
    """Callback method to run the selected tests in the UI."""
    self.reset_rollback_importer()
    test_suite = unittest.TestSuite()

    indices = self.test_view.selectedIndexes()
    if not indices:
        return

    # Remove any child nodes if parent nodes are in the list.  This will prevent duplicate tests from being run.
    paths = [index.internalPointer().path() for index in indices]
    test_paths = []
    for path in paths:
        tokens = path.split('.')
        for i in range(len(tokens) - 1):
            p = '.'.join(tokens[0:i+1])
            if p in paths:
                break
        else:
            test_paths.append(path)

    # Now get the tests with the pruned paths
    for path in test_paths:
        mayaunittest.get_tests(test=path, test_suite=test_suite)

    self.output_console.clear()
    self.model.run_tests(self.stream, test_suite)

Also it is nice to be able to run all failed tests. It is almost identical to run_all_test except we only use the tests with fail or error statuses.

def run_failed_tests(self):
    """Callback method to run all the tests with fail or error statuses."""
    self.reset_rollback_importer()
    test_suite = unittest.TestSuite()
    for node in self.model.node_lookup.values():
        if isinstance(node.test, unittest.TestCase) and node.get_status() in {TestStatus.fail, TestStatus.error}:
            mayaunittest.get_tests(test=node.path(), test_suite=test_suite)
    self.output_console.clear()
    self.model.run_tests(self.stream, test_suite)

With all the above code, we now have a way to interactively run our unit tests inside of Maya. Also all of this code is available for free on Github.

3 thoughts on “Unit Testing in Maya (Part 2)”

David January 18, 2017 at 2:30 am

Hi Chad,

Maybe I did something wrong but when I open the unittest window it is empty even if I put some test files directly to one of the folder from MODULE_PATH. I used your TestCase class for base class. What I missed?

David January 19, 2017 at 3:02 pm

Hey,

I’ve just found the problem. So basicly there are two ways to install a module.
1. Place module_root to MAYA_MODULE_PATH
2. Place the *.mod file to one of the folder from MAYA_MODULE_PATHs

I used the 2nd one. It means when “maya_module_tests()” runs it will find nothing because there are only *.mod files there so it is not possible to yield “*/tests” folders.

It should look for “tests” folders recursively.

Chad January 19, 2017 at 8:55 pm

I’m glad you got it working. The setup in the code is just an example. You are free to modify it however you like!

Leave A Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

<code> For inline code.
[sourcecode] For multiline code.