UTCore: Test Group Support

Overview

Test suite grouping in UT Core enables efficient and targeted testing. It allows developers to organise tests by level enabling them to run only relevant tests, saving time and resources. This improves organisation, supports phased testing, simplifies regression testing, and facilitates test automation, ultimately leading to higher quality software and faster development.

typedef enum
{
    UT_TESTS_L1 = 1,     /*!< Level 1 basic tests are expected to be in this group */
    UT_TESTS_L2,         /*!< Level 2 advanced tests are expected to be in this group */
    UT_TESTS_L3,         /*!< Level 3 module tests are expected to be in this group */
    UT_TESTS_L4,         /*!< Level 4 module control functions (e.g., start/stop module), not part of a testing suite */
    UT_TESTS_HUMAN_L2,   /*!< Level 2 suite requires human interaction */
    UT_TESTS_HUMAN_L3,   /*!< Level 3 suite requires human interaction */
    UT_TESTS_HUMAN_L4,   /*!< Level 4 suite requires human interaction */
    UT_TESTS_VDEVICE,    /*!< Level 3 suite for setup-specific tests, not runnable on a real device */
    UT_TESTS_UNKNOWN,    /*!< Placeholder for existing suites */
    UT_TESTS_MAX         /*!< Out-of-range marker (not a valid status) */
} UT_groupID_t;

With this feature, users can register a suite under a specific group and enable/disable groups at runtime using command-line switches:

-d to disable a group
-e to enable a group

For example, to disable a test group:

<binary> -d 1

Since UT Core supports both C and C++ (CPP) variants, the registration process differs slightly for each. The details for both are provided below.

C Variant

Registering a Test Suite in C

The following example demonstrates how to register a test suite with a group in the C variant:

#include <string.h>
#include <stdlib.h>
#include <ut.h>

void test_l1_function1(void)
{
    UT_FAIL("Need to implement");
    /* Positive */
    /* Negative */
} 

void test_l1_function2(void)
{
    UT_FAIL("Need to implement");
    /* Positive */
    /* Negative */
} 

static UT_test_suite_t *pSuite = NULL;

/**
 * @brief Register the main tests for this module
 * 
 * @return int - 0 on success, otherwise failure
 */
int main(int argc, char *argv[])
{
    UT_init(argc, argv);

    /* Add a suite to the registry with a group ID */
    pSuite = UT_add_suite_withGroupID("[L1 test_Example]", NULL, NULL, UT_TESTS_L1);
    
    if (pSuite == NULL) 
    {
        return -1;
    }

    UT_add_test(pSuite, "blah_level1_test_function", test_l1_function1);
    UT_add_test(pSuite, "blah_level1_test_function", test_l1_function2);

    UT_run_tests();

    return 0;
}

CPP Variant

Registering a Test Suite in C++

The following example demonstrates how to register a test suite with a group in the C++ variant:

#include <ut.h>

class TestL1Example : public UTCore
{
public:
    TestL1Example() : UTCore() {}

    ~TestL1Example() override = default;

    void SetUp() override
    {
        // Code to set up resources before each test
    }

    void TearDown() override
    {
        // Code to clean up resources after each test
    }
};

// Automatically register the test suite before execution
UT_ADD_TEST_TO_GROUP(TestL1Example, UT_TESTS_L1)

UT_ADD_TEST(TestL1Example, TestL1Equal)
{
    UT_ASSERT_EQUAL(1, 1); // Basic test case
}

UT_ADD_TEST(TestL1Example, TestL1NotEqual)
{
    UT_ASSERT_NOT_EQUAL(1, 2);
}

Enabling/Disabling Test Groups at Runtime

After compilation, users can enable or disable test groups using command-line arguments.

For example, to disable a test group:

<binary> -d 1

This will disable:
	•	C Variant: [L1 test_Example] suite
	•	CPP Variant: TestL1Example suite

All other test suites will continue to run when the executable is launched.

Note : by default all tests are enabled