The Math Function Unit Testing Design Pattern

This article provides an overview of a new design pattern for unit testing in any programming language, ‘The Math Function Unit Testing Design Pattern’. The data-driven design pattern involves repeatable, automated testing, and minimises unit test code (see example) by delegating to library modules the unit test driver, reading/writing of test data and assertion and formatting of results.

After a section on the background, the article describes the design pattern in two parts: the first deals with the concepts of transactions and APIs and why we should base our unit testing around APIs, while the second describes the design pattern at high level.

There are modules in Oracle PL/SQL, JavaScript, Powershell and Python implementing utilities to support the design pattern, and demonstrating its use. A final section provides links to their GitHub projects and some related articles.

Infographic: The Bauhaus, Where Form Follows Function
[Interesting Twitter thread on the history of Bauhaus by @culturaltutor]

↓ Background
↓ Systems, Transactions and APIs
↓ API Testing with The Math Function Unit Testing Design Pattern
↓ See Also

Background

↑ Contents

Some years ago, I became involved in automated testing within Oracle database systems for the first time, after many years working on projects that used only manual testing. The potential benefits of repeatable, automated testing were clear, but I soon realised that there was one big problem: Almost all the material available on how to implement such testing was based on concepts developed for object oriented programming in Java in the early 2000s.

These concepts generally involved splitting code into small units and writing large numbers of short tests (later often called ‘microtests’), with the proviso that the tests should run very quickly and in isolation, without any database connection.

Some database developers tried to adopt these ideas and apply them to database programs by simply copying the approach, while dropping the isolation proviso. The main problem with this is that most of the code in database systems tends to be in the form of SQL statements, including complex logic and table joins. SQL is sometimes described as the only widely successful 4GL (4’th generation language), specifying joins and logic declaratively, and it is generally considered best practice to maximise its use, with procedural code (PL/SQL in Oracle) largely used to wrap the SQL into callable subprograms.

It seemed clear that microtesting was not well suited to testing programs written mostly in declarative languages such as SQL, so I felt I had to develop new ideas. These ideas were first expressed in a presentation at the Oracle User Group Ireland Conference in March 2018, The Database API Viewed As A Mathematical Function: Insights into Testing.

Since then I have developed what I now call ‘The Math Function Unit Testing Design Pattern’, and have written modules in Oracle PL/SQL, JavaScript, Powershell and Python implementing utilities to support it, and demonstrating its use.

Systems, Transactions and APIs

↑ Contents
↓ The Data Processing Module
↓ Client Transactions
↓ Types of Testing
↓ Non-Transactional Modules

The Data Processing Module

↑ Systems, Transactions and APIs

Application systems consist of different types of component, including user interfaces, databases and code. In order to facilitate automated unit testing it is convenient to separate the data processing module from components such as user interfaces, where testing is harder to automate.

The diagram shows a schematic representation of a data processing module consisting of a database (where necessary) and a set of API (Application Programming Interface) functions called by external client applications. The APIs may take inputs from various possible sources, not only parameters but potentially also, for example, database tables and operating system files; similarly they may output in various possible ways, not only via return values, but potentially writing also, for example, to database tables and operating system files. The APIs may be structured in many ways including public entry point subprograms in packages, or database views.

In an API-centred module all interfacing between the module and external applications are through these ‘transactional’ APIs. Each call effects a single transaction, where a transaction is defined simply as a complete logical unit of data activity. In this context, other components that belong to the same overall system as the data processing module, such as web pages, would be considered as external.

Where a database is present, the transaction may move the database from one logically consistent state to another consistent state, or leave it unaltered.

The transaction may also transfer a logically complete set of data to the client application, in response to the inputs supplied. The data transferred may or may not depend on data in the database, and may involve data transformations.

Client Transactions

↑ Systems, Transactions and APIs

This diagram shows a schematic representation of a single transaction triggered by a call from a client application to a single API, with database access.

This diagram shows a schematic representation of a single transaction triggered by a call from a client application to a single API, without database access.

Types of Testing

↑ Systems, Transactions and APIs
↓ Microtesting
↓ System Testing
↓ API-Based Unit Testing

Microtesting

↑ Types of Testing

Microtesting is a style of testing based on writing lots of short tests of small subprograms, focussed around internal branching logic, with dependencies excluded.

One of its proponents describes it this way in The Technical Meaning Of Microtest:

“A single microtest is a small fast chunk of code that we run, outside of our shipping source but depending on it, to confirm or deny simple statements about how that shipping source works, with a particular, but not exclusive, focus on the branching logic within it. “

As mentioned in the background section it seems to have major shortcomings when applied to higher-level programming.

However, even for application to lower-level programming it seems questionable whether the limited scope of its aims justifies the effort required: It’s quite hard in practice to test only the internal logic in isolation from all dependencies.

System Testing

↑ Types of Testing

In order for the system to be correct it is necessary for the APIs implementing the transactions to work consistently with each other, and testing that they do may be considered as system testing, along with testing of other components such as user interfaces.

This higher level testing is usually harder to automate.

API-Based Unit Testing

↑ Types of Testing

Once we view the data processing module as a set of APIs interfacing with external systems, it seems obvious that we would want to test these APIs individually before system testing. It also seems clear that we would want to test them in the common sense of testing, that for any given input the API should produce the expected output for that input, and that we should not in general exclude, or mock, subprogram calls except in special circumstances.

In well designed systems the complexity of a transaction would remain manageable even as systems increase in overall size, so that testing them would also be manageable.

Ideally, we would automate unit testing to bring all the well known benefits of automation in terms of quality and overall effort. In order to achieve this, we need to think carefully about the design of our unit test programs; in particular, we want to design so that as much testing code is reusable as possible. This is what ‘The Math Function Unit Testing Design Pattern’, described in overview in the remainder of this article, is about.

Non-Transactional Modules

↑ Systems, Transactions and APIs

In certain situations there may be a need for a module that maintains temporary internal state between API calls. In these cases there will be an underlying transaction comprised of a number of API calls that are not themselves transactional. In order to unit test such a module we can construct a wrapper function around the calls that represents the underlying transaction. Here is an example:

Timer_Set - Oracle PL/SQL Code Timing Module

API Testing with The Math Function Unit Testing Design Pattern

↑ Contents
↓ Unit Test Steps
↓ Wrapper Function
↓ Scenarios and Categories
↓ Generic Data Model
↓ Design Pattern Components
↓ Example of Wrapper Function

The design pattern is intended for fully automated, repeatable unit testing of transactional APIs, and its main features are:

The unit under test, the API, is viewed from the perspective of a mathematical function having an ‘extended signature’, comprising any actual parameters and return value, together with other inputs and outputs of any kind
A wrapper function is constructed based on this conceptual function, and this wrapper function is ‘externally pure’, in the sense that any data changes made are rolled back before returning, and it is essentially deterministic
The wrapper function performs the steps necessary to test the API in a single scenario
It takes all inputs of the extended signature as a parameter, creates any test data needed from them, effects a transaction with the API, and returns all outputs as a return value
Any test data, and any data changes made by the API, are reverted before return
The wrapper function has a fixed signature with input as a set of input groups containing arrays of records, and return value a set of output groups containing arrays of records
A library test driver module reads data for all scenarios, with both inputs to the API and the expected outputs, and metadata records describing the specific data structure
The module takes the actual outputs from the wrapper function and merges them in alongside the expected outputs to create an output results object
The output results object is processed to produce summary and detailed results in HTML or other formats

Unit Test Steps

↑ API Testing with The Math Function Unit Testing Design Pattern

At a high level the Math Function Unit Testing design pattern involves three main steps:

Create an input file containing all test scenarios with input data and expected output data for each scenario
Create a results object based on the input file, but with actual outputs merged in, based on calls to the unit under test, via the wrapper function
Use the results object to generate unit test results files formatted in HTML or text, or other format

Wrapper Function

↑ API Testing with The Math Function Unit Testing Design Pattern

This diagram shows a schematic representation of the data flows involved in a single call to the API wrapper function.

The wrapper function takes all inputs in a single, composite parameter and it outputs via a composite return value. Where the API reads from an external source the wrapper writes those data items to the source ahead of its call, and where the API writes outputs to an external source the wrapper reads from the source in order to include them in its return value.

Any external writing is reverted before the wrapper returns. If there is any indeterminacy in the API outputs the wrapper should substitute a deterministic value. In this way the wrapper function can be described as ‘externally’ pure.

Scenarios and Categories

↑ API Testing with The Math Function Unit Testing Design Pattern

The art of unit testing lies in choosing a set of scenarios that will produce a high degree of confidence in the functioning of the unit under test across the often very large range of possible inputs.

A useful approach to this can be to think in terms of categories of inputs, where we reduce large ranges to representative categories. I explore this approach further in this article:

Unit Testing, Scenarios and Categories: The SCAN Method

The diagram shows a schematic representation of a sequence of calls to the API wrapper function with input data points corresponding to categories within category sets, resulting in actual output data points that can be compared with expected outputs for the scenario.

The wrapper function specific to the API is called within a loop over scenarios by a library test driver module. The library module creates an output object combining the expected and actual outputs for all scenarios.

If we can use a generic data model for inputs and outputs of the wrapper function, this would allow us to store scenario data in a standard format, centralise the code for reading and writing scenario data from/to file, and centralise the code for looping over the scenarios calling the wrapper function.

This would mean that the only API-specific test code needed would be for converting between specific and generic inputs and outputs of the extended signature, and calling the API.

Example of a Category Structure Diagram

In determining the categories to include in scenarios it may be a good approach to draw a diagram, such as this example, taken from Net_Pipe - Oracle PL/SQL Network Analysis Module:

Generic Data Model

↑ API Testing with The Math Function Unit Testing Design Pattern

The diagram shows a generic data model consisting of a 3-level nested array for both inputs and return value for the wrapper function.

With a generic model of this kind we can use the standard JSON (Jason Object Notation) format for storing scenario data, which can be read and written in a single library module.

Example of a JSON Structure Diagram

A good start in determining the extended signature of the wrapper function may be to draw a diagram, such as this example, taken from Net_Pipe - Oracle PL/SQL Network Analysis Module:

In this example, the input Link group correspondes to a table (External Inputs in the earlier diagram), while the output group corresponds to a return value (Program Outputs in the earlier diagram)

Design Pattern Components

↑ API Testing with The Math Function Unit Testing Design Pattern

The diagram shows how the various code components work together and exchange data to go from an input scenarios JSON file to results files formatted in HTML or other formats.

Note that the green library packages do not need to be written for each API: One instance of the external package is needed in the language of the API, and only one instance of the formatting package is needed globally since it uses JSON files for input (I wrote this in JavaScript).

Example of Wrapper Function

↑ API Testing with The Math Function Unit Testing Design Pattern
↓ Scenario 6 - Network Diagram
↓ Wrapper Function - Purely_Wrap_All_Nets

This article is intended as a high level overview of the design pattern, but it is worth including one example of the wrapper function code. This example is taken from a GitHub project:

Oracle PL/SQL Network Analysis Module

“The module contains a PL/SQL package for the efficient analysis of networks that can be specified by a view representing their node pair links. The package has a pipelined function that returns a record for each link in all connected subnetworks, with the root node id used to identify the subnetwork that a link belongs to.”

We’ll show the network used as scenario 6 in testing first, followed by the wrapper function code.

Although there is a certain amount of internal complexity in the algorithm used in the API, the inputs and outputs are simple flat arrays of records, and the unit test code is quite simple.

Scenario 6 - Network Diagram

↑ Example of Wrapper Function

Wrapper Function - Purely_Wrap_All_Nets

↑ Example of Wrapper Function

Here is the complete function:

FUNCTION Purely_Wrap_All_Nets(
            p_inp_3lis                     L3_chr_arr)   -- input list of lists (group, record, field)
            RETURN                         L2_chr_arr IS -- output list of lists (group, record)
  l_act_2lis        L2_chr_arr := L2_chr_arr();
  l_csr             SYS_REFCURSOR;
BEGIN
  FOR i IN 1..p_inp_3lis(1).COUNT LOOP
    INSERT INTO network_links VALUES (p_inp_3lis(1)(i)(1), p_inp_3lis(1)(i)(2), p_inp_3lis(1)(i)(3));
  END LOOP;
  l_act_2lis.EXTEND;
  OPEN l_csr FOR SELECT * FROM TABLE(Net_Pipe.All_Nets);
  l_act_2lis(1) := Utils.Cursor_To_List(x_csr    => l_csr);
  ROLLBACK;
  RETURN l_act_2lis;
END Purely_Wrap_All_Nets;

This is a good example of how little code may be necessary when following the Math Function Unit Testing design pattern: The complexity of the code reflects only the complexity of the external interface. Here, there are single input and output groups, and so the code is equally simple, despite the unit under test having a larger degree of internal complexity.

Contents

Background

Systems, Transactions and APIs

The Data Processing Module

Client Transactions

Types of Testing

Microtesting

System Testing

API-Based Unit Testing

Non-Transactional Modules

API Testing with The Math Function Unit Testing Design Pattern

Unit Test Steps

Wrapper Function

Scenarios and Categories

Example of a Category Structure Diagram

Generic Data Model

Example of a JSON Structure Diagram

Design Pattern Components

Example of Wrapper Function

Scenario 6 - Network Diagram

Wrapper Function - Purely_Wrap_All_Nets

See Also

Articles and Presentations

JavaScript

Powershell

Oracle PL/SQL

Python