Unit testing assertion package supporting Math Function Unit Testing design pattern
The Math Function Unit Testing design pattern, implemented in nodejs
:detective:
This module supports a new design pattern for unit testing that can be applied in any language, and is here implemented in nodejs. The module name is derived from 'TRansactional API Testing' (TRAPIT), and the 'unit' should be considered to be a transactional unit (this is not micro-testing).
The Trapit module supports the full process for testing JavaScript programs, and, for non-JavaScript programs following the design pattern, formats the results by reading in a results object from a JSON file materialized by the external unit test program.
There are three short screen recordings on the module in the root folder:
These are also available in this Twitter thread.
↓ Background
↓ Usage 1 - JavaScript Unit Testing
↓ Usage 2 - Formatting Unit Test Results for External Programs
↓ API
↓ Installation
↓ Unit Testing
↓ Folder Structure
↓ See Also
↓ License
On March 23, 2018 I made the following presentation at the Oracle User Group conference in Dublin:
Database API Viewed As A Mathematical Function: Insights into Testing
The first section was summarised as:
Developing a universal design pattern for testing APIs using the concept of a 'pure' function as a wrapper to manage the 'impurity' inherent in database APIs
Although the presentation focussed on database testing the design pattern is clearly quite general.
The main features of the design pattern:
At a high level the design pattern:
The Math Function Unit Testing design pattern is centred around the idea of a 'pure' wrapper function that maps from extended input parameters to an extended return value, with both sides using a generic nested object structure.
Here is a diagram illustrating the concept of the 'externally pure' wrapper function:
The Trapit module supports the full process for testing JavaScript programs, and, for non-JavaScript programs, performs the formatting step by reading in the results object from a JSON file materialized by the external program. We'll show how this works in more detail diagrammatically in both cases here, with examples of use.
Advantages of the design pattern include:
↑ In this README...
↓ JavaScript Example 1 - colGroup
↓ JavaScript Example 2 - helloWorld
In order to use the design pattern for unit testing, the following preliminary steps are required:
Create a JSON file containing the input test data including expected return values in the required format. The input JSON file essentially consists of two objects:
meta: inp and out objects each containing group objects with arrays of field namesscenarios: scenario objects containing inp and out objects, with inp and out objects containing, for each group defined in meta, an array of input records and an array of expected output records, respectively, records being in delimited fields formatCreate a unit test script containing the wrapper function and a 1-line main section calling the Trapit library function, passing in the wrapper as a callback function. The wrapper function should call the unit under test passing the appropriate parameters and return its outputs, with the following signature:
Input parameter: 3-level list with test inputs as an object with groups as properties having 2-level arrays of record/field as values: {GROUP: [[String]], ...}
Return Value: 2-level list with test outputs as an object with groups as properties having an array of records as delimited fields strings as value: {GROUP: [String], ...}
This wrapper function may need to write inputs to, and read outputs from, files or tables, but should be 'externally pure' in the sense that any changes made are rolled back before returning, including any made by the unit under test, and should be 'essentially' deterministic.
The diagram shows the flows between input and output files:
and the three code components, where the design pattern centralizes as much code as possible in the library package:
Once the preliminary steps are executed, the script (test-uut.js, say) can be executed as follows:
$ node [path]/test-uut
The script produces listings of the results in HTML and/or text format in a subfolder with name derived from the unit test title in the input JSON file.
↑ Usage 1 - JavaScript Unit Testing
↓ Input JSON File
↓ Unit Test Wrapper Function
↓ Unit Test Scenarios
This example is a JavaScript class with a constructor function that reads in a CSV file and counts instances of distinct values in a given column. The constructor function appends a timestamp and call details to a log file. The class has methods to list the value/count pairs in several orderings.
There is a main script that shows how the class might be called outside of unit testing:
$ node examples/col-group/main-col-group
with output to console:
Counts sorted by (as is)
========================
Team #apps
----------- -----
team_name_2 1
team_name_1 1
West Brom 1219
Swansea 1180
Blackburn 33
Bolton 37
Chelsea 1147
Arsenal 534
Everton 1147
Tottenham 1288
Fulham 1209
QPR 1517
Liverpool 1227
Sunderland 1162
Man City 1099
Man Utd 1231
Newcastle 1247
Stoke City 1170
Wolves 31
Aston Villa 685
Wigan 1036
Norwich 1229
West Ham 1126
Reading 1167
...
and to log file, fantasy_premier_league_player_stats.csv.log:
Sun Sep 23 2018 13:29:07: File ./examples/col-group/fantasy_premier_league_player_stats.csv, delimiter ',', column 6
The example illustrates how a wrapper function can handle 'impure' features of the unit under test:
...and also how the JSON input file can allow for nondeterministic outputs giving rise to deterministic test outcomes:
↑ JavaScript Example 1 - colGroup
↓ col-group_temp.json
↓ col-group.json
The JSON input file contains meta and scenarios properties, as mentioned above, with structure reflecting the (extended) inputs and outputs of the unit under test. I like to make a diagram of the input and output groups, which for this example is:
An easy way to generate a starting point for the input JSON file is to use a powershell utility Powershell Utilites module to generate a template file with a single scenario with placeholder records from simple CSV files. The CSV files, col-group_inp.csv, containing input group, field pairs, and the second, col-group_out.csv, the same for output for the JSON structure diagram above would look like this:
The powershell utility can be run from a powershell window like this:
Import-Module TrapitUtils
Write-UT_Template 'col-group' '|'
This generates the JSON template file, col-group_temp.json:
{
"meta": {
"title": "title",
"delimiter": "|",
"inp": {
"Log": [
"Line"
],
"Scalars": [
"Delimiter",
"Column#"
],
"Lines": [
"Line"
]
},
"out": {
"Log": [
"#Lines",
"Date Offset",
"Text"
],
"listAsIs": [
"#Instances"
],
"sortByKey": [
"Key",
"Value"
],
"sortByValue": [
"Key",
"Value"
]
}
},
"scenarios": {
"scenario 1": {
"active_yn": "Y",
"inp": {
"Log": [
""
],
"Scalars": [
"|"
],
"Lines": [
""
]
},
"out": {
"Log": [
"||"
],
"listAsIs": [
""
],
"sortByKey": [
"|"
],
"sortByValue": [
"|"
]
}
}
}
}
The template is then updated with test data for 4 scenarios (showing just the first here):
{ "meta": {
"title": "Col Group",
"inp": {
"Log": [
"Line"
],
"Scalars": [
"Delimiter",
"Column#"
],
"Lines": [
"Line"
]
},
"out": {
"Log": [
"#Lines",
"Date Offset",
"Text"
],
"listAsIs": [
"#Instances"
],
"sortByKey": [
"Key",
"Value"
],
"sortByValue": [
"Key",
"Value"
]
}
},
"scenarios" : {
"Col 1/3; 2 duplicate lines; double-delimiter; 1-line log":
{
"active_yn" : "Y",
"inp": {
"Log": [
],
"Scalars": [
",|2"
],
"Lines": [
"0,1,Cc,3",
"00,1,A,9",
"000,1,B,27",
"0000,1,A,81"
]
},
"out": {
"Log": [
"1|IN [0, 2000]|LIKE /.*: File ./examples/col-group/ut_group.csv, delimiter ',', column 2/"
],
"listAsIs": [
"3"
],
"sortByKey": [
"A|2",
"Bx|1",
"Cc|1"
],
"sortByValue": [
"B|1",
"Cc|1",
"A|2"
]
}
},
...3 more scenarios
}}
Notice the syntax for the expected values for the second and third fields in the 3-field output record for the log group. This specifies matching against a numeric range and a regular expression, respectively, as follows:
↑ JavaScript Example 1 - colGroup
The text box below shows the entire specific unit test code for this example (short isn't it? 🙂) containing the pure wrapper function, purelyWrapUnit, and the one line main section calling the library function, Trapit.fmtTestUnit.
const [ColGroup, Trapit, fs ] =
[require('./col-group'), require('trapit'), require('fs')],
[DELIM, ROOT, ] =
['|', './examples/col-group/'];
const [INPUT_JSON, INPUT_FILE, LOG_FILE ] =
[ROOT + 'col-group.json', ROOT + 'ut_group.csv', ROOT + 'ut_group.csv.log'];
const [GRP_LOG, GRP_SCA, GRP_LIN, GRP_LAI, GRP_SBK, GRP_SBV ] =
['Log', 'Scalars', 'Lines', 'listAsIs', 'sortByKey', 'sortByValue'];
function fromCSV(csv, col) {return csv.split(DELIM)[col]};
function joinTuple(t) {return t.join(DELIM)}
function setup(inp) {
fs.writeFileSync(INPUT_FILE, inp[GRP_LIN].join('\n'));
if (inp[GRP_LOG].length > 0) {
fs.writeFileSync(LOG_FILE, inp[GRP_LOG].join('\n') + '\n');
}
return new ColGroup(INPUT_FILE, fromCSV(inp[GRP_SCA][0], 0), fromCSV(inp[GRP_SCA][0], 1));
}
function teardown() {
fs.unlinkSync(INPUT_FILE);
fs.unlinkSync(LOG_FILE);
}
function purelyWrapUnit(inpGroups) {
const colGroup = setup(inpGroups);
const linesArray = String(fs.readFileSync(LOG_FILE)).split('\n'),
lastLine = linesArray[linesArray.length - 2],
text = lastLine, //lastLine.substring(26),
date = lastLine.substring(0, 24),
logDate = new Date(date),
now = new Date(),
diffDate = now.getTime() - logDate.getTime();
teardown();
return {
[GRP_LOG] : [(linesArray.length - 1) + DELIM + diffDate + DELIM + text],
[GRP_LAI] : [colGroup[GRP_LAI]().length.toString()],
[GRP_SBK] : colGroup[GRP_SBK]().map(joinTuple),
[GRP_SBV] : colGroup[GRP_SBV]().map(joinTuple)
};
}
Trapit.fmtTestUnit(INPUT_JSON, ROOT, purelyWrapUnit);
Calling this script, from, say, a Powershell window in the Trapit root folder generates the detailed results files in a subfolder.
$ node ./examples/col-group/test-col-group
↑ JavaScript Example 1 - colGroup
↓ Input Data Category Sets
↓ Scenario Results
The art of unit testing lies in choosing a set of scenarios that will produce a high degree of confidence in the functioning of the unit under test across the often very large range of possible inputs.
A useful approach to this can be to think in terms of categories of inputs, where we reduce large ranges to representative categories. In our case we might consider the following category sets, and create scenarios accordingly:
↑ Unit Test Scenarios
↓ Value Size (key)
↓ Multiplicity
Check short string key values and long ones do not cause errors
Multiplicity is a very commonly applicable generic category set: programs often behave differently according to multiplicity of some variable, where zero is often an edge case, while 1 and more than 1 value may both need to be checked. Here we can consider its application to three variables:
Apply to:
↑ Unit Test Scenarios
↓ Results Summary
↓ Results for Scenario 1: Col 3/4; 4 lines; comma-delimiter; no log
As well as the results folder, the script also generates a summary of the unit test results to the console window:
Unit Test Results Summary for Folder ./examples/col-group/
==========================================================
File: col-group.json
Title: Col Group
Inp Groups: 3
Out Groups: 4
Tests: 4
Fails: 2
Folder: col-group
It shows that there were two scenarios tested, both of which failed (deliberately). The full results listings are in the subfolder col-group in both text and HTML format. Here is the summary page in text format:
Unit Test Report: Col Group
===========================
# Scenario Fails (of 4) Status
--- ----------------------------------------------------------------------------- ------------ -------
1* Col 3/4; 4 lines; comma-delimiter; no log 1 FAILURE
2* Col 1/3; 2 duplicate lines; double-delimiter; 1-line log 1 FAILURE
3 Col 10/10; 3 lines, 2 with same 50-character key; comma-delimiter; 1-line log 0 SUCCESS
4 Col 10/10; empty file edge-case; comma-delimiter; 1-line log 0 SUCCESS
Test scenarios: 2 failed of 4: FAILURE
======================================
Here are the results for the first scenario in text format:
SCENARIO 1: Col 3/4; 4 lines; comma-delimiter; no log {
=======================================================
INPUTS
======
GROUP 1: Log: Empty
===================
GROUP 2: Scalars {
==================
# Delimiter Column#
- --------- -------
1 , 2
}
=
GROUP 3: Lines {
================
# Line
- -----------
1 0,1,Cc,3
2 00,1,A,9
3 000,1,B,27
4 0000,1,A,81
}
=
OUTPUTS
=======
GROUP 1: Log {
==============
# #Lines Date Offset Text
- ------ ----------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------
1 1 IN [0, 2000]: 490 LIKE /.*: File ./examples/col-group/ut_group.csv, delimiter ',', column 2/: Sat Jul 10 2021 14:47:35: File ./examples/col-group/ut_group.csv, delimiter ',', column 2
} 0 failed of 1: SUCCESS
========================
GROUP 2: listAsIs {
===================
# #Instances
- ----------
1 3
} 0 failed of 1: SUCCESS
========================
GROUP 3: sortByKey {
====================
# Key Value
-- --- -----
1 A 2
2 Bx 1
2* B 1
3 Cc 1
} 1 failed of 3: FAILURE
========================
GROUP 4: sortByValue {
======================
# Key Value
- --- -----
1 B 1
2 Cc 1
3 A 2
} 0 failed of 3: SUCCESS
========================
} 1 failed of 4: FAILURE
========================
Note the record #2 above marked with a '*' in 'GROUP 3: sortByKey'. Where the actual value differs from expected the actual record is listed below the expected, with the '*' marker against the record number, and in the HTML report the record is coloured red.
↑ Usage 1 - JavaScript Unit Testing
↓ Input JSON File
↓ Unit Test Wrapper Function
↓ Unit Test Scenarios
module.exports = {
helloWorld: () => {return 'Hello World!'}
}
This is a pure function form of Hello World program, returning a value rather than writing to screen itself. It is of course trivial, but has some interest as an edge case with no inputs and extremely simple JSON input structure and test code.
There is a main script that shows how the function might be called outside of unit testing:
$ node examples/hello-world/main-hello-world
with output to console:
Hello World!
↑ JavaScript Example 2 - helloWorld
↓ hello-world.json
The JSON structure diagram for this trivial example is:
The input JSON file, showing empty input property in the meta and scenarios objects, is:
{ "meta": {
"title": "Hello World",
"inp": {},
"out": {
"Group": [
"Greeting"
]
}
},
"scenarios" : {
"Scenario":
{
"inp": {},
"out": {
"Group": [
"Hello World!"
]
}
}
}}
↑ JavaScript Example 2 - helloWorld
The text box below shows the entire specific unit test code for this example. In this trivial case, we can pass the pure wrapper function as a lambda expression.
const [Trapit, Hw ] =
[require('trapit'), require('./hello-world')],
[ROOT, GROUP ] =
['./examples/hello-world/', 'Group' ];
const INPUT_JSON = ROOT + 'hello-world.json';
Trapit.fmtTestUnit(INPUT_JSON, ROOT, (inpGroups) => { return {[GROUP] : [Hw.helloWorld()]} });
Calling this script, from, say, a Powershell window in the Trapit root folder generates the detailed results files in a subfolder.
$ node ./examples/hello-world/test-hello-world
↑ JavaScript Example 2 - helloWorld
↓ Input Data Category Sets
↓ Scenario Results
With no input data, the set of input data category sets is of course empty 🙂.
As well as the results folder, the script also generates a summary of the unit test results to the console window:
Unit Test Results Summary for Folder ./examples/hello-world/
============================================================
File: hello-world.json
Title: Hello World
Inp Groups: 0
Out Groups: 1
Tests: 1
Fails: 0
Folder: hello-world
It shows that there was one scenario tested, which succeeded. The full results listings are in the subfolder hello-world in both text and HTML format. Here is the full set of results in text format:
Unit Test Report: Hello World
=============================
# Scenario Fails (of 1) Status
--- -------- ------------ -------
1 Scenario 0 SUCCESS
Test scenarios: 0 failed of 1: SUCCESS
======================================
SCENARIO 1: Scenario {
======================
INPUTS
======
OUTPUTS
=======
GROUP 1: Group {
================
# Greeting
- ------------
1 Hello World!
} 0 failed of 1: SUCCESS
========================
} 0 failed of 1: SUCCESS
========================
↑ In this README...
↑ Results Summaries for External Folders
For non-JavaScript programs tested using the Math Function Unit Testing design pattern, the results object is materialized using a library package in the relevant language. The diagram below shows how the processing is split into two steps:
This creates a subfolder for each JSON file with name based on the unit test title within the file, and also outputs a table of summary results for each file. The diagram below shows how in the first step the processing is split between three code units in a similar way to the JavaScript case:
In the second step the output results JSON file is read into an object by the Trapit library package, which then formats the results in exactly the same way as for JavaScript testing.
The following script (format-externals.js) can be used to execute the second step, passing as a parameter the name of a subfolder for the output JSON files, relative to the root folder specified in the script.
const [ROOT, subFolder, Trapit ] =
['./externals', process.argv[2], require('trapit')];
Trapit.tabMkUTExternalResultsFolders(ROOT + (subFolder === undefined ? '' : '/' + subFolder), 'B'); // H/T/B : Format in HTML/Text/Both
The script can be run from a Powershell window in the root trapit folder for a subfolder in the externals folder containing a set of JSON results files:
$ node externals\format-externals subfolder
↑ Usage 2 - Formatting Test Results for External Programs
↓ oracle_api_demos
↓ oracle_plsql
↓ powershell
↓ python
Here we give the top-level results summaries output to console for each of the groups of externally-sourced JSON files. Links to the source GitHub project are included for each group.
↑ Results Summaries for External Folders
The results JSON file is sourced from the following GitHub project, and the formatted results files can be seen in the indicated subfolders:
Running the format-externals script for subfolder oracle_api_demos from a Powershell window in the root trapit folder:
$ node externals\format-externals oracle_api_demos
gives the following output to console, as well as writing the results subfolders as indicated:
Unit Test Results Summary for Folder ./externals/oracle_api_demos
=================================================================
File Title Inp Groups Out Groups Tests Fails Folder
---------------------------------------------------- ------------------------------------------------------- ---------- ---------- ----- ----- -------------------------------------------------------
tt_emp_batch.purely_wrap_load_emps_out.json Oracle PL/SQL API Demos: TT_Emp_Batch.Load_Emps 5 5 9 0 oracle-pl_sql-api-demos_-tt_emp_batch.load_emps
tt_emp_ws.purely_wrap_get_dept_emps_out.json Oracle PL/SQL API Demos: TT_Emp_WS.Get_Dept_Emps 2 2 5 0 oracle-pl_sql-api-demos_-tt_emp_ws.get_dept_emps
*tt_emp_ws.purely_wrap_save_emps_out.json Oracle PL/SQL API Demos: TT_Emp_WS.Save_Emps 1 4 4 1 oracle-pl_sql-api-demos_-tt_emp_ws.save_emps
tt_view_drivers.purely_wrap_hr_test_view_v_out.json Oracle PL/SQL API Demos: TT_View_Drivers.HR_Test_View_V 2 2 4 0 oracle-pl_sql-api-demos_-tt_view_drivers.hr_test_view_v
1 externals failed, see ./externals/oracle_api_demos for scenario listings
tt_emp_ws.purely_wrap_save_emps_out.json
↑ Results Summaries for External Folders
The results JSON files are sourced from the following GitHub projects, and the formatted results files can be seen in the indicated subfolders:
Running the format-externals script for subfolder oracle_plsql from a Powershell window in the root trapit folder:
$ node externals\format-externals oracle_plsql
gives the following output to console, as well as writing the results subfolders as indicated:
Unit Test Results Summary for Folder ./externals/oracle_plsql
=============================================================
File Title Inp Groups Out Groups Tests Fails Folder
-------------------------------------------- ------------------------------ ---------- ---------- ----- ----- ------------------------------
tt_log_set.purely_wrap_log_set_out.json Oracle PL/SQL Log Set 6 6 21 0 oracle-pl_sql-log-set
tt_net_pipe.purely_wrap_all_nets_out.json Oracle PL/SQL Network Analysis 1 2 3 0 oracle-pl_sql-network-analysis
tt_timer_set.purely_wrap_timer_set_out.json Oracle PL/SQL Timer Set 2 9 8 0 oracle-pl_sql-timer-set
tt_utils.purely_wrap_utils_out.json Oracle PL/SQL Utilities 15 16 4 0 oracle-pl_sql-utilities
0 externals failed, see ./externals/oracle_plsql for scenario listings
↑ Results Summaries for External Folders
The results JSON file is sourced from the following GitHub project, and the formatted results files can be seen in the indicated subfolder:
Running the format-externals script for subfolder powershell from a Powershell window in the root trapit folder:
$ node externals\format-externals powershell
gives the following output to console, as well as writing the results subfolders as indicated:
Unit Test Results Summary for Folder ./externals/powershell
===========================================================
File Title Inp Groups Out Groups Tests Fails Folder
-------------------------------- --------------------- ---------- ---------- ----- ----- ---------------------
get_ut_template_object_out.json Get-UT_TemplateObject 3 4 10 0 get-ut_templateobject
ps_utils_out.json Powershell Utils 7 6 6 0 powershell-utils
0 externals failed, see ./externals/powershell for scenario listings
↑ Results Summaries for External Folders
The results JSON file is sourced from the following GitHub project, and the formatted results files can be seen in the indicated subfolder:
Running the format-externals script for subfolder python from a Powershell window in the root trapit folder:
$ node externals\format-externals python
gives the following output to console, as well as writing the results subfolders as indicated:
Unit Test Results Summary for Folder ./externals/python
=======================================================
File Title Inp Groups Out Groups Tests Fails Folder
------------------ -------- ---------- ---------- ----- ----- --------
timerset_out.json timerset 2 8 7 0 timerset
0 externals failed, see ./externals/python for scenario listings
↑ In this README...
↓ Trapit.testUnit(inpFile, root, purelyWrapUnit, formatType = 'B')
↓ Trapit.fmtTestUnit(inpFile, root, purelyWrapUnit, formatType = 'B')
↓ Trapit.mkUTExternalResultsFolders(extFolder, formatType = 'B')
↓ Trapit.tabMkUTExternalResultsFolders(extFolder, formatType = 'B')
const Trapit = require('trapit');
This is the base entry point for testing JavaAcript programs. It writes the output results folder and returns a value containing summary data for the unit test. It has the following parameters:
inpFile: JSON input fileroot: root folder, where the results output files are to be written, in a subfolder with name based on the report titlepurelyWrapUnit: wrapper function, which calls the unit under test passing the appropriate parameters and returning its outputs, with the following signature:
formatType: format type = H/T/B - Format in HTML/Text/Both; default 'B'and object return value with the following fields:
nTest: number of test scenariosnFail: number of test scenarios that failedstatus: status = SUCCESS/FAILresFolder: name of results subfoldernInpGroups: number of input groupsnOutGroups: number of output groupstitle: unit test titleThis is a wrapper function that calls the base entry point Trapit.testUnit with the same parameters and prints its return object to console.
This is the base entry point for formatting results JSON files from external programs. It writes the output results folders for each file in the external folder, and returns a value containing unit test summary data for the JSON files as an array of objects. It has the following parameters:
extFolder: external folder, where the results output files are to be written, in a subfolder with name based on the report titleformatType: format type = H/T/B - Format in HTML/Text/Both; default 'B'and array return value with the following fields:
file: JSON results file namenTest: number of test scenariosnFail: number of test scenarios that failedstatus: status = SUCCESS/FAILresFolder: name of results subfoldernInpGroups: number of input groupsnOutGroups: number of output groupstitle: unit test titleThis is a wrapper function that calls the base entry point Trapit.mkUTExternalResultsFolders with the same parameters and prints its return array in tabular format to console.
With npm installed, run
$ npm install trapit
↑ In this README...
↓ Wrapper Function Extended Signature
↓ Unit Test Scenarios
The package itself is tested using the Math Function Unit Testing design pattern. A 'pure' wrapper function is constructed that takes input parameters and returns a value, and is tested within a loop over scenario records read from a JSON file.
In this case, the pure function getUTResults is unit tested explicitly, while the function Test-Unit is called as the main section of the unit test script, test-trapit.js.
The program is data-driven from the input file trapit.json and produces listings of the results in HTML and/or text format.
To run the unit test program, open a powershell window in the test folder and run:
$ npm test
This creates a subfolder trapit in the test folder with text results in trapit.txt, the HTML results summary file, trapit.html, and a separate HTML file for each scenario linked to in the summary file.
[An easy way to generate a starting point for the input JSON file is to use a powershell utility Powershell Utilites module to generate a template file with a single scenario with placeholder records from simple .csv files. See the script trapit.ps1 in the test subfolder for an example]
↑ Unit Testing
↓ Input Data Category Sets
↓ Scenario Results
↑ Unit Test Scenarios
↓ Multiplicity
↓ Test Status
↓ Match Type
↓ Exception
The art of unit testing lies in choosing a set of scenarios that will produce a high degree of confidence in the functioning of the unit under test across the often very large range of possible inputs.
A useful approach to this can be to think in terms of categories of inputs, where we reduce large ranges to representative categories. In our case we might consider the following category sets, and create scenarios accordingly:
Input and output sections are split into groups, and we should check the zero edge cases as well as 1 and multiple groups in each section. Similarly there may be zero or more records in each type of group, and 1 or more fields in each type of group.
Apply to:
Check both passing and failing tests processed correctly.
Values may have exact matching specified, or inexact, with regular expression matching for strings and ranges for numbers, and both types of matching should be checked for both data types. Null values may cause incorrect matching and this can also be checked.
There can be a mismatch in the number of groups or of fields within a group between the metadata and the scenarios section, caused by a programming error, and it is desired to return an easy to understand exception in these cases.
↑ Unit Test Scenarios
↓ Results Summary
↓ Results for Scenario 1: No inp, 1 out group - simple edge-case - Passing
The summary report in text format shows the scenarios tested:
# Scenario Fails (of 5) Status
--- ---------------------------------------------------------------------------------------------- ------------ -------
1 No inp, 1 out group - simple edge-case - Passing 0 SUCCESS
2 1 inp, no out group - simple edge-case - Passing 0 SUCCESS
3 No inp, 1 out group - null value in number range bug fix - should fail 0 SUCCESS
4 2 inp/out groups - 1-field/2-field groups; empty/non-empty groups; non-exact matches - Passing 0 SUCCESS
5 2 inp/out groups - 1-field/2-field groups; empty/non-empty groups; non-exact matches - Failing 0 SUCCESS
6 No inp, 1 out group - fields mismatch - Exception 0 SUCCESS
7 No inp, 1 out group - groups mismatch - Exception 0 SUCCESS
SCENARIO 1: No inp, 1 out group - simple edge-case - Passing {
==============================================================
INPUTS
======
GROUP 1: Report {
=================
# Title
- ---------------------
1 Hello World Edge Case
}
=
GROUP 2: Input Fields: Empty
============================
GROUP 3: Output Fields {
========================
# Group Field
- -------------------- ------
1 1-field output group Single
}
=
GROUP 4: Input Values: Empty
============================
GROUP 5: Expected Values {
==========================
# Scenario Group Row CSV
- ------------------- -------------------- ------------
1 No inp, 1 out group 1-field output group Hello World!
}
=
GROUP 6: Actual Values {
========================
# Scenario Group Row CSV
- ------------------- -------------------- ------------
1 No inp, 1 out group 1-field output group Hello World!
}
=
OUTPUTS
=======
GROUP 1: Summaries {
====================
# Type Scenario Group Header 1 Header 2 Footer # Records # Fails Status
- ------------ ------------------- -------------------- --------------------------------------- -------------- -------------------------------------- --------- ------- -------
1 Report Unit Test Report: Hello World Edge Case Test scenarios: 0 failed of 1: SUCCESS 1 0 SUCCESS
2 Scenario No inp, 1 out group SCENARIO 1: No inp, 1 out group INPUTS-OUTPUTS 0 failed of 1: SUCCESS 1 0 SUCCESS
3 Output group No inp, 1 out group 1-field output group GROUP 1: 1-field output group 0 failed of 1: SUCCESS 1 0 SUCCESS
} 0 failed of 3: SUCCESS
========================
GROUP 2: Input Fields: Empty as expected: SUCCESS
=================================================
GROUP 3: Output Fields {
========================
# Scenario Group Name Length
- ------------------- -------------------- ------ ------
1 No inp, 1 out group 1-field output group # 1
2 No inp, 1 out group 1-field output group Single 12
} 0 failed of 2: SUCCESS
========================
GROUP 4: Input Values: Empty as expected: SUCCESS
=================================================
GROUP 5: Output Values {
========================
# Scenario Group Row CSV
- ------------------- -------------------- --------------
1 No inp, 1 out group 1-field output group 1~Hello World!
} 0 failed of 1: SUCCESS
========================
} 0 failed of 5: SUCCESS
========================
You can review the formatted unit test results here, Unit Test Report: Trapit, and the files are available in the test/trapit subfolder [trapit.html is the root page for the HTML version and trapit.txt has the results in text format].
The project folder structure is shown below.
There are four subfolders below the trapit root folder:
examples: Two working JavaScript examples are included in their own subfolders, with both test scripts and a main script that shows how the unit under test would normally be calledexternals: Root folder for the examples provided of formatting externally-sourced results JSON files. These files all come from other GitHub projects with links given in the section Results Summaries for External Folders above. These are grouped into subfolders which can be extended as desiredlib: This holds the project source codetest: Root folder for unit testing of the Trapit project itself, with subfolder having the results filesMIT
FAQs
Trapit - JavaScript Unit Tester/Formatter
The npm package trapit receives a total of 30 weekly downloads. As such, trapit popularity was classified as not popular.
We found that trapit demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 0 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
A malicious npm package disguised as a WhatsApp client is exploiting authentication flows with a remote kill switch to exfiltrate data and destroy files.
Security News
PyPI now supports digital attestations, enhancing security and trust by allowing package maintainers to verify the authenticity of Python packages.
Security News
GitHub removed 27 malicious pull requests attempting to inject harmful code across multiple open source repositories, in another round of low-effort attacks.