github.com/viant/assertly

Data structure testing library (assertly)

This library is compatible with Go 1.10+

Please refer to CHANGELOG.md if you encounter breaking changes.

• Introduction
• Motivation
• Usage
• Validation
• Directive
• Macros
• External Resource
• License
• Credits and Acknowledgements

Introduction

This library enables complex data structure testing, specifically:

1. Realtime transformation or casting of incompatible data types with directives system.
2. Consistent way of testing of unordered structures.
3. Contains, Range, RegExp support on any data structure deeph level.
4. Switch case directive to provide expected value alternatives based on actual switch/case input match.
5. Macro system enabling complex predicate and expression evaluation, and customization.

Motivation

This library has been created as a way to unify original testing approaches introduced to dsunit and endly

Usage

Complete data validation with concrete types

import(
	"github.com/stretchr/testify/assert"
	"github.com/viant/assertly"
)


func Test_XX(t *testing.T) {
    
   	
   	
   	var actualRecords []*User = //get actual
   	var expectedRecords []*User = //get expected
   	assertly.AssertValues(t, expectedRecords, actualRecords)
   	
   	//or with custom path and testing.T integration
   	validation, err := assertly.Assert(expected, actual, assertly.NewDataPath("/"))
   	assert.EqualValues(t, 0, validation.FailedCount, validation.Report())

   	
}

Partial data validation with directive and reg expression

func Test_XX(t *testing.T) {
    
    var actualConfig = &Config{
        Endpoint: &Endpoint{
            Port: 8080,
            TimeoutMs: 2000,
        },
        LogTypes: map[string]*LogType{
            "type1": &LogType{
                Locations:[]*Location{
                    {
                        URL:"file:///data/log/type1",
                    },
                },
                MaxQueueSize: 2048,
                QueueFlashCount: 1024,
                FlushFrequencyInMs: 500,
            },
            "type2":  &LogType{
                Locations:[]*Location{
                    {
                        URL:"file:///data/log/type2",
                    },
                },
                MaxQueueSize: 4096,
                QueueFlashCount: 2048,
                FlushFrequencyInMs: 1000,
            },
        },
    }
                           
       
    var expectedConfig = expected: `{
      "Endpoint": {
        "Port": 8080,
        "TimeoutMs": 2000
      },
      "LogTypes": {
        "type1": {
          "Locations":[
            {
              "URL":"~/type1/"
            }
          ],
          "MaxQueueSize": 2048,
          "QueueFlashCount": 1024,
          "FlushFrequencyInMs": 500
        },
        "@exists@type2": true 
      }
    }`,
    
    assertly.AssertValues(t, expectedConfig, actualConfig)
   	
}

• reg expression: "URL":"~/type1/"
• directive: @exists@

Validation with custom macro value provider

type fooProvider struct{}

func (*fooProvider) Get(context toolbox.Context, arguments ...interface{}) (interface{}, error) {
	var args = []string{}
	for _, arg := range arguments {
		args = append(args, toolbox.AsString(arg))
	}
	return fmt.Sprintf("foo{%v}", strings.Join(args, ",")), nil
}

func Test_XX(t *testing.T) {
	ctx := NewDefaultContext()
	var provider toolbox.ValueProvider = &fooProvider{}
	ctx.Evaluator.ValueProviderRegistry.Register("foo", provider)

	var actual = map[string]string{
		"k1":"v1",
		"k2":"Macro test: foo{1,abc} !",
	}
	
	var expected = map[string]string{
		"k1":"v1",
		"k2":"Macro test: <ds:foo[1,\"abc\"]> !",
	}


	AssertValuesWithContext(ctx, t, expected, actual)
}

Validation with custom predicate

type rangePredicate struct {
	min int
	max int
	actual int
	err error
}

func (p *rangePredicate) String() string {
	return fmt.Sprintf("min: %v, max: %v, actual: %v, err: %v", p.min, p.max, p.actual, p.err)
}

func (p *rangePredicate) Apply(value interface{}) bool {
	p.actual, p.err = toolbox.ToInt(value)
	return p.actual >= p.min && p.actual <= p.max
}



type inRangePredicateProvider struct{}
func (*inRangePredicateProvider) Get(context toolbox.Context, arguments ...interface{}) (interface{}, error) {
	if len(arguments) != 2 {
		return nil, fmt.Errorf("expected 2 arguments (min, max) but had: %v", len(arguments))
	}
	min, err := toolbox.ToInt(arguments[0])
	if err != nil {
		return nil, fmt.Errorf("invalid min %v", err)
	}
	max, err := toolbox.ToInt(arguments[1])
	if err != nil {
		return nil, fmt.Errorf("invalid min %v", err)
	}
	var predicate toolbox.Predicate =  &rangePredicate{min:min, max: max}
	return &predicate, nil
}



func Test_XX(t *testing.T) {
	ctx := NewDefaultContext()
	var provider toolbox.ValueProvider = &inRangePredicateProvider{}
	ctx.Evaluator.ValueProviderRegistry.Register("inRange", provider)


	var actual = map[string]int{
		"k1":1,
		"k2":3,
	}


	var expected = map[string]string{
		"k1":"1",
		"k2":"<ds:inRange[2,10]>",
	}


	AssertValuesWithContext(ctx, t, expected, actual)
}

Validation

Validation rules:

1. JSON textual data is converted into data structure
2. New Line Delimited JSON is converted into data structure collection.
3. Object/Struct is converted into data structure
4. Only existing keys/fields in expected data structure are validated
5. Only existing items in the array/slice are validated
6. Directive and macros/predicate provide validation extension
7. The following expression can be used on any data structure level:

Assertion Type	input	expected expression	example
equal	actual	expected	a:a
not equal	actual	!expected	a:!b
contains	actual	/expected/	abcd:/bc/
not contains	actual	!/expected/	abcd:!/xc/
regExpr	actual	~/expected/	1234a:/\d+/
not regExpr	actual	!~/expected/	1234:!/\w/
between	actual	/[minExpected..maxExpected]/	12:/[1..13]/
exists	n/a	{ "key": "@exists@" }
not exists	n/a	{ "key": "@!exists@" }

example:

func Test_XX(t *testing.T) {
    
var expected = `
{
  "Meta": "abc",
  "Table": "/table_/",
  "Rows": [
    {
      "id": 1,
      "name": "~/name (\\d+)/",
      "@exists@":"dob"
    },
    {
      "id": 2,
      "name": "name 2",
      "settings": {
        "k1": "v2"
      }
    },
    {
      "id": 2,
      "name": "name 2"
    }
  ]
}`,
var actual = `
{
  "Table": "table_xx",
  "Rows": [
    {
      "id": 1,
      "name": "name 12",
      "dob":"2018-01-01"
    },
    {
      "id": 2,
      "name": "name 2",
      "settings": {
        "k1": "v20"
      }
    },
    {
      "id": 4,
      "name": "name 2"
    }
  ]
}`,
	
    validation, err := assertly.Assert(expected, actual, assertly.NewDataPath("/"))
   	assert.EqualValues(t, 0, validation.FailedCount, validation.Report())
}

Directive

Directive is piece of information instructing validator to either convert data just before validation takes place or to validate a date according to provided rules.

• KeyExistsDirective = "@exists@"
• KeyDoesNotExistsDirective = "@!exists@"
• TimeFormatDirective = "@timeFormat@"
• TimeLayoutDirective = "@timeLayout@"
• SwitchByDirective = "@switchCaseBy@"
• CastDataTypeDirective = "@cast@"
• IndexByDirective = "@indexBy@"
• CaseSensitiveDirective = "@caseSensitive@"
• KeyCaseSensitiveDirective = "@CaseSensitive@"
• NumericPrecisionPointDirective = "@numericPrecisionPoint@"
• CoalesceWithZeroDirective = "@coalesceWithZero@"
• AssertPathDirective = "@assertPath@"
• LengthDirective = "@length@"
• StrictMapCheckDirective = "@strictMapCheck@"
• TimeSinceWithin = "@timeSinceWithin@"

Assert Path

@assertPath@ directive allows validation only specified path within given node, the following construct can be used:

• directive prefixed

{
    "@assertPath@Responses[0].Code":200,
    "@assertPath@Responses[1].Code":200   
}

• directive with subpath and values map

{
    "@assertPath@":{
      "Responses[0].Code":200,
      "Responses[1].Code":200
      }   
}

• directive with the same data point validation

{
    "@assertPath@":[
        {
          "Responses[0].Code":200,
          "Responses[0].Body":"/some fragment/"
      },
      {
           "Responses[0].Body":"~/.+\\d{3}.+/"
      }   
  ]
}

Index by

@indexBy@ - index by directive indexes a slice for validation process, specifically.

1. Two unordered array/slice/collection that can be index by a unique fields
2. A map with a actual array/slice/collection that can be ordered by unique fields

Example 1

#expected

{
"@indexBy@":"id",
"1" :{"id":1, "name":"name1"},
"2" :{"id":2, "name":"name2"}
}

#actual

[
{"id":1, "name":"name1"},
{"id":2, "name":"name2"}
]

Example 2

#expected

{"@indexBy@":"id"}
{"id":1, "name":"name1"}
{"id":2, "name":"name2"}

#actual

{"id":1, "name":"name1"}
{"id":2, "name":"name2"}

Example 3

#expected

{"@indexBy@":"request.id"}
{"request":{"id":1111, "name":"name1"}, "ts":189321233}
{"request":{"id":2222, "name":"name2"}, "ts":189321235}

#actual

{"request":{"id":2222, "name":"name2"}, "ts":189321235}
{"request":{"id":1111, "name":"name1"}, "ts":189321233}

Switch/case

@switchCaseBy@ - switch directive instructs a validator to select matching expected subset based on some actual value. . For non deterministic system there could be various alternative output for the same input.

Example 1

#expected

[
  {
    "@switchCaseBy@":["experimentID"]
  },
  {
    "1":{"experimentID":1, "seq":1, "outcome":[1.53,7.42,6.34]},
    "2":{"experimentID":2, "seq":1, "outcome":[3.53,6.32,3.34]}
  },
  {
    "1":{"experimentID":1, "seq":2, "outcome":[5.63,4.3]},
    "2":{"experimentID":1, "seq":2, "outcome":[3.65,3.2]}
  }
]

#actual

{"experimentID":1, "seq":1, "outcome":[1.53,7.42,6.34]}
{"experimentID":1, "seq":2, "outcome":[5.63,4.3]}

Example 2

#expected

[
  {
    "@switchCaseBy@":["experimentID"]
  },
  {
    "1":{"experimentID":1, "seq":1, "outcome":[1.53,7.42,6.34]},
    "2":{"experimentID":2, "seq":1, "outcome":[3.53,6.32,3.34]},
    "shared": {"k1":"v1", "k2":"v2"}
  },
  {
    "1":{"experimentID":1, "seq":2, "outcome":[5.63,4.3]},
    "2":{"experimentID":1, "seq":2, "outcome":[3.65,3.2]},
    "shared": {"k1":"v10", "k2":"v20"}
  }
]

#actual

{"experimentID":1, "seq":1, "outcome":[1.53,7.42,6.34], "k1":"v1", "k2":"v2"}
{"experimentID":1, "seq":2, "outcome":[5.63,4.3], "k1":"v10", "k2":"v20"}

Time format

@timeFormat@ - time format directive instructs a validator to convert data into time with specified time format before actual validation takes place.

Time format is expressed in java style date format.

Example

#expected

expected := map[string]interface{}{
    "@timeFormat@date": "yyyy-MM-dd",
    "@timeFormat@ts": "yyyy-MM-dd hh:mm:ss"
    "@timeFormat@" "yyyy-MM-dd hh:mm:ss" //default time format       
    "id":123,
    "date": "2019-01-01",
    "ts": "2019-01-01 12:00:01",
}

#actual

expected := map[string]interface{}{
	"id":123,
    "date": "2019-01-01 12:00:01",,
    "ts": "2019-01-01 12:00:01",
}

Time layout

@timeLayout@ - time format directive instructs a validator to convert data into time with specified time format before actual validation takes place.

Time layout uses golang time layout.

Example

#expected

expected := map[string]interface{}{
    "@timeFormat@date": "yyyy-MM-dd",
    "@timeFormat@ts": "yyyy-MM-dd hh:mm:ss"
    "@timeFormat@" "yyyy-MM-dd hh:mm:ss" //default time format       
    "id":123,
    "date": "2019-01-01",
    "ts": "2019-01-01 12:00:01",
}

#actual

expected := map[string]interface{}{
	"id":123,
    "date": "2019-01-01 12:00:01",,
    "ts": "2019-01-01 12:00:01",
}

Cast data type

@cast@ - instruct a validator to convert data to the specified data type before actual validation takes place.

Supported data type casting:

• int
• float
• boolean

Example

#expected

[
  {
    "@cast@field1":"float","@cast@field2":"int"
  },
  {
       "field1":2.3,
       "field2":123
  },
  {
     "field1":6.3,
     "field2":551
  }
]

#actual

{"field1":"2.3","field2":"123"}
{"field1":"6.3","field2":"551"}

KeyCaseSensitiveDirective

By default map key match is case sensitive, directive allows to disable that behaviours.

CaseSensitiveDirective

By default text value match is case sensitive, directive allows to disable that behaviours.

NumericPrecisionPoint

NumericPrecisionPoint controls numeric precision validation comparision

Example

#expected

[
  {
    "@numericPrecisionPoint@":"7"
  },
  {
      "field1":0.006521405,
       "field2":123
  },
  {
     "field1":0.006521408,
     "field2":551
  }
]

#actual

[
   {
       "field1":0.0065214,
        "field2":123
   },
   {
      "field1":0.0065214,
      "field2":551
   }
]

CoalesceWithZero

Coalesce with zero directive sets all nil numeric values to zero

Length Directive

Checks length or map or slice

Example

#expected

{
"@length@k1":3
}

#actual

   {
       "k1":[1,2,3]
   }

Source directive

Source directive is helper directive providing additional information about data point source, i.e. file.json#L113

Macro and predicates

The macro is an expression with parameters that expands original text value. The general format of macro: <ds:MACRO_NAME [json formated array of parameters]>

The following macro are build-in:

Name	Parameters	Description	Example
env	name env variable	Returns value env variable	<ds:env["user"]>
nil	n/a	Returns nil value	<ds:nil>
cast	type name	Returns value env variable	<ds:cast["int", "123"]>
current_timestamp	n/a	Returns time.Now()	<ds:current_timestamp>
dob	user age, month, day, format(yyyy-MM-dd as default)	Returns Date Of Birth	<ds:dob>

Predicates

Predicate allows expected value to be evaluated with actual data using custom predicate logic.

Name	Parameters	Description	Example
between	from, to values	Evaluate actual value with between predicate	<ds:between[1.888889, 1.88889]>
within_sec	base time, delta, optional date format	Evaluate if actual time is within delta of the base time	<ds:within_sec["now", 6, "yyyyMMdd HH:mm:ss"]>

Example

    expected := `<ds:between[1,10]>`
    actual := 3

    expected := `1<ds:env["USER"]>3`,
    actual := fmt.Sprintf("1%v3", os.Getenv("USER"))

    expected := `<ds:dob[3, 6, 3>`
    actual := 2015-06-03

    expected := `<ds:dob[3, 6, 3,"yyyy-MM-dd"]>`
    actual := 2015-06-03

    expected := `<ds:dob[3, 6, 3,"yyyy"]>`
    actual := 2015

    expected := `<ds:dob[3, 9, 2,"yyyy-MM"]>`
    actual := 2015-09

    expected := `<ds:dob[5, 12, 25,"-MM-dd"]>`
    actual := 12-25

External resource

• Advanced Data Testing with Go
• Advanced Data Testing with Go Presentation

GoCover

License

The source code is made available under the terms of the Apache License, Version 2, as stated in the file LICENSE.

Individual files may be made available under their own specific license, all compatible with Apache License, Version 2. Please see individual files for details.

Credits and Acknowledgements

Library Author: Adrian Witas