json-spec

CabezaDeTermo::JsonSpec

A framework to declare expectations and verify that a json object satisfies those expectations. You can use this expectations to validate jsons you send or receive in your application, or to test your API with unit tests.

Status

Installation

Add this line to your application's Gemfile:

gem 'json-spec', '~> 0.2'

And then execute:

$ bundle

Or install it yourself as:

$ gem install json-spec

Usage

If you want to read the supported expressions and expectations by json-spec, see the API documentation.

If you want to jump to a running example, see the full example.

Otherwise walk through this tutorial where we will be writting a spec to validate a partial specification of the composer.json used by a dyslexic cousin of this project:

{
	"name": "cabeza-de-termo/json-spec",
	"type": "library",
	"description": "A framework to declare expectations and verify that a json object complies with those expectations. You can use this json expectations to validate jsons you send or receive in your application, or to test your API with unit tests.",
	"keywords": ["json", "assertions", "expectations", "validation", "phpunit"],
	"homepage": "https://github.com/cabeza-de-termo/php-json-spec",
	"license": "MIT",
	"authors": [
		{
			"name": "Martin Rubi",
			"email": "[e-mail]"
		}
	],
	"require": {
		"php": ">=5.4.0"
	},
	"require-dev": {
		"phpunit/phpunit": "^4",
		"phpdocumentor/phpdocumentor": "2.*"
	},
	"autoload": {
		"psr-4": {
			"CabezaDeTermo\\JsonSpec\\": ["src/"],
			"CabezaDeTermo\\JsonSpec\\Tests\\": ["tests/"]
		}
	}
}

Starting with a simple validation

We will declare each expression we expect the composer.json to have, and for each expression we will declare expectations on it:

require 'cabeza-de-termo/json-spec/json-spec'

valid_licenses = 
	['Apache-2.0', 'BSD-2-Clause', 'BSD-3-Clause', 
	 'BSD-4-Clause' ,'GPL-2.0', 'GPL-2.0+', 'GPL-3.0', 
	 'GPL-3.0+', 'LGPL-2.1', 'LGPL-2.1+', 'LGPL-3.0',
	 'LGPL-3.0+', 'MIT']

json_spec = CabezaDeTermo::JsonSpec::JsonSpec.new do
	expect_an(:object) do
		expect('name') .to_be_defined .not_blank
		expect('type') .to_be_defined .not_blank
		expect('description') .to_be_defined .not_blank
		expect('keywords') .to_be_a(:list) .can_be_absent .not_empty do
			each do
				expect_a(:scalar) .not_blank
			end
		end
		expect('homepage') .to_be_url
		expect('license') .to_be_defined .to_be_in(valid_licenses)
		expect('authors') .to_be_a(:list) .to_be_defined .not_empty do
			each do
				expect_an(:object) do
					expect('name') .to_be_defined .not_blank
					expect('email') .to_be_defined .to_be_email
				end
			end
		end
		expect('require') .to_be_an(:object) .can_be_absent
		expect('require-dev') .to_be_an(:object) .can_be_absent
		expect('autoload') .to_be_an(:object) .to_be_defined do
			expect('psr-0') .to_be_an(:object) .can_be_absent
			expect('psr-4') .to_be_an(:object) .can_be_absent
		end
	end
end

That's it. Now we can validate a json string by running:

validator = json_spec.validate_string json_string

puts validator.errors
puts validator.unexpected_fields

If you want to run this example, see the first example in the examples/ folder.

Defining default expectations for each expression

One thing you may have noticed about the previous example is that it includes a lot of repeated .to_be_defined expectations on many fields. It would be easier if we could just state that .to_be_defined is expected for every field.

We can do that in two different ways.

If we want to declare default expectations at a global scope, i.e., for every expression used in any place, then we can do:

CabezaDeTermo::JsonSpec::ExpectationsLibrary.define do
	default_expectations do
		for_every_field do
			to_be_defined
			to_be_string
			not_blank
			...
		end
	end
end

If we want to declare default expectations for a json_spec only then we can do:

json_spec.define do
	default_expectations do
		for_every_field do
			to_be_defined
			to_be_string
			not_blank
			...
		end
	end
end

We can declare default expectations at any scope for different expressions:

default_expectations do
	for_every_object do
		...
	end

	for_every_list do
		...
	end

	for_every_scalar do
		...
	end

	for_every_field do
		...
	end
end

If we want to get rid of the current default expressions, in the corresponding scope we declare any of:

default_expectations do
	drop_all_expectations
	drop_expectations_for(:objects)
	drop_expectations_for(:lists)
	drop_expectations_for(:fields)
	drop_expectations_for(:scalars)
end

You can see which default expectations are used by the framework in the DefaultLibraryInitializer class.

So, back to our example, now the validation looks like:

CabezaDeTermo::JsonSpec::ExpectationsLibrary.define do
	default_expectations do
		for_every_field do
			to_be_defined
		end
	end
end

valid_licenses = 
	['Apache-2.0', 'BSD-2-Clause', 'BSD-3-Clause', 
	 'BSD-4-Clause' ,'GPL-2.0', 'GPL-2.0+', 'GPL-3.0', 
	 'GPL-3.0+', 'LGPL-2.1', 'LGPL-2.1+', 'LGPL-3.0',
	 'LGPL-3.0+', 'MIT']

json_spec = CabezaDeTermo::JsonSpec::JsonSpec.new do
	expect_an(:object) do
		expect('name') .not_blank
		expect('type') .not_blank
		expect('description') .not_blank
		expect('keywords') .to_be_a(:list) .can_be_absent .not_empty do
			each do
				expect(:scalar) .not_blank
			end
		end
		expect('homepage') .to_be_url
		expect('license') .to_be_in(valid_licenses)
		expect('authors') .to_be_a(:list) .not_empty do
			each do
				expect_an(:object) do
					expect('name') .not_blank
					expect('email') .to_be_email
				end
			end
		end
		expect('require') .to_be_an(:object) .can_be_absent
		expect('require-dev') .to_be_an(:object) .can_be_absent
		expect('autoload') .to_be_an(:object) do
			expect('psr-0') .to_be_an(:object) .can_be_absent
			expect('psr-4') .to_be_an(:object) .can_be_absent
		end
	end
end

If you want to run this example, see the second example in the examples/ folder.

Declaring expectations for fields not known in advance.

If you run the previous example, you may notice that it outputs the following:

- Failed expectations: 0

- Unexpected fields: 5
	Field: '@.require'	 message: "An unexpected 'php' field was found."
	Field: '@.require-dev'	 message: "An unexpected 'phpunit/phpunit' field was found."
	Field: '@.require-dev'	 message: "An unexpected 'phpdocumentor/phpdocumentor' field was found."
	Field: '@.autoload.psr-4'	 message: "An unexpected 'CabezaDeTermoJsonSpec\' field was found."
	Field: '@.autoload.psr-4'	 message: "An unexpected 'CabezaDeTermo\JsonSpec\Tests\' field was found."

That is because the composer.json has the following sections:

"require": {
	"php": ">=5.4.0"
},
"require-dev": {
	"phpunit/phpunit": "^4",
    "phpdocumentor/phpdocumentor": "2.*"
}

...

	"psr-4": {
		"CabezaDeTermoJsonSpec\\\\": ["src/"],
		"CabezaDeTermo\\\\JsonSpec\\\\Tests\\\\": ["tests/"]
	}

with fields with names that we don't know in advance. They can be anything, as long as they comply with what composer.json expects. So we also want to declare expectations on unkown fields.

To do that, we use the :each_field expectation:

expect('require') .to_be_an(:object) .can_be_absent do
	each_field do
		expect_name .not_blank
		expect_a(:scalar) .not_blank
	end
end
...
expect('psr-0') .to_be_an(:object) .can_be_absent do
	each_field do
		expect_name

		expect_a(:list) .not_empty do
			each do
				expect_a(:scalar) .not_blank
			end
		end
	end
end

If you want to run this example, see the third example in the examples/ folder.

Refactoring the expectations

So by now we have the following expectations:

json_spec = CabezaDeTermo::JsonSpec::JsonSpec.new do
	expect_an(:object) do
		expect('name') .not_blank
		expect('type') .not_blank
		expect('description') .not_blank

		expect('keywords') .to_be_a(:list) .can_be_absent .not_empty do
			each do
				expect_a(:scalar) .not_blank
			end
		end

		expect('homepage') .to_be_url
		expect('license') .to_be_in(valid_licenses)

		expect('authors') .to_be_a(:list) .not_empty do
			each do
				expect_an(:object) do
					expect('name') .not_blank
					expect('email') .to_be_email
				end
			end
		end

		expect('require') .to_be_an(:object) .can_be_absent do
			each_field do
				expect_name .not_blank
				expect_a(:scalar) .not_blank
			end
		end

		expect('require-dev') .to_be_an(:object) .can_be_absent do
			each_field do
				expect_name.not_blank
				expect_a(:scalar) .not_blank
			end
		end

		expect('autoload') .to_be_an(:object) do
			expect('psr-0') .to_be_an(:object) .can_be_absent do
				each_field do
					expect_name

					expect_a(:list) .not_empty do
						each do
							expect_a(:scalar) .not_blank
						end
					end
				end
			end

			expect('psr-4') .to_be_an(:object) .can_be_absent do
				each_field do
					expect_name

					expect_a(:list) .not_empty do
						each do
							expect_a(:scalar) .not_blank
						end
					end
				end
			end

			expect('classmap') .to_be_a(:list) .can_be_absent

			expect('files') .to_be_a(:list) .can_be_absent
		end
	end
end

This have several problems. One is its extension. The spec just got too long. Second, now it is a structure of expectations on json expressions that lacks of some intention revealing names about the expressions. For instance, this section

expect_an(:object) do
	expect('name') .not_blank
	expect('email') .to_be_email
end

refers to an author, but if we only look at it without its parent expression, we don't know that. And third, and worst than the previous reasons, is that we are duplicating expectations for require and require-dev.

So it would be nice to be able to organize the expectations somehow.

We can put expectations in methods, and then call those methods from within the json_spec. This methods can be in any object, but we are going to create a ComposerJson class to keep the composer.json expectations in one place. So now we have the class

class ComposerJson
	def self.validate_string(json_string)
		self.new.spec.validate_string(json_string)
	end

	def spec()
		CabezaDeTermo::JsonSpec::JsonSpec.new do |json_spec|
			json_spec.expect_an(:object) do |object|
				object.expect('name') .not_blank
				object.expect('type') .not_blank
				object.expect('description') .not_blank
				object.expect('keywords') .to_be_as_defined_in(self, :keywords_spec)
				object.expect('homepage').to_be_url
				object.expect('license') .to_be_in(self.valid_licenses)
				object.expect('authors') .to_be_as_defined_in(self, :authors_spec)
				object.expect('require') .to_be_as_defined_in(self, :require_spec)
				object.expect('require-dev') .to_be_as_defined_in(self, :require_spec)
				object.expect('autoload') .to_be_as_defined_in(self, :autoload_spec)
			end
		end
	end

	def keywords_spec(json_spec)
		json_spec .to_be_a(:list) .can_be_absent .not_empty do
			each do
				expect_a(:scalar) .not_blank
			end
		end
	end

	def authors_spec(json_spec)
		json_spec .to_be_a(:list) .not_empty do |list|
			list.each do |each|
				each.to_be_as_defined_in(self, :author_spec)
			end
		end
	end

	def author_spec(json_spec)
		json_spec .expect_an(:object) do
			expect('name') .not_blank
			expect('email') .to_be_email
		end
	end

	def require_spec(json)
		json .to_be_an(:object) .can_be_absent do
			each_field do
				expect_name .not_blank
				expect_a(:scalar) .not_blank
			end
		end
	end

	def autoload_spec(json)
		json .to_be_an(:object) do |object|
			object.expect('psr-0') .to_be_as_defined_in(self, :psr_spec)
			object.expect('psr-4') .to_be_as_defined_in(self, :psr_spec)
			object.expect('classmap') .to_be_as_defined_in(self, :classmaps_spec)
			object.expect('files') .to_be_as_defined_in(self, :files_spec)
		end
	end

	def psr_spec(json_spec)
		json_spec .to_be_an(:object) .can_be_absent do
			each_field do
				expect_name

				expect_a(:list) .not_empty do
					each do
						expect_a(:scalar) .not_blank
					end
				end
			end
		end
	end

	def classmaps_spec(json_spec)
		json_spec .to_be_a(:list) .can_be_absent .not_empty
	end

	def files_spec(json_spec)
		json_spec .to_be_a(:list) .can_be_absent .not_empty
	end

	def valid_licenses()
		['Apache-2.0', 'BSD-2-Clause', 'BSD-3-Clause', 
		 'BSD-4-Clause' ,'GPL-2.0', 'GPL-2.0+', 'GPL-3.0', 
		 'GPL-3.0+', 'LGPL-2.1', 'LGPL-2.1+', 'LGPL-3.0',
		 'LGPL-3.0+', 'MIT']
	end
end

and to run the validation all we have to do is

validator = ComposerJson.validate_string(json_string)

If you want to validate a not very complex json, you can get away with it without factorizing the expectations. But as soon as the validation gets more complex, you can refactor the expectations using :to_be_as_defined_in(some_object, :some_method).

One thing to notice in this example is that we declared things like

CabezaDeTermo::JsonSpec::JsonSpec.new do |json_spec|
	json_spec.expect_an(:object) do |object|
		...
	end
end

instead of

CabezaDeTermo::JsonSpec::JsonSpec.new do
	json_spec.expect_an(:object) do
		...
	end
end

That is because when we don't pass a parameter to the defintion block, it changes the binding of self, and then we can not declare thinkgs like

.to_be_as_defined_in(self, :keywords_spec)

Changing the binding of self without asking first is not a polite thing to do, but in the definition blocks of this framework it will only do that if no parameter is given to the block.

If you want to run this example, see the fourth example in the examples/ folder.

Expecting different structures for the same expression

If we look at the composer.json definition, we will notice that sometimes it accepts different structures for the same field. For instance, for psr-4 values, it can take a list of folder strings or a single folder string.

To expect different structures on the same object, we use expect(:any_of):

expect('psr-4') .to_be_a(:list) do
	each do
		expect(:any_of) do
			expect_a(:scalar) .to_be_folder
		or_also
			expect_a(:list) .not_empty do
				each do
					expect_a(:scalar) .to_be_folder
				end
			end
		end
	end
end

Defining custom expectations

So far we only used the expectations defined by the json-spec framework. But more likely we will want to define our custom expectations. There are several reasons for that. Different APIs use different formats, or in some contexts we may want to use more intention revealing expectation names, to name a few.

There are many ways to define new expectations, let's go through them:

• Delegate the new expectation to an existing one.

Suppose we want to check if a value is equal to 42. We can achieve that by doing

json_spec .to_be_equal_to(42)

But if we want a more meaningful expectation name, we can add it to the ExpectationsLibrary.

Here's an example of doing so:

CabezaDeTermo::JsonSpec::ExpectationsLibrary.define do
	expectations do
		define :to_be_the_answer_to_life_the_universe_and_everything do
		  expecting :to_be_equal_to, 42
		  message "Nop, this is not the answer to life, the universe and everything."
	 	end
	end
end

json_spec .to_be_the_answer_to_life_the_universe_and_everything
# is now equivalent to:
json_spec .to_be_equal_to(42)

Here's another interesting example of defining new expectations by delegation:

CabezaDeTermo::JsonSpec::ExpectationsLibrary.define do
	expectations do
		define :to_be_date do
		  expecting :to_match, /^\d\d\d\d-\d\d-\d\d$/
		  message "Not a valid date format."
	 	end
	end
end

This framework does not include a to_be_date expectation because it varies a lot from one API to another, but you can easyly add the one that suits your needs.

Now compare using in your specs:

json_spec .to_match(/^\d\d\d\d-\d\d-\d\d$/)
# vs
json_spec .to_be_date

• Negate an existing expectation.

If you want to expect that something is not what another expectation asserts, do:

CabezaDeTermo::JsonSpec::ExpectationsLibrary.define do
	expectations do
		define :not_date do
		  negating :to_be_date
		  message "Expected an invalid date, got a valid one."
	 	end
	end
end

• Chain several existing expectations:

CabezaDeTermo::JsonSpec::ExpectationsLibrary.define do
	expectations do
		define :to_be_defined do
			expecting_all_of :to_exist
			and_also :not_null

			message "Failed asserting that the field '<%= field %>' with value = '<%= format value %>' is defined."
		end
	end
end

• Expect one existing expectation among several ones:

CabezaDeTermo::JsonSpec::ExpectationsLibrary.define do
	expectations do
		define :to_be_accessor do
			expecting_any_of :to_be_getter
			or_also :to_be_setter

			message "The value is not an accessor string."
		end
	end
end

So far we only composed existing expectations. Now we are going to define new ones.

• Define a new expectation with a closure:

CabezaDeTermo::JsonSpec::ExpectationsLibrary.define do
	expectations do
		define :to_be_greater_than do
			with_block { |value_holder, expected_value| expected_value < value_holder.value }
			message "<%= value %> is not greater than <%= expectation.args[0] %>."
		end
	end
end

• Or finally, if your expectation is more complex, create a class for it:

require 'cabeza-de-termo/json-spec/expectations/expectation'

class IsSomeComplexStuff <  CabezaDeTermo::JsonSpec::Expectation
	def is_satisfied_by?(value_holder)
		# Check stuff and answer true or false.
		# You can get the inspected value with value_holder.value
	end
end

and then define the expectation in the ExpectationsLibrary

CabezaDeTermo::JsonSpec::ExpectationsLibrary.define do
	expectations do
		define :to_be_some_complex_stuff do
			with_class IsSomeComplexStuff
			message "<%= value %> is not some complex stuff."
		end
	end
end

If you want to run this example, see the fifth example in the examples/ folder.

Defining custom messages

We saw how we can define new Expectations with their own validation error message. But it would be nice to be able to change the validation error messages for the existing expectations as well.

We can do that at 2 different scopes, just like when we defined default expectations.

To override the validation messages globally, use

CabezaDeTermo::JsonSpec::ExpectationsLibrary.define do
	expectations do
		define :to_be_defined do
			message "If the question is to be or not to be, this value has chosen not to be. Alas, poor <%= field %>. My ValidationError rises at it."
		end
	end
end

If we want to override the message only for a single json-spec, do

json_spec.define do
	expectations do
		define :to_be_integer do
			message "An integer, a integer! My <%= field %> for an integer!"
		end
	end
end

You have to admit that these custom messages, although they are in a slang so boring and outdated that will get you to sleep in, like, 10 minutes, have a lot more of what we might call poetic flight than the default ones.

Just like when defining new Expectations, we have several ways to define custom messages. Lets walk through them:

• Using a ErbMessageFormatter object

CabezaDeTermo::JsonSpec::ExpectationsLibrary.define do
	expectations do
		define :to_be_integer do
			message "An integer, a integer! My <%= field %> for an integer!"
		end
	end
end

This is the simpliest way, and it should be enough most of the times. You can reference :value_holder, :value, :accessors_chain, :field and :expectation objects from within a erb block: <%= ... %>.

• Using a BlockMessageFormatter object

CabezaDeTermo::JsonSpec::ExpectationsLibrary.define do
	expectations do
		define :to_be_integer do
			message_block { |expectation, value_holder| "Not a valid integer." }
		end
	end
end

• Using you own MessageFormatter subclass

Define a new class with a :message_on(expectation, value_holder) method

class MyCustomMessageFormatter
	def message_on(expectation, value_holder)
		"Nop!"
	end
end

and then override the message

CabezaDeTermo::JsonSpec::ExpectationsLibrary.define do
	expectations do
		define :to_be_integer do
			message_formatter MyCustomMessageFormatter.new
		end
	end
end

If you want to run the custom messages example, see the sixth example in the examples/ folder.

Conditional expectations and expression modifiers

So far we only talked about Expectations. However, sometimes expectations are not enough. For instance, sometimes you may want to decide whether to keep running expectations or not, but without failing. An example of that are the :can_be_something statements. :can_be_absent states that a field may be missing and it's ok, but if it is present we want to keep running expectations on that expression. In the json-spec framework, each expression does not hold a collection of expectations, but an ExpectationsRunner instead. If you plug your own AbstractExpectationsRunner you can alter the execution flow of the expectations for that expression.

And how would you plug an ExpectationsRunner in a json_expression?

With the help of an ExpressionModifier, which you can add to the ExpectationsLibrary using its :define method, just like with the custom expectations.

Or perhaps some expectation needs to remove other expectations for an expression to make sense. In that case you can also use your own ExpressionModifier.

This sounds more complicated than it actually is. Check the CanBeNullModifier to see a real example and see that it's actually quite easy to do weird stuff on the expressions and expectations execution flow.

Putting it all together

Ok, so we can define default expectations for each expression, add new custom expectations, add new custom modifiers and replace the default expectation messages without the need of subclassing any existing class of the framework. That is really handy for simple validations. But what if we have done a nice and complete set of expectations and messages that we want to keep together to use it in several places?

In that case, it is a good idea to bundle it all together in one place.

That place can be a LibraryInitializer.

Create you own class that implements the LibraryInitializer protocol, and there create a new and fully configured ExpectationsLibrary.

Something like this:

class ComposerLibraryInitializer
	def self.new_library()
		self.new.new_library
	end

	def new_library()
		initialize_library CabezaDeTermo::JsonSpec::DefaultLibraryInitializer.new_library
	end

	def initialize_library(library)
		# add more custom stuff to the library here

		library
	end
end

and now plug your library in the spec that will use it:

CabezaDeTermo::JsonSpec::JsonSpec.new do
	use_expectations_library ComposerLibraryInitializer.new_library

	expect_an(:object) do
		...
	end
end

or, if you are going to use that library all around in your application, you can set it globally:

CabezaDeTermo::JsonSpec::ExpectationsLibrary.set_current ComposerLibraryInitializer.new_library

To see how all the pieces fitted together, check and run the seventh example in the examples/ folder.

Inspecting the expectations

With the default expectations and expression modifiers, it is quite sure that sooner or later you will want to see what expectations are actually set to each expression in a json_spec. If you need to debug the expectations, run

	puts json_spec.explain

and will get something like this

{
	"name":
		anything .not_blank()
	"type":
		anything .not_blank()
	"description":
		anything .not_blank()
	"keywords":
		[
			scalar .not_blank()
		]
			if present
			 .to_be_list(Array) .not_empty()
	"homepage":
		anything .to_be_url()
	"license":
		anything .to_be_valid_lincense(Apache-2.0, BSD-2-Clause, BSD-3-Clause, BSD-4-Clause, GPL-2.0, GPL-2.0+, GPL-3.0, GPL-3.0+, LGPL-2.1, LGPL-2.1+, LGPL-3.0, LGPL-3.0+, MIT)
	"authors":
		[
			{
				"name":
					anything .not_blank()
				"email":
					anything .to_be_email()
			} .to_be_object(Hash)
		] .to_be_list(Array) .not_empty()
	"require":
		{
			each field
				name .not_blank()
				value
					scalar .to_be_version()
		}
			if present
			 .to_be_object(Hash)
	"require-dev":
		{
			each field
				name .not_blank()
				value
					scalar .to_be_version()
		}
			if present
			 .to_be_object(Hash)
	"autoload":
		{
			"psr-0":
				{
					each field
						name .to_be_string(String)
						value
							any of
								scalar .to_be_folder()
							or
								[
									scalar .to_be_folder()
								] .to_be_list(Array) .not_empty()
				}
					if present
					 .to_be_object(Hash)
			"psr-4":
				{
					each field
						name .to_be_psr4_key()
						value
							any of
								scalar .to_be_folder()
							or
								[
									scalar .to_be_folder()
								] .to_be_list(Array) .not_empty()
				}
					if present
					 .to_be_object(Hash)
			"classmap":
				[
				]
					if present
					 .to_be_list(Array) .not_empty()
			"files":
				[
				]
					if present
					 .to_be_list(Array) .not_empty()
		} .to_be_object(Hash)
} .to_be_object(Hash)

Development environment

So, you are too lazy to setup the development environment for this project. Yeah, I feel you. I am too.

Anyways, you can use the Vagrant configuration for that. To do so:

• Install VirtualBox
• Install Vagrant
• git clone --recursive [e-mail]:cabeza-de-termo/ruby-json-spec.git
• cd ruby-json-spec/cachivache
• vagrant up

and that will install all the necessary things to run the tests and examples in this project.

Then to start playing around with the code, do

• vagrant ssh
• cd src

and that's it. You have a fully prepared, ready to use development environment.

Running the tests

• bundle install
• bundle exec rake

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/cabeza-de-termo/ruby-json-spec.

License

The gem is available as open source under the terms of the MIT License.