Test Reporting

Zephyr Sync is a tool, that allows your project to perform synchronization of automated test results to Zephyr - a JIRA addon for Test Management. The advanced configuration of the tool supports multiple report types to work with, as well as some restrictions to be applied during the sync.

A scala build tool

A scala build tool

A scala build tool

Extensions for producing nice looking test reports for appium tests in Xamarin Test Cloud

Cassandra Diagnostics functional test for Influx reporter

Run Serenity tests and generate Serenity reports using Cucumber 6

Tooling to generate confluence pages based on functional tests written in Cucumber

Railflow JUnit Annotations provides a custom JUnit 4 and 5 listeners and set of annotations/utility classes to generate enriched XML test reports which can be then uploaded into TestRail using Railflow NPM package or Railflow CI plugins

Provide fast and easy test reporting during team city build process

Aggregate Coverage Report

Parent module for com.github.bogdanlivadariu jenkins dependency modules

Coverage Report for Carbon Configuration by merging unit and osgi tests

A plugin used to generate reports of test outcomes and code coverage by tests executed by the CCL Testing Framework and through the maven-ccl-plugin

ThreadFix is a software vulnerability aggregation and management system that reduces the time it takes to fix software vulnerabilities. ThreadFix imports the results from dynamic, static and manual testing to provide a centralized view of software security defects across development teams and applications. The system allows companies to correlate testing results and streamline software remediation efforts by simplifying feeds to software issue trackers. By auto generating application firewall rules, this tool allows organizations to continue remediation work uninterrupted. ThreadFix empowers managers with vulnerability trending reports that show progress over time, giving them justification for their efforts. ThreadFix is developed and maintained by Denim Group, Ltd (http://www.denimgroup.com) For information about commercial support and other services, contact Denim Group about ThreadFix http://www.denimgroup.com/threadfix/ The ASTAM Correlator is a branch of the public Community Edition of the ThreadFix software. This branch is maintained by Secure Decisions (http://www.securedecisions.com), a Division of Applied Visions, Inc (http://www.avi.com).

Testerra test automation framework - report-tests module

High-level contextual steps in your tests for any reporting tool

Testerra test automation framework - report module

This project provides the best of maven easily consumable a maven parent project, called MavenParent. With this project you get: takari-lifecycle, which has a jdt based compiler that provides incremental recompilation, and proper APT support. Checkstyle and PMD checks that will work by default in eclipse with m2e-codequality. OWASP vulnerability checks of you dependencies. Easy application of BSD-3 clause license to all your code with mycila-license plugin. The ability to easily enforce semantic version on a jar project with japicmp. Prevent duplicates on your classpath with basepom's duplicate-finder. All of the above can be configured or disabled easily with properties, just check their docs, except takari. A whole slew of pretty maven reports including the above are generated in the maven site. The ability to easily deploy to oss nexus, and publish maven generated sites back to github's sites pages.

Handles AspectJ usage within Maven. Functionality provided is: weaving of aspects (or existing aspects from libraries) with the test and/or main classes, weaving of pre-existing jars and ajdoc reporting.

Testerum test automation platform

Sweet reports for the icing over your automated tests

Testerum test automation platform

Testerum test automation platform

Support for generating custom reports from JGiven's JSON Report files, per test method, per test class, or all-inclusive. Includes default HTML templates and PDF functionality, which can be overriden by subclassing.

Cobertura maven plugin that separates UT / IT / overall coverage reports, allows single tests execution and takes care of reports conversion to SonarQube generic test coverage format.

Step by step test reporting framework based on webdriver, cucumber and testng, based on project seleniumtestsframework

Step by step test reporting framework based on webdriver, cucumber and testng, based on project seleniumtestsframework

Step by step test reporting framework based on webdriver, cucumber and testng, based on project seleniumtestsframework

Step by step test reporting framework based on webdriver, cucumber and testng, based on project seleniumtestsframework

Step by step test reporting framework based on webdriver, cucumber and testng, based on project seleniumtestsframework

Handles AspectJ usage within Maven. Functionality provided is: weaving of aspects (or existing aspects from libraries) with the test and/or main classes, weaving of pre-existing jars and ajdoc reporting.

pact-jvm-provider-gradle ======================== Gradle plugin for verifying pacts against a provider. The Gradle plugin creates a task `pactVerify` to your build which will verify all configured pacts against your provider. __*Important Note: Any properties that need to be set when using the Gradle plugin need to be provided with `-P` and not `-D` as with the other Pact-JVM modules!*__ ## To Use It ### For Gradle versions prior to 2.1 #### 1.1. Add the pact-jvm-provider-gradle jar file to your build script class path: ```groovy buildscript { repositories { mavenCentral() } dependencies { classpath 'au.com.dius:pact-jvm-provider-gradle:4.0.0' } } ``` #### 1.2. Apply the pact plugin ```groovy apply plugin: 'au.com.dius.pact' ``` ### For Gradle versions 2.1+ ```groovy plugins { id "au.com.dius.pact" version "4.0.0" } ``` ### 2. Define the pacts between your consumers and providers ```groovy pact { serviceProviders { // You can define as many as you need, but each must have a unique name provider1 { // All the provider properties are optional, and have sensible defaults (shown below) protocol = 'http' host = 'localhost' port = 8080 path = '/' // Again, you can define as many consumers for each provider as you need, but each must have a unique name hasPactWith('consumer1') { // currently supports a file path using file() or a URL using url() pactSource = file('path/to/provider1-consumer1-pact.json') } // Or if you have many pact files in a directory hasPactsWith('manyConsumers') { // Will define a consumer for each pact file in the directory. // Consumer name is read from contents of pact file pactFileLocation = file('path/to/pacts') } } } } ``` ### 3. Execute `gradle pactVerify` ## Specifying the provider hostname at runtime If you need to calculate the provider hostname at runtime, you can give a Closure as the provider `host`. ```groovy pact { serviceProviders { provider1 { host = { lookupHostName() } hasPactWith('consumer1') { pactFile = file('path/to/provider1-consumer1-pact.json') } } } } ``` You can also give a Closure as the provider `port`. ## Specifying the pact file or URL at runtime If you need to calculate the pact file or URL at runtime, you can give a Closure as the provider `pactFile`. ```groovy pact { serviceProviders { provider1 { host = 'localhost' hasPactWith('consumer1') { pactFile = { lookupPactFile() } } } } } ``` ## Starting and shutting down your provider If you need to start-up or shutdown your provider, define Gradle tasks for each action and set `startProviderTask` and `terminateProviderTask` properties of each provider. You could use the jetty tasks here if you provider is built as a WAR file. ```groovy // This will be called before the provider task task('startTheApp') { doLast { // start up your provider here } } // This will be called after the provider task task('killTheApp') { doLast { // kill your provider here } } pact { serviceProviders { provider1 { startProviderTask = startTheApp terminateProviderTask = killTheApp hasPactWith('consumer1') { pactFile = file('path/to/provider1-consumer1-pact.json') } } } } ``` Following typical Gradle behaviour, you can set the provider task properties to the actual tasks, or to the task names as a string (for the case when they haven't been defined yet). ## Preventing the chaining of provider verify task to `pactVerify` Normally a gradle task named `pactVerify_${provider.name}` is created and added as a task dependency for `pactVerify`. You can disable this dependency on a provider by setting `isDependencyForPactVerify` to `false` (defaults to `true`). ```groovy pact { serviceProviders { provider1 { isDependencyForPactVerify = false hasPactWith('consumer1') { pactFile = file('path/to/provider1-consumer1-pact.json') } } } } ``` To run this task, you would then have to explicitly name it as in ```gradle pactVerify_provider1```, a normal ```gradle pactVerify``` would skip it. This can be useful when you want to define two providers, one with `startProviderTask`/`terminateProviderTask` and as second without, so you can manually start your provider (to debug it from your IDE, for example) but still want a `pactVerify` to run normally from your CI build. ## Enabling insecure SSL For providers that are running on SSL with self-signed certificates, you need to enable insecure SSL mode by setting `insecure = true` on the provider. ```groovy pact { serviceProviders { provider1 { insecure = true // allow SSL with a self-signed cert hasPactWith('consumer1') { pactFile = file('path/to/provider1-consumer1-pact.json') } } } } ``` ## Specifying a custom trust store For environments that are running their own certificate chains: ```groovy pact { serviceProviders { provider1 { trustStore = new File('relative/path/to/trustStore.jks') trustStorePassword = 'changeit' hasPactWith('consumer1') { pactFile = file('path/to/provider1-consumer1-pact.json') } } } } ``` `trustStore` is either relative to the current working (build) directory. `trustStorePassword` defaults to `changeit`. NOTE: The hostname will still be verified against the certificate. ## Modifying the HTTP Client Used The default HTTP client is used for all requests to providers (created with a call to `HttpClients.createDefault()`). This can be changed by specifying a closure assigned to createClient on the provider that returns a CloseableHttpClient. For example: ```groovy pact { serviceProviders { provider1 { createClient = { provider -> // This will enable the client to accept self-signed certificates HttpClients.custom().setSSLHostnameVerifier(new NoopHostnameVerifier()) .setSslcontext(new SSLContextBuilder().loadTrustMaterial(null, { x509Certificates, s -> true }) .build()) .build() } hasPactWith('consumer1') { pactFile = file('path/to/provider1-consumer1-pact.json') } } } } ``` ## Modifying the requests before they are sent Sometimes you may need to add things to the requests that can't be persisted in a pact file. Examples of these would be authentication tokens, which have a small life span. The Pact Gradle plugin provides a request filter that can be set to a closure on the provider that will be called before the request is made. This closure will receive the HttpRequest prior to it being executed. ```groovy pact { serviceProviders { provider1 { requestFilter = { req -> // Add an authorization header to each request req.addHeader('Authorization', 'OAUTH eyJhbGciOiJSUzI1NiIsImN0eSI6ImFw...') } hasPactWith('consumer1') { pactFile = file('path/to/provider1-consumer1-pact.json') } } } } ``` __*Important Note:*__ You should only use this feature for things that can not be persisted in the pact file. By modifying the request, you are potentially modifying the contract from the consumer tests! ## Turning off URL decoding of the paths in the pact file By default the paths loaded from the pact file will be decoded before the request is sent to the provider. To turn this behaviour off, set the property `pact.verifier.disableUrlPathDecoding` to `true`. __*Important Note:*__ If you turn off the url path decoding, you need to ensure that the paths in the pact files are correctly encoded. The verifier will not be able to make a request with an invalid encoded path. ## Project Properties The following project properties can be specified with `-Pproperty=value` on the command line: |Property|Description| |--------|-----------| |`pact.showStacktrace`|This turns on stacktrace printing for each request. It can help with diagnosing network errors| |`pact.showFullDiff`|This turns on displaying the full diff of the expected versus actual bodies| |`pact.filter.consumers`|Comma seperated list of consumer names to verify| |`pact.filter.description`|Only verify interactions whose description match the provided regular expression| |`pact.filter.providerState`|Only verify interactions whose provider state match the provided regular expression. An empty string matches interactions that have no state| |`pact.filter.pacturl`|This filter allows just the just the changed pact specified in a webhook to be run. It should be used in conjunction with `pact.filter.consumers` | |`pact.verifier.publishResults`|Publishing of verification results will be skipped unless this property is set to 'true'| |`pact.matching.wildcard`|Enables matching of map values ignoring the keys when this property is set to 'true'| |`pact.verifier.disableUrlPathDecoding`|Disables decoding of request paths| |`pact.pactbroker.httpclient.usePreemptiveAuthentication`|Enables preemptive authentication with the pact broker when set to `true`| |`pact.provider.tag`|Sets the provider tag to push before publishing verification results| ## Provider States For a description of what provider states are, see the pact documentations: http://docs.pact.io/documentation/provider_states.html ### Using a state change URL For each provider you can specify a state change URL to use to switch the state of the provider. This URL will receive the providerState description and all the parameters from the pact file before each interaction via a POST. As for normal requests, a request filter (`stateChangeRequestFilter`) can also be set to manipulate the request before it is sent. ```groovy pact { serviceProviders { provider1 { hasPactWith('consumer1') { pactFile = file('path/to/provider1-consumer1-pact.json') stateChangeUrl = url('http://localhost:8001/tasks/pactStateChange') stateChangeUsesBody = false // defaults to true stateChangeRequestFilter = { req -> // Add an authorization header to each request req.addHeader('Authorization', 'OAUTH eyJhbGciOiJSUzI1NiIsImN0eSI6ImFw...') } } // or hasPactsWith('consumers') { pactFileLocation = file('path/to/pacts') stateChangeUrl = url('http://localhost:8001/tasks/pactStateChange') stateChangeUsesBody = false // defaults to true } } } } ``` If the `stateChangeUsesBody` is not specified, or is set to true, then the provider state description and parameters will be sent as JSON in the body of the request : ```json { "state" : "a provider state description", "params": { "a": "1", "b": "2" } } ``` If it is set to false, they will be passed as query parameters. #### Teardown calls for state changes You can enable teardown state change calls by setting the property `stateChangeTeardown = true` on the provider. This will add an `action` parameter to the state change call. The setup call before the test will receive `action=setup`, and then a teardown call will be made afterwards to the state change URL with `action=teardown`. ### Using a Closure You can set a closure to be called before each verification with a defined provider state. The closure will be called with the state description and parameters from the pact file. ```groovy pact { serviceProviders { provider1 { hasPactWith('consumer1') { pactFile = file('path/to/provider1-consumer1-pact.json') // Load a fixture file based on the provider state and then setup some database // data. Does not require a state change request so returns false stateChange = { providerState -> // providerState is an instance of ProviderState def fixture = loadFixtuerForProviderState(providerState) setupDatabase(fixture) } } } } } ``` #### Teardown calls for state changes You can enable teardown state change calls by setting the property `stateChangeTeardown = true` on the provider. This will add an `action` parameter to the state change closure call. The setup call before the test will receive `setup`, as the second parameter, and then a teardown call will be made afterwards with `teardown` as the second parameter. ```groovy pact { serviceProviders { provider1 { hasPactWith('consumer1') { pactFile = file('path/to/provider1-consumer1-pact.json') // Load a fixture file based on the provider state and then setup some database // data. Does not require a state change request so returns false stateChange = { providerState, action -> if (action == 'setup') { def fixture = loadFixtuerForProviderState(providerState) setupDatabase(fixture) } else { cleanupDatabase() } false } } } } } ``` #### Returning values that can be injected You can have values from the provider state callbacks be injected into most places (paths, query parameters, headers, bodies, etc.). This works by using the V3 spec generators with provider state callbacks that return values. One example of where this would be useful is API calls that require an ID which would be auto-generated by the database on the provider side, so there is no way to know what the ID would be beforehand. There are methods on the consumer DSLs that can provider an expression that contains variables (like '/api/user/${id}' for the path). The provider state callback can then return a map for values, and the `id` attribute from the map will be expanded in the expression. For URL callbacks, the values need to be returned as JSON in the response body. ## Filtering the interactions that are verified You can filter the interactions that are run using three project properties: `pact.filter.consumers`, `pact.filter.description` and `pact.filter.providerState`. Adding `-Ppact.filter.consumers=consumer1,consumer2` to the command line will only run the pact files for those consumers (consumer1 and consumer2). Adding `-Ppact.filter.description=a request for payment.*` will only run those interactions whose descriptions start with 'a request for payment'. `-Ppact.filter.providerState=.*payment` will match any interaction that has a provider state that ends with payment, and `-Ppact.filter.providerState=` will match any interaction that does not have a provider state. ## Verifying pact files from a pact broker You can setup your build to validate against the pacts stored in a pact broker. The pact gradle plugin will query the pact broker for all consumers that have a pact with the provider based on its name. For example: ```groovy pact { serviceProviders { provider1 { // You can get the latest pacts from the broker hasPactsFromPactBroker('http://pact-broker:5000/') // And/or you can get the latest pact with a specific tag hasPactsFromPactBrokerWithTag('http://pact-broker:5000/',"tagname") } } } ``` This will verify all pacts found in the pact broker where the provider name is 'provider1'. If you need to set any values on the consumers from the pact broker, you can add a Closure to configure them. ```groovy pact { serviceProviders { provider1 { hasPactsFromPactBroker('http://pact-broker:5000/') { consumer -> stateChange = { providerState -> /* state change code here */ true } } } } } ``` **NOTE: Currently the pacts are fetched from the broker during the configuration phase of the build. This means that if the broker is not available, you will not be able to run any Gradle tasks.** This should be fixed in a forth coming release. In the mean time, to only load the pacts when running the validate task, you can do something like: ```groovy pact { serviceProviders { provider1 { // Only load the pacts from the broker if the start tasks from the command line include pactVerify if ('pactVerify' in gradle.startParameter.taskNames) { hasPactsFromPactBroker('http://pact-broker:5000/') { consumer -> stateChange = { providerState -> /* state change code here */ true } } } } } } ``` ### Using an authenticated Pact Broker You can add the authentication details for the Pact Broker like so: ```groovy pact { serviceProviders { provider1 { hasPactsFromPactBroker('http://pact-broker:5000/', authentication: ['Basic', pactBrokerUser, pactBrokerPassword]) } } } ``` `pactBrokerUser` and `pactBrokerPassword` can be defined in the gradle properties. Or with a bearer token: ```groovy pact { serviceProviders { provider1 { hasPactsFromPactBroker('http://pact-broker:5000/', authentication: ['Bearer', pactBrokerToken]) } } } ``` Preemptive Authentication can be enabled by setting the `pact.pactbroker.httpclient.usePreemptiveAuthentication` property to `true`. ### Allowing just the changed pact specified in a webhook to be verified [4.0.6+] When a consumer publishes a new version of a pact file, the Pact broker can fire off a webhook with the URL of the changed pact file. To allow only the changed pact file to be verified, you can override the URL by using the `pact.filter.consumers` and `pact.filter.pacturl` project properties. For example, running: ```console gradle pactVerify -Ppact.filter.consumers='Foo Web Client' -Ppact.filter.pacturl=https://test.pact.dius.com.au/pacts/provider/Activity%20Service/consumer/Foo%20Web%20Client/version/1.0.1 ``` will only run the verification for Foo Web Client with the given pact file URL. ## Verifying pact files from a S3 bucket **NOTE:** You will need to add the Amazon S3 SDK jar file to your project. Pact files stored in an S3 bucket can be verified by using an S3 URL to the pact file. I.e., ```groovy pact { serviceProviders { provider1 { hasPactWith('consumer1') { pactFile = 's3://bucketname/path/to/provider1-consumer1-pact.json' } } } } ``` **NOTE:** you can't use the `url` function with S3 URLs, as the URL and URI classes from the Java SDK don't support URLs with the s3 scheme. # Publishing pact files to a pact broker The pact gradle plugin provides a `pactPublish` task that can publish all pact files in a directory to a pact broker. To use it, you need to add a publish configuration to the pact configuration that defines the directory where the pact files are and the URL to the pact broker. For example: ```groovy pact { publish { pactDirectory = '/pact/dir' // defaults to $buildDir/pacts pactBrokerUrl = 'http://pactbroker:1234' } } ``` You can set any tags that the pacts should be published with by setting the `tags` property. A common use of this is setting the tag to the current source control branch. This supports using pact with feature branches. ```groovy pact { publish { pactDirectory = '/pact/dir' // defaults to $buildDir/pacts pactBrokerUrl = 'http://pactbroker:1234' tags = [project.pactBrokerTag] } } ``` _NOTE:_ The pact broker requires a version for all published pacts. The `pactPublish` task will use the version of the gradle project by default. You can override this with the `providerVersion` property. Make sure you have set one otherwise the broker will reject the pact files. ## Publishing to an authenticated pact broker To publish to a broker protected by basic auth, include the username/password in the `pactBrokerUrl`. For example: ```groovy pact { publish { pactBrokerUrl = 'https://username:password@mypactbroker.com' } } ``` You can add the username and password as properties. ```groovy pact { publish { pactBrokerUrl = 'https://mypactbroker.com' pactBrokerUsername = 'username' pactBrokerPassword = 'password' } } ``` or with a bearer token ```groovy pact { publish { pactBrokerUrl = 'https://mypactbroker.com' pactBrokerToken = 'token' } } ``` ## Excluding pacts from being published You can exclude some of the pact files from being published by providing a list of regular expressions that match against the base names of the pact files. For example: ```groovy pact { publish { pactBrokerUrl = 'https://mypactbroker.com' excludes = [ '.*\\-\\d+$' ] // exclude all pact files that end with a dash followed by a number in the name } } ``` # Verifying a message provider The Gradle plugin has been updated to allow invoking test methods that can return the message contents from a message producer. To use it, set the way to invoke the verification to `ANNOTATED_METHOD`. This will allow the pact verification task to scan for test methods that return the message contents. Add something like the following to your gradle build file: ```groovy pact { serviceProviders { messageProvider { verificationType = 'ANNOTATED_METHOD' packagesToScan = ['au.com.example.messageprovider.*'] // This is optional, but leaving it out will result in the entire // test classpath being scanned hasPactWith('messageConsumer') { pactFile = url('url/to/messagepact.json') } } } } ``` Now when the `pactVerify` task is run, will look for methods annotated with `@PactVerifyProvider` in the test classpath that have a matching description to what is in the pact file. ```groovy class ConfirmationKafkaMessageBuilderTest { @PactVerifyProvider('an order confirmation message') String verifyMessageForOrder() { Order order = new Order() order.setId(10000004) order.setExchange('ASX') order.setSecurityCode('CBA') order.setPrice(BigDecimal.TEN) order.setUnits(15) order.setGst(new BigDecimal('15.0')) order.setFees(BigDecimal.TEN) def message = new ConfirmationKafkaMessageBuilder() .withOrder(order) .build() JsonOutput.toJson(message) } } ``` It will then validate that the returned contents matches the contents for the message in the pact file. ## Publishing to the Gradle Community Portal To publish the plugin to the community portal: $ ./gradlew :pact-jvm-provider-gradle_2.11:publishPlugins # Verification Reports The default behaviour is to display the verification being done to the console, and pass or fail the build via the normal Gradle mechanism. Additional reports can be generated from the verification. ## Enabling additional reports The verification reports can be controlled by adding a reports section to the pact configuration in the gradle build file. For example: ```groovy pact { reports { defaultReports() // adds the standard console output markdown // report in markdown format json // report in json format } } ``` Any report files will be written to "build/reports/pact". ## Additional Reports The following report types are available in addition to console output (which is enabled by default): `markdown`, `json`. # Publishing verification results to a Pact Broker For pacts that are loaded from a Pact Broker, the results of running the verification can be published back to the broker against the URL for the pact. You will be able to see the result on the Pact Broker home screen. To turn on the verification publishing, set the project property `pact.verifier.publishResults` to `true`. By default, the Gradle project version will be used as the provider version. You can override this by setting the `providerVersion` property. ```groovy pact { serviceProviders { provider1 { providerVersion = { branchName() + '-' + abbreviatedId() } hasPactsFromPactBroker('http://pact-broker:5000/', authentication: ['Basic', pactBrokerUser, pactBrokerPassword]) } } } ``` ## Tagging the provider before verification results are published [4.0.1+] You can have a tag pushed against the provider version before the verification results are published. There are two ways to do this with the Gradle plugin. You can provide a closure in a similar way to the provider version, i.e. ```groovy pact { serviceProviders { provider1 { providerVersion = { branchName() + '-' + abbreviatedId() } providerTag = { branchName() } hasPactsFromPactBroker('http://pact-broker:5000/', authentication: ['Basic', pactBrokerUser, pactBrokerPassword]) } } } ``` or you can set the `pact.provider.tag` JVM system property. For example: ```console $ ./gradlew -d pactverify -Ppact.verifier.publishResults=true -Dpact.provider.tag=Test2 ```

pact-jvm-provider-gradle ======================== Gradle plugin for verifying pacts against a provider. The Gradle plugin creates a task `pactVerify` to your build which will verify all configured pacts against your provider. ## To Use It ### For Gradle versions prior to 2.1 #### 1.1. Add the pact-jvm-provider-gradle jar file to your build script class path: ```groovy buildscript { repositories { mavenCentral() } dependencies { classpath 'au.com.dius:pact-jvm-provider-gradle_2.10:3.2.11' } } ``` #### 1.2. Apply the pact plugin ```groovy apply plugin: 'au.com.dius.pact' ``` ### For Gradle versions 2.1+ ```groovy plugins { id "au.com.dius.pact" version "3.2.11" } ``` ### 2. Define the pacts between your consumers and providers ```groovy pact { serviceProviders { // You can define as many as you need, but each must have a unique name provider1 { // All the provider properties are optional, and have sensible defaults (shown below) protocol = 'http' host = 'localhost' port = 8080 path = '/' // Again, you can define as many consumers for each provider as you need, but each must have a unique name hasPactWith('consumer1') { // currently supports a file path using file() or a URL using url() pactSource = file('path/to/provider1-consumer1-pact.json') } // Or if you have many pact files in a directory hasPactsWith('manyConsumers') { // Will define a consumer for each pact file in the directory. // Consumer name is read from contents of pact file pactFileLocation = file('path/to/pacts') } } } } ``` ### 3. Execute `gradle pactVerify` ## Specifying the provider hostname at runtime If you need to calculate the provider hostname at runtime, you can give a Closure as the provider `host`. ```groovy pact { serviceProviders { provider1 { host = { lookupHostName() } hasPactWith('consumer1') { pactFile = file('path/to/provider1-consumer1-pact.json') } } } } ``` _Since version 3.3.2+/2.4.17+_ you can also give a Closure as the provider `port`. ## Specifying the pact file or URL at runtime [versions 3.2.7/2.4.9+] If you need to calculate the pact file or URL at runtime, you can give a Closure as the provider `pactFile`. ```groovy pact { serviceProviders { provider1 { host = 'localhost' hasPactWith('consumer1') { pactFile = { lookupPactFile() } } } } } ``` ## Starting and shutting down your provider If you need to start-up or shutdown your provider, define Gradle tasks for each action and set `startProviderTask` and `terminateProviderTask` properties of each provider. You could use the jetty tasks here if you provider is built as a WAR file. ```groovy // This will be called before the provider task task('startTheApp') { doLast { // start up your provider here } } // This will be called after the provider task task('killTheApp') { doLast { // kill your provider here } } pact { serviceProviders { provider1 { startProviderTask = startTheApp terminateProviderTask = killTheApp hasPactWith('consumer1') { pactFile = file('path/to/provider1-consumer1-pact.json') } } } } ``` Following typical Gradle behaviour, you can set the provider task properties to the actual tasks, or to the task names as a string (for the case when they haven't been defined yet). ## Preventing the chaining of provider verify task to `pactVerify` [version 3.4.1+] Normally a gradle task named `pactVerify_${provider.name}` is created and added as a task dependency for `pactVerify`. You can disable this dependency on a provider by setting `isDependencyForPactVerify` to `false` (defaults to `true`). ```groovy pact { serviceProviders { provider1 { isDependencyForPactVerify = false hasPactWith('consumer1') { pactFile = file('path/to/provider1-consumer1-pact.json') } } } } ``` To run this task, you would then have to explicitly name it as in ```gradle pactVerify_provider1```, a normal ```gradle pactVerify``` would skip it. This can be useful when you want to define two providers, one with `startProviderTask`/`terminateProviderTask` and as second without, so you can manually start your provider (to debug it from your IDE, for example) but still want a `pactVerify` to run normally from your CI build. ## Enabling insecure SSL [version 2.2.8+] For providers that are running on SSL with self-signed certificates, you need to enable insecure SSL mode by setting `insecure = true` on the provider. ```groovy pact { serviceProviders { provider1 { insecure = true // allow SSL with a self-signed cert hasPactWith('consumer1') { pactFile = file('path/to/provider1-consumer1-pact.json') } } } } ``` ## Specifying a custom trust store [version 2.2.8+] For environments that are running their own certificate chains: ```groovy pact { serviceProviders { provider1 { trustStore = new File('relative/path/to/trustStore.jks') trustStorePassword = 'changeit' hasPactWith('consumer1') { pactFile = file('path/to/provider1-consumer1-pact.json') } } } } ``` `trustStore` is either relative to the current working (build) directory. `trustStorePassword` defaults to `changeit`. NOTE: The hostname will still be verified against the certificate. ## Modifying the HTTP Client Used [version 2.2.4+] The default HTTP client is used for all requests to providers (created with a call to `HttpClients.createDefault()`). This can be changed by specifying a closure assigned to createClient on the provider that returns a CloseableHttpClient. For example: ```groovy pact { serviceProviders { provider1 { createClient = { provider -> // This will enable the client to accept self-signed certificates HttpClients.custom().setSSLHostnameVerifier(new NoopHostnameVerifier()) .setSslcontext(new SSLContextBuilder().loadTrustMaterial(null, { x509Certificates, s -> true }) .build()) .build() } hasPactWith('consumer1') { pactFile = file('path/to/provider1-consumer1-pact.json') } } } } ``` ## Modifying the requests before they are sent **NOTE on breaking change: Version 2.1.8+ uses Apache HttpClient instead of HttpBuilder so the closure will receive a HttpRequest object instead of a request Map.** Sometimes you may need to add things to the requests that can't be persisted in a pact file. Examples of these would be authentication tokens, which have a small life span. The Pact Gradle plugin provides a request filter that can be set to a closure on the provider that will be called before the request is made. This closure will receive the HttpRequest prior to it being executed. ```groovy pact { serviceProviders { provider1 { requestFilter = { req -> // Add an authorization header to each request req.addHeader('Authorization', 'OAUTH eyJhbGciOiJSUzI1NiIsImN0eSI6ImFw...') } hasPactWith('consumer1') { pactFile = file('path/to/provider1-consumer1-pact.json') } } } } ``` __*Important Note:*__ You should only use this feature for things that can not be persisted in the pact file. By modifying the request, you are potentially modifying the contract from the consumer tests! ## Turning off URL decoding of the paths in the pact file [version 3.3.3+] By default the paths loaded from the pact file will be decoded before the request is sent to the provider. To turn this behaviour off, set the system property `pact.verifier.disableUrlPathDecoding` to `true`. __*Important Note:*__ If you turn off the url path decoding, you need to ensure that the paths in the pact files are correctly encoded. The verifier will not be able to make a request with an invalid encoded path. ## Project Properties The following project properties can be specified with `-Pproperty=value` on the command line: |Property|Description| |--------|-----------| |pact.showStacktrace|This turns on stacktrace printing for each request. It can help with diagnosing network errors| |pact.showFullDiff|This turns on displaying the full diff of the expected versus actual bodies [version 3.3.6+]| |pact.filter.consumers|Comma seperated list of consumer names to verify| |pact.filter.description|Only verify interactions whose description match the provided regular expression| |pact.filter.providerState|Only verify interactions whose provider state match the provided regular expression. An empty string matches interactions that have no state| |pact.verifier.publishResults|Publishing of verification results will be skipped unless this property is set to 'true'| |pact.matching.wildcard|Enables matching of map values ignoring the keys when this property is set to 'true'| |pact.verifier.disableUrlPathDecoding|Disables decoding of request paths| |pact.pactbroker.httpclient.usePreemptiveAuthentication|Enables preemptive authentication with the pact broker when set to `true`| ## Provider States For a description of what provider states are, see the pact documentations: http://docs.pact.io/documentation/provider_states.html ### Using a state change URL For each provider you can specify a state change URL to use to switch the state of the provider. This URL will receive the providerState description and all the parameters from the pact file before each interaction via a POST. As for normal requests, a request filter (`stateChangeRequestFilter`) can also be set to manipulate the request before it is sent. ```groovy pact { serviceProviders { provider1 { hasPactWith('consumer1') { pactFile = file('path/to/provider1-consumer1-pact.json') stateChangeUrl = url('http://localhost:8001/tasks/pactStateChange') stateChangeUsesBody = false // defaults to true stateChangeRequestFilter = { req -> // Add an authorization header to each request req.addHeader('Authorization', 'OAUTH eyJhbGciOiJSUzI1NiIsImN0eSI6ImFw...') } } // or hasPactsWith('consumers') { pactFileLocation = file('path/to/pacts') stateChangeUrl = url('http://localhost:8001/tasks/pactStateChange') stateChangeUsesBody = false // defaults to true } } } } ``` If the `stateChangeUsesBody` is not specified, or is set to true, then the provider state description and parameters will be sent as JSON in the body of the request : ```json { "state" : "a provider state description", "params": { "a": "1", "b": "2" } } ``` If it is set to false, they will be passed as query parameters. #### Teardown calls for state changes [version 3.2.5/2.4.7+] You can enable teardown state change calls by setting the property `stateChangeTeardown = true` on the provider. This will add an `action` parameter to the state change call. The setup call before the test will receive `action=setup`, and then a teardown call will be made afterwards to the state change URL with `action=teardown`. ### Using a Closure [version 2.2.2+] You can set a closure to be called before each verification with a defined provider state. The closure will be called with the state description and parameters from the pact file. ```groovy pact { serviceProviders { provider1 { hasPactWith('consumer1') { pactFile = file('path/to/provider1-consumer1-pact.json') // Load a fixture file based on the provider state and then setup some database // data. Does not require a state change request so returns false stateChange = { providerState -> // providerState is an instance of ProviderState def fixture = loadFixtuerForProviderState(providerState) setupDatabase(fixture) } } } } } ``` #### Teardown calls for state changes [version 3.2.5/2.4.7+] You can enable teardown state change calls by setting the property `stateChangeTeardown = true` on the provider. This will add an `action` parameter to the state change closure call. The setup call before the test will receive `setup`, as the second parameter, and then a teardown call will be made afterwards with `teardown` as the second parameter. ```groovy pact { serviceProviders { provider1 { hasPactWith('consumer1') { pactFile = file('path/to/provider1-consumer1-pact.json') // Load a fixture file based on the provider state and then setup some database // data. Does not require a state change request so returns false stateChange = { providerState, action -> if (action == 'setup') { def fixture = loadFixtuerForProviderState(providerState) setupDatabase(fixture) } else { cleanupDatabase() } false } } } } } ``` #### Returning values that can be injected (3.6.11+) You can have values from the provider state callbacks be injected into most places (paths, query parameters, headers, bodies, etc.). This works by using the V3 spec generators with provider state callbacks that return values. One example of where this would be useful is API calls that require an ID which would be auto-generated by the database on the provider side, so there is no way to know what the ID would be beforehand. There are methods on the consumer DSLs that can provider an expression that contains variables (like '/api/user/${id}' for the path). The provider state callback can then return a map for values, and the `id` attribute from the map will be expanded in the expression. For URL callbacks, the values need to be returned as JSON in the response body. ## Filtering the interactions that are verified You can filter the interactions that are run using three project properties: `pact.filter.consumers`, `pact.filter.description` and `pact.filter.providerState`. Adding `-Ppact.filter.consumers=consumer1,consumer2` to the command line will only run the pact files for those consumers (consumer1 and consumer2). Adding `-Ppact.filter.description=a request for payment.*` will only run those interactions whose descriptions start with 'a request for payment'. `-Ppact.filter.providerState=.*payment` will match any interaction that has a provider state that ends with payment, and `-Ppact.filter.providerState=` will match any interaction that does not have a provider state. ## Verifying pact files from a pact broker [version 3.1.1+/2.3.1+] You can setup your build to validate against the pacts stored in a pact broker. The pact gradle plugin will query the pact broker for all consumers that have a pact with the provider based on its name. For example: ```groovy pact { serviceProviders { provider1 { // You can get the latest pacts from the broker hasPactsFromPactBroker('http://pact-broker:5000/') // And/or you can get the latest pact with a specific tag hasPactsFromPactBrokerWithTag('http://pact-broker:5000/',"tagname") } } } ``` This will verify all pacts found in the pact broker where the provider name is 'provider1'. If you need to set any values on the consumers from the pact broker, you can add a Closure to configure them. ```groovy pact { serviceProviders { provider1 { hasPactsFromPactBroker('http://pact-broker:5000/') { consumer -> stateChange = { providerState -> /* state change code here */ true } } } } } ``` **NOTE: Currently the pacts are fetched from the broker during the configuration phase of the build. This means that if the broker is not available, you will not be able to run any Gradle tasks.** This should be fixed in a forth coming release. In the mean time, to only load the pacts when running the validate task, you can do something like: ```groovy pact { serviceProviders { provider1 { // Only load the pacts from the broker if the start tasks from the command line include pactVerify if ('pactVerify' in gradle.startParameter.taskNames) { hasPactsFromPactBroker('http://pact-broker:5000/') { consumer -> stateChange = { providerState -> /* state change code here */ true } } } } } } ``` ### Using an authenticated Pact Broker You can add the authentication details for the Pact Broker like so: ```groovy pact { serviceProviders { provider1 { hasPactsFromPactBroker('http://pact-broker:5000/', authentication: ['Basic', pactBrokerUser, pactBrokerPassword]) } } } ``` `pactBrokerUser` and `pactBrokerPassword` can be defined in the gradle properties. Or with a bearer token: ```groovy pact { serviceProviders { provider1 { hasPactsFromPactBroker('http://pact-broker:5000/', authentication: ['Bearer', pactBrokerToken]) } } } ``` Preemptive Authentication can be enabled by setting the `pact.pactbroker.httpclient.usePreemptiveAuthentication` Java system property to `true`. ## Verifying pact files from a S3 bucket [version 3.3.2+/2.4.17+] Pact files stored in an S3 bucket can be verified by using an S3 URL to the pact file. I.e., ```groovy pact { serviceProviders { provider1 { hasPactWith('consumer1') { pactFile = 's3://bucketname/path/to/provider1-consumer1-pact.json' } } } } ``` **NOTE:** you can't use the `url` function with S3 URLs, as the URL and URI classes from the Java SDK don't support URLs with the s3 scheme. # Publishing pact files to a pact broker [version 2.2.7+] The pact gradle plugin provides a `pactPublish` task that can publish all pact files in a directory to a pact broker. To use it, you need to add a publish configuration to the pact configuration that defines the directory where the pact files are and the URL to the pact broker. For example: ```groovy pact { publish { pactDirectory = '/pact/dir' // defaults to $buildDir/pacts pactBrokerUrl = 'http://pactbroker:1234' } } ``` You can set any tags that the pacts should be published with by setting the `tags` property. A common use of this is setting the tag to the current source control branch. This supports using pact with feature branches. ```groovy pact { publish { pactDirectory = '/pact/dir' // defaults to $buildDir/pacts pactBrokerUrl = 'http://pactbroker:1234' tags = [project.pactBrokerTag] } } ``` _NOTE:_ The pact broker requires a version for all published pacts. The `pactPublish` task will use the version of the gradle project by default. Make sure you have set one otherwise the broker will reject the pact files. _Version 3.2.2/2.4.3+_ you can override the version in the publish block. ## Publishing to an authenticated pact broker To publish to a broker protected by basic auth, include the username/password in the `pactBrokerUrl`. For example: ```groovy pact { publish { pactBrokerUrl = 'https://username:password@mypactbroker.com' } } ``` ### [version 3.3.9+] You can add the username and password as properties since version 3.3.9+ ```groovy pact { publish { pactBrokerUrl = 'https://mypactbroker.com' pactBrokerUsername = 'username' pactBrokerPassword = 'password' } } ``` or with a bearer token ```groovy pact { publish { pactBrokerUrl = 'https://mypactbroker.com' pactBrokerToken = 'token' } } ``` ## Excluding pacts from being published [version 3.5.19+] You can exclude some of the pact files from being published by providing a list of regular expressions that match against the base names of the pact files. For example: ```groovy pact { publish { pactBrokerUrl = 'https://mypactbroker.com' excludes = [ '.*\\-\\d+$' ] // exclude all pact files that end with a dash followed by a number in the name } } ``` # Verifying a message provider [version 2.2.12+] The Gradle plugin has been updated to allow invoking test methods that can return the message contents from a message producer. To use it, set the way to invoke the verification to `ANNOTATED_METHOD`. This will allow the pact verification task to scan for test methods that return the message contents. Add something like the following to your gradle build file: ```groovy pact { serviceProviders { messageProvider { verificationType = 'ANNOTATED_METHOD' packagesToScan = ['au.com.example.messageprovider.*'] // This is optional, but leaving it out will result in the entire // test classpath being scanned hasPactWith('messageConsumer') { pactFile = url('url/to/messagepact.json') } } } } ``` Now when the `pactVerify` task is run, will look for methods annotated with `@PactVerifyProvider` in the test classpath that have a matching description to what is in the pact file. ```groovy class ConfirmationKafkaMessageBuilderTest { @PactVerifyProvider('an order confirmation message') String verifyMessageForOrder() { Order order = new Order() order.setId(10000004) order.setExchange('ASX') order.setSecurityCode('CBA') order.setPrice(BigDecimal.TEN) order.setUnits(15) order.setGst(new BigDecimal('15.0')) order.setFees(BigDecimal.TEN) def message = new ConfirmationKafkaMessageBuilder() .withOrder(order) .build() JsonOutput.toJson(message) } } ``` It will then validate that the returned contents matches the contents for the message in the pact file. ## Publishing to the Gradle Community Portal To publish the plugin to the community portal: $ ./gradlew :pact-jvm-provider-gradle_2.11:publishPlugins # Verification Reports [versions 3.2.7/2.4.9+] The default behaviour is to display the verification being done to the console, and pass or fail the build via the normal Gradle mechanism. From versions 3.2.7/2.4.9+, additional reports can be generated from the verification. ## Enabling additional reports The verification reports can be controlled by adding a reports section to the pact configuration in the gradle build file. For example: ```groovy pact { reports { defaultReports() // adds the standard console output markdown // report in markdown format json // report in json format } } ``` Any report files will be written to "build/reports/pact". ## Additional Reports The following report types are available in addition to console output (which is enabled by default): `markdown`, `json`. # Publishing verification results to a Pact Broker [version 3.5.4+] For pacts that are loaded from a Pact Broker, the results of running the verification can be published back to the broker against the URL for the pact. You will be able to see the result on the Pact Broker home screen. To turn on the verification publishing, set the project property `pact.verifier.publishResults` to `true` [version 3.5.18+].

Railflow TestNG Annotations provides a custom TestNG reporter and set of annotations/utility classes to generate enriched XML test reports which can be then uploaded into TestRail using Railflow NPM package or Railflow CI plugins

Step by step test reporting framework based on webdriver and testng

Small module allowing to get a local reporting of Microprofile data - intended for tests and demos.

# Pact junit runner ## Overview Library provides ability to play contract tests against a provider service in JUnit fashionable way. Supports: - Out-of-the-box convenient ways to load pacts - Easy way to change assertion strategy - **org.junit.BeforeClass**, **org.junit.AfterClass** and **org.junit.ClassRule** JUnit annotations, that will be run once - before/after whole contract test suite. - **org.junit.Before**, **org.junit.After** and **org.junit.Rule** JUnit annotations, that will be run before/after each test of an interaction. - **au.com.dius.pact.provider.junit.State** custom annotation - before each interaction that requires a state change, all methods annotated by `@State` with appropriate the state listed will be invoked. ## Example of HTTP test ```java @RunWith(PactRunner.class) // Say JUnit to run tests with custom Runner @Provider("myAwesomeService") // Set up name of tested provider @PactFolder("pacts") // Point where to find pacts (See also section Pacts source in documentation) public class ContractTest { // NOTE: this is just an example of embedded service that listens to requests, you should start here real service @ClassRule //Rule will be applied once: before/after whole contract test suite public static final ClientDriverRule embeddedService = new ClientDriverRule(8332); @BeforeClass //Method will be run once: before whole contract test suite public static void setUpService() { //Run DB, create schema //Run service //... } @Before //Method will be run before each test of interaction public void before() { // Rest data // Mock dependent service responses // ... embeddedService.addExpectation( onRequestTo("/data"), giveEmptyResponse() ); } @State("default", "no-data") // Method will be run before testing interactions that require "default" or "no-data" state public void toDefaultState() { // Prepare service before interaction that require "default" state // ... System.out.println("Now service in default state"); } @TestTarget // Annotation denotes Target that will be used for tests public final Target target = new HttpTarget(8332); // Out-of-the-box implementation of Target (for more information take a look at Test Target section) } ``` ## Example of AMQP Message test ```java @RunWith(PactRunner.class) // Say JUnit to run tests with custom Runner @Provider("myAwesomeService") // Set up name of tested provider @PactBroker(host="pactbroker", port = "80") public class ConfirmationKafkaContractTest { @TestTarget // Annotation denotes Target that will be used for tests public final Target target = new AmqpTarget(); // Out-of-the-box implementation of Target (for more information take a look at Test Target section) @BeforeClass //Method will be run once: before whole contract test suite public static void setUpService() { //Run DB, create schema //Run service //... } @Before //Method will be run before each test of interaction public void before() { // Message data preparation // ... } @PactVerifyProvider('an order confirmation message') String verifyMessageForOrder() { Order order = new Order() order.setId(10000004) order.setPrice(BigDecimal.TEN) order.setUnits(15) def message = new ConfirmationKafkaMessageBuilder() .withOrder(order) .build() JsonOutput.toJson(message) } } ``` ## Pact source The Pact runner will automatically collect pacts based on annotations on the test class. For this purpose there are 3 out-of-the-box options (files from a directory, files from a set of URLs or a pact broker) or you can easily add your own Pact source. **Note:** You can only define one source of pacts per test class. ### Download pacts from a pact-broker To use pacts from a Pact Broker, annotate the test class with `@PactBroker(host="host.of.pact.broker.com", port = "80")`. From _version 3.2.2/2.4.3+_ you can also specify the protocol, which defaults to "http". The pact broker will be queried for all pacts with the same name as the provider annotation. For example, test all pacts for the "Activity Service" in the pact broker: ```java @RunWith(PactRunner.class) @Provider("Activity Service") @PactBroker(host = "localhost", port = "80") public class PactJUnitTest { @TestTarget public final Target target = new HttpTarget(5050); } ``` #### _Version 3.2.3/2.4.4+_ - Using Java System properties The pact broker loader was updated to allow system properties to be used for the hostname, port or protocol. The port was changed to a string to allow expressions to be set. To use a system property or environment variable, you can place the property name in `${}` expression de-markers: ```java @PactBroker(host="${pactbroker.hostname}", port = "80") ``` You can provide a default value by separating the property name with a colon (`:`): ```java @PactBroker(host="${pactbroker.hostname:localhost}", port = "80") ``` #### _Version ???/???+_ - More Java System properties The default values of the `@PactBroker` annotation now enable variable interpolation. The following keys may be managed through the environment * `pactbroker.host` * `pactbroker.port` * `pactbroker.protocol` * `pactbroker.tags` (comma separated) * `pactbroker.auth.scheme` * `pactbroker.auth.username` * `pactbroker.auth.password` #### _Version 3.2.4/2.4.6+_ - Using tags with the pact broker The pact broker allows different versions to be tagged. To load all the pacts: ```java @PactBroker(host="pactbroker", port = "80", tags = {"latest", "dev", "prod"}) ``` The default value for tags is `latest` which is not actually a tag but instead corresponds to the latest version ignoring the tags. If there are multiple consumers matching the name specified in the provider annotation then the latest pact for each of the consumers is loaded. For any other value the latest pact tagged with the specified tag is loaded. Specifying multiple tags is an OR operation. For example if you specify `tags = {"dev", "prod"}` then both the latest pact file tagged with `dev` and the latest pact file taggged with `prod` is loaded. #### _Version 3.3.4/2.4.19+_ - Using basic auth with the with the pact broker You can use basic authentication with the `@PactBroker` annotation by setting the `authentication` value to a `@PactBrokerAuth` annotation. For example: ```java @PactBroker(host = "${pactbroker.url:localhost}", port = "1234", tags = {"latest", "prod", "dev"}, authentication = @PactBrokerAuth(username = "test", password = "test")) ``` The `username` and `password` values also take Java system property expressions. ### Pact Url To use pacts from urls annotate the test class with ```java @PactUrl(urls = {"http://build.server/zoo_app-animal_service.json"} ) ``` ### Pact folder To use pacts from a resource folder of the project annotate test class with ```java @PactFolder("subfolder/in/resource/directory") ``` ### Custom pacts source It's possible to use a custom Pact source. For this, implement interface `au.com.dius.pact.provider.junit.loader.PactLoader` and annotate the test class with `@PactSource(MyOwnPactLoader.class)`. **Note:** class `MyOwnPactLoader` must have a default empty constructor or a constructor with one argument of class `Class` which at runtime will be the test class so you can get custom annotations of test class. ## Test target The field in test class of type `au.com.dius.pact.provider.junit.target.Target` annotated with `au.com.dius.pact.provider.junit.target.TestTarget` will be used for actual Interaction execution and asserting of contract. **Note:** there must be exactly 1 such field, otherwise an `InitializationException` will be thrown. ### HttpTarget `au.com.dius.pact.provider.junit.target.HttpTarget` - out-of-the-box implementation of `au.com.dius.pact.provider.junit.target.Target` that will play pacts as http request and assert response from service by matching rules from pact. _Version 3.2.2/2.4.3+_ you can also specify the protocol, defaults to "http". ### AmqpTarget `au.com.dius.pact.provider.junit.target.AmqpTarget` - out-of-the-box implementation of `au.com.dius.pact.provider.junit.target.Target` that will play pacts as an AMQP message and assert response from service by matching rules from pact. #### Modifying the requests before they are sent [Version 3.2.3/2.4.5+] Sometimes you may need to add things to the requests that can't be persisted in a pact file. Examples of these would be authentication tokens, which have a small life span. The HttpTarget supports request filters by annotating methods on the test class with `@TargetRequestFilter`. These methods must be public void methods that take a single HttpRequest parameter. For example: ```java @TargetRequestFilter public void exampleRequestFilter(HttpRequest request) { request.addHeader("Authorization", "OAUTH hdsagasjhgdjashgdah..."); } ``` __*Important Note:*__ You should only use this feature for things that can not be persisted in the pact file. By modifying the request, you are potentially modifying the contract from the consumer tests! #### Turning off URL decoding of the paths in the pact file [version 3.3.3+] By default the paths loaded from the pact file will be decoded before the request is sent to the provider. To turn this behaviour off, set the system property `pact.verifier.disableUrlPathDecoding` to `true`. __*Important Note:*__ If you turn off the url path decoding, you need to ensure that the paths in the pact files are correctly encoded. The verifier will not be able to make a request with an invalid encoded path. ### Custom Test Target It's possible to use custom `Target`, for that interface `Target` should be implemented and this class can be used instead of `HttpTarget`. # Verification Reports [versions 3.2.7/2.4.9+] The default test behaviour is to display the verification being done to the console, and pass or fail the test via the normal JUnit mechanism. From versions 3.2.7/2.4.9+, additional reports can be generated from the tests. ## Enabling additional reports via annotations on the test classes A `@VerificationReports` annotation can be added to any pact test class which will control the verification output. The annotation takes a list report types and an optional report directory (defaults to "target/pact/reports"). The currently supported report types are `console`, `markdown` and `json`. For example: ```java @VerificationReports({"console", "markdown"}) public class MyPactTest { ``` will enable the markdown report in addition to the normal console output. And, ```java @VerificationReports(value = {"markdown"}, reportDir = "/myreports") public class MyPactTest { ``` will disable the normal console output and write the markdown reports to "/myreports". ## Enabling additional reports via Java system properties or environment variables The additional reports can also be enabled with Java System properties or environment variables. The following two properties have been introduced: `pact.verification.reports` and `pact.verification.reportDir`. `pact.verification.reports` is the comma separated list of report types to enable (e.g. `console,json,markdown`). `pact.verification.reportDir` is the directory to write reports to (defaults to "target/pact/reports"). ## Additional Reports The following report types are available in addition to console output (`console`, which is enabled by default): `markdown`, `json`. You can also provide a fully qualified classname as report so custom reports are also supported. This class must implement `au.com.dius.pact.provider.reporters.VerifierReporter` interface in order to be correct custom implementation of a report.

Maven plugin to verify a provider ================================= Maven plugin for verifying pacts against a provider. The Maven plugin provides a `verify` goal which will verify all configured pacts against your provider. ## To Use It ### 1. Add the pact-jvm-provider-maven plugin to your `build` section of your pom file. ```xml <build> [...] <plugins> [...] <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.11</version> </plugin> [...] </plugins> [...] </build> ``` ### 2. Define the pacts between your consumers and providers You define all the providers and consumers within the configuration element of the maven plugin. ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.11</version> <configuration> <serviceProviders> <!-- You can define as many as you need, but each must have a unique name --> <serviceProvider> <name>provider1</name> <!-- All the provider properties are optional, and have sensible defaults (shown below) --> <protocol>http</protocol> <host>localhost</host> <port>8080</port> <path>/</path> <consumers> <!-- Again, you can define as many consumers for each provider as you need, but each must have a unique name --> <consumer> <name>consumer1</name> <!-- currently supports a file path using pactFile or a URL using pactUrl --> <pactFile>path/to/provider1-consumer1-pact.json</pactFile> </consumer> </consumers> </serviceProvider> </serviceProviders> </configuration> </plugin> ``` ### 3. Execute `mvn pact:verify` You will have to have your provider running for this to pass. ## Verifying all pact files in a directory for a provider You can specify a directory that contains pact files, and the Pact plugin will scan for all pact files that match that provider and define a consumer for each pact file in the directory. Consumer name is read from contents of pact file. ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.11</version> <configuration> <serviceProviders> <!-- You can define as many as you need, but each must have a unique name --> <serviceProvider> <name>provider1</name> <!-- All the provider properties are optional, and have sensible defaults (shown below) --> <protocol>http</protocol> <host>localhost</host> <port>8080</port> <path>/</path> <pactFileDirectory>path/to/pacts</pactFileDirectory> </serviceProvider> </serviceProviders> </configuration> </plugin> ``` ### Verifying all pact files from multiple directories for a provider [3.5.18+] If you want to specify multiple directories, you can use `pactFileDirectories`. The plugin will only fail the build if no pact files are loaded after processing all the directories in the list. ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.18</version> <configuration> <serviceProviders> <serviceProvider> <name>provider1</name> <pactFileDirectories> <pactFileDirectory>path/to/pacts1</pactFileDirectory> <pactFileDirectory>path/to/pacts2</pactFileDirectory> </pactFileDirectories> </serviceProvider> </serviceProviders> </configuration> </plugin> ``` ## Enabling insecure SSL For providers that are running on SSL with self-signed certificates, you need to enable insecure SSL mode by setting `<insecure>true</insecure>` on the provider. ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.11</version> <configuration> <serviceProviders> <serviceProvider> <name>provider1</name> <pactFileDirectory>path/to/pacts</pactFileDirectory> <insecure>true</insecure> </serviceProvider> </serviceProviders> </configuration> </plugin> ``` ## Specifying a custom trust store For environments that are running their own certificate chains: ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.11</version> <configuration> <serviceProviders> <serviceProvider> <name>provider1</name> <pactFileDirectory>path/to/pacts</pactFileDirectory> <trustStore>relative/path/to/trustStore.jks</trustStore> <trustStorePassword>changeit</trustStorePassword> </serviceProvider> </serviceProviders> </configuration> </plugin> ``` `trustStore` is either relative to the current working (build) directory. `trustStorePassword` defaults to `changeit`. NOTE: The hostname will still be verified against the certificate. ## Modifying the requests before they are sent Sometimes you may need to add things to the requests that can't be persisted in a pact file. Examples of these would be authentication tokens, which have a small life span. The Pact Maven plugin provides a request filter that can be set to a Groovy script on the provider that will be called before the request is made. This script will receive the HttpRequest bound to a variable named `request` prior to it being executed. ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.11</version> <configuration> <serviceProviders> <serviceProvider> <name>provider1</name> <requestFilter> // This is a Groovy script that adds an Authorization header to each request request.addHeader('Authorization', 'oauth-token eyJhbGciOiJSUzI1NiIsIm...') </requestFilter> <consumers> <consumer> <name>consumer1</name> <pactFile>path/to/provider1-consumer1-pact.json</pactFile> </consumer> </consumers> </serviceProvider> </serviceProviders> </configuration> </plugin> ``` __*Important Note:*__ You should only use this feature for things that can not be persisted in the pact file. By modifying the request, you are potentially modifying the contract from the consumer tests! ## Modifying the HTTP Client Used The default HTTP client is used for all requests to providers (created with a call to `HttpClients.createDefault()`). This can be changed by specifying a closure assigned to createClient on the provider that returns a CloseableHttpClient. For example: ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.11</version> <configuration> <serviceProviders> <serviceProvider> <name>provider1</name> <createClient> // This is a Groovy script that will enable the client to accept self-signed certificates import org.apache.http.ssl.SSLContextBuilder import org.apache.http.conn.ssl.NoopHostnameVerifier import org.apache.http.impl.client.HttpClients HttpClients.custom().setSSLHostnameVerifier(new NoopHostnameVerifier()) .setSslcontext(new SSLContextBuilder().loadTrustMaterial(null, { x509Certificates, s -> true }) .build()) .build() </createClient> <consumers> <consumer> <name>consumer1</name> <pactFile>path/to/provider1-consumer1-pact.json</pactFile> </consumer> </consumers> </serviceProvider> </serviceProviders> </configuration> </plugin> ``` ## Turning off URL decoding of the paths in the pact file By default the paths loaded from the pact file will be decoded before the request is sent to the provider. To turn this behaviour off, set the system property `pact.verifier.disableUrlPathDecoding` to `true`. __*Important Note:*__ If you turn off the url path decoding, you need to ensure that the paths in the pact files are correctly encoded. The verifier will not be able to make a request with an invalid encoded path. ## Plugin Properties The following plugin properties can be specified with `-Dproperty=value` on the command line or in the configuration section: |Property|Description| |--------|-----------| |pact.showStacktrace|This turns on stacktrace printing for each request. It can help with diagnosing network errors| |pact.showFullDiff|This turns on displaying the full diff of the expected versus actual bodies| |pact.filter.consumers|Comma separated list of consumer names to verify| |pact.filter.description|Only verify interactions whose description match the provided regular expression| |pact.filter.providerState|Only verify interactions whose provider state match the provided regular expression. An empty string matches interactions that have no state| |pact.verifier.publishResults|Publishing of verification results will be skipped unless this property is set to 'true' [version 3.5.18+]| |pact.matching.wildcard|Enables matching of map values ignoring the keys when this property is set to 'true'| Example in the configuration section: ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.11</version> <configuration> <serviceProviders> <serviceProvider> <name>provider1</name> <consumers> <consumer> <name>consumer1</name> <pactFile>path/to/provider1-consumer1-pact.json</pactFile> </consumer> </consumers> </serviceProvider> </serviceProviders> <configuration> <pact.showStacktrace>true</pact.showStacktrace> </configuration> </configuration> </plugin> ``` ## Provider States For each provider you can specify a state change URL to use to switch the state of the provider. This URL will receive the providerState description and parameters from the pact file before each interaction via a POST. The stateChangeUsesBody controls if the state is passed in the request body or as query parameters. These values can be set at the provider level, or for a specific consumer. Consumer values take precedent if both are given. ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.11</version> <configuration> <serviceProviders> <serviceProvider> <name>provider1</name> <stateChangeUrl>http://localhost:8080/tasks/pactStateChange</stateChangeUrl> <stateChangeUsesBody>false</stateChangeUsesBody> <!-- defaults to true --> <consumers> <consumer> <name>consumer1</name> <pactFile>path/to/provider1-consumer1-pact.json</pactFile> <stateChangeUrl>http://localhost:8080/tasks/pactStateChangeForConsumer1</stateChangeUrl> <stateChangeUsesBody>false</stateChangeUsesBody> <!-- defaults to true --> </consumer> </consumers> </serviceProvider> </serviceProviders> </configuration> </plugin> ``` If the `stateChangeUsesBody` is not specified, or is set to true, then the provider state description and parameters will be sent as JSON in the body of the request. If it is set to false, they will passed as query parameters. As for normal requests (see Modifying the requests before they are sent), a state change request can be modified before it is sent. Set `stateChangeRequestFilter` to a Groovy script on the provider that will be called before the request is made. #### Teardown calls for state changes You can enable teardown state change calls by setting the property `<stateChangeTeardown>true</stateChangeTeardown>` on the provider. This will add an `action` parameter to the state change call. The setup call before the test will receive `action=setup`, and then a teardown call will be made afterwards to the state change URL with `action=teardown`. ## Verifying pact files from a pact broker You can setup your build to validate against the pacts stored in a pact broker. The pact plugin will query the pact broker for all consumers that have a pact with the provider based on its name. To use it, just configure the `pactBrokerUrl` or `pactBroker` value for the provider with the base URL to the pact broker. For example: ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.11</version> <configuration> <serviceProviders> <serviceProvider> <name>provider1</name> <stateChangeUrl>http://localhost:8080/tasks/pactStateChange</stateChangeUrl> <pactBrokerUrl>http://pact-broker:5000/</pactBrokerUrl> </serviceProvider> </serviceProviders> </configuration> </plugin> ``` ### Verifying pacts from an authenticated pact broker If your pact broker requires authentication (basic authentication is only supported), you can configure the username and password to use by configuring the `authentication` element of the `pactBroker` element of your provider. For example: ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.11</version> <configuration> <serviceProviders> <serviceProvider> <name>provider1</name> <stateChangeUrl>http://localhost:8080/tasks/pactStateChange</stateChangeUrl> <pactBroker> <url>http://pactbroker:1234</url> <authentication> <username>test</username> <password>test</password> </authentication> </pactBroker> </serviceProvider> </serviceProviders> </configuration> </plugin> ``` #### Using the Maven servers configuration [version 3.5.6+] From version 3.5.6, you can use the servers setup in the Maven settings. To do this, setup a server as per the [Maven Server Settings](https://maven.apache.org/settings.html#Servers). Then set the server ID in the pact broker configuration in your POM. ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.6</version> <configuration> <serviceProviders> <serviceProvider> <name>provider1</name> <stateChangeUrl>http://localhost:8080/tasks/pactStateChange</stateChangeUrl> <pactBroker> <url>http://pactbroker:1234</url> <serverId>test-pact-broker</serverId> <!-- This must match the server id in the maven settings --> </pactBroker> </serviceProvider> </serviceProviders> </configuration> </plugin> ``` ### Verifying pacts from an pact broker that match particular tags If your pacts in your pact broker have been tagged, you can set the tags to fetch by configuring the `tags` element of the `pactBroker` element of your provider. For example: ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.11</version> <configuration> <serviceProviders> <serviceProvider> <name>provider1</name> <stateChangeUrl>http://localhost:8080/tasks/pactStateChange</stateChangeUrl> <pactBroker> <url>http://pactbroker:1234</url> <tags> <tag>TEST</tag> <tag>DEV</tag> </tags> </pactBroker> </serviceProvider> </serviceProviders> </configuration> </plugin> ``` This example will fetch and validate the pacts for the TEST and DEV tags. ## Filtering the interactions that are verified You can filter the interactions that are run using three properties: `pact.filter.consumers`, `pact.filter.description` and `pact.filter.providerState`. Adding `-Dpact.filter.consumers=consumer1,consumer2` to the command line or configuration section will only run the pact files for those consumers (consumer1 and consumer2). Adding `-Dpact.filter.description=a request for payment.*` will only run those interactions whose descriptions start with 'a request for payment'. `-Dpact.filter.providerState=.*payment` will match any interaction that has a provider state that ends with payment, and `-Dpact.filter.providerState=` will match any interaction that does not have a provider state. ## Not failing the build if no pact files are found [version 3.5.19+] By default, if there are no pact files to verify, the plugin will raise an exception. This is to guard against false positives where the build is passing but nothing has been verified due to mis-configuration. To disable this behaviour, set the `failIfNoPactsFound` parameter to `false`. # Verifying a message provider The Maven plugin has been updated to allow invoking test methods that can return the message contents from a message producer. To use it, set the way to invoke the verification to `ANNOTATED_METHOD`. This will allow the pact verification task to scan for test methods that return the message contents. Add something like the following to your maven pom file: ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.11</version> <configuration> <serviceProviders> <serviceProvider> <name>messageProvider</name> <verificationType>ANNOTATED_METHOD</verificationType> <!-- packagesToScan is optional, but leaving it out will result in the entire test classpath being scanned. Set it to the packages where your annotated test method can be found. --> <packagesToScan> <packageToScan>au.com.example.messageprovider.*</packageToScan> </packagesToScan> <consumers> <consumer> <name>consumer1</name> <pactFile>path/to/messageprovider-consumer1-pact.json</pactFile> </consumer> </consumers> </serviceProvider> </serviceProviders> </configuration> </plugin> ``` Now when the pact verify task is run, will look for methods annotated with `@PactVerifyProvider` in the test classpath that have a matching description to what is in the pact file. ```groovy class ConfirmationKafkaMessageBuilderTest { @PactVerifyProvider('an order confirmation message') String verifyMessageForOrder() { Order order = new Order() order.setId(10000004) order.setExchange('ASX') order.setSecurityCode('CBA') order.setPrice(BigDecimal.TEN) order.setUnits(15) order.setGst(new BigDecimal('15.0')) odrer.setFees(BigDecimal.TEN) def message = new ConfirmationKafkaMessageBuilder() .withOrder(order) .build() JsonOutput.toJson(message) } } ``` It will then validate that the returned contents matches the contents for the message in the pact file. ## Changing the class path that is scanned By default, the test classpath is scanned for annotated methods. You can override this by setting the `classpathElements` property: ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.11</version> <configuration> <serviceProviders> <serviceProvider> <name>messageProvider</name> <verificationType>ANNOTATED_METHOD</verificationType> <consumers> <consumer> <name>consumer1</name> <pactFile>path/to/messageprovider-consumer1-pact.json</pactFile> </consumer> </consumers> </serviceProvider> </serviceProviders> <classpathElements> <classpathElement> build/classes/test </classpathElement> </classpathElements> </configuration> </plugin> ``` # Publishing pact files to a pact broker The pact maven plugin provides a `publish` mojo that can publish all pact files in a directory to a pact broker. To use it, you need to add a publish configuration to the POM that defines the directory where the pact files are and the URL to the pact broker. For example: ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.11</version> <configuration> <pactDirectory>path/to/pact/files</pactDirectory> <!-- Defaults to ${project.build.directory}/pacts --> <pactBrokerUrl>http://pactbroker:1234</pactBrokerUrl> <projectVersion>1.0.100</projectVersion> <!-- Defaults to ${project.version} --> <trimSnapshot>true</trimSnapshot> <!-- Defaults to false --> </configuration> </plugin> ``` You can now execute `mvn pact:publish` to publish the pact files. _NOTE:_ The pact broker requires a version for all published pacts. The `publish` task will use the version of the project by default, but can be overwritten with the `projectVersion` property. Make sure you have set one otherwise the broker will reject the pact files. _NOTE_: By default, the pact broker has issues parsing `SNAPSHOT` versions. You can configure the publisher to automatically remove `-SNAPSHOT` from your version number by setting `trimSnapshot` to true. This setting does not modify non-snapshot versions. You can set any tags that the pacts should be published with by setting the `tags` list property (version 3.5.12+). A common use of this is setting the tag to the current source control branch. This supports using pact with feature branches. ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.12</version> <configuration> <pactDirectory>path/to/pact/files</pactDirectory> <!-- Defaults to ${project.build.directory}/pacts --> <pactBrokerUrl>http://pactbroker:1234</pactBrokerUrl> <projectVersion>1.0.100</projectVersion> <!-- Defaults to ${project.version} --> <tags> <tag>feature/feature_name</tag> </tags> </configuration> </plugin> ``` ## Publishing to an authenticated pact broker For an authenticated pact broker, you can pass in the credentials with the `pactBrokerUsername` and `pactBrokerPassword` properties. Currently it only supports basic authentication. For example: ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.11</version> <configuration> <pactBrokerUrl>http://pactbroker:1234</pactBrokerUrl> <pactBrokerUsername>USERNAME</pactBrokerUsername> <pactBrokerPassword>PASSWORD</pactBrokerPassword> </configuration> </plugin> ``` #### Using the Maven servers configuration [version 3.5.6+] From version 3.5.6, you can use the servers setup in the Maven settings. To do this, setup a server as per the [Maven Server Settings](https://maven.apache.org/settings.html#Servers). Then set the server ID in the pact broker configuration in your POM. ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.11</artifactId> <version>3.5.19</version> <configuration> <pactBrokerUrl>http://pactbroker:1234</pactBrokerUrl> <pactBrokerServerId>test-pact-broker</pactBrokerServerId> <!-- This must match the server id in the maven settings --> </configuration> </plugin> ``` ## Excluding pacts from being published [version 3.5.19+] You can exclude some of the pact files from being published by providing a list of regular expressions that match against the base names of the pact files. For example: ```groovy pact { publish { pactBrokerUrl = 'https://mypactbroker.com' excludes = [ '.*\\-\\d+$' ] // exclude all pact files that end with a dash followed by a number in the name } } ``` ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.19</version> <configuration> <pactBrokerUrl>http://pactbroker:1234</pactBrokerUrl> <excludes> <exclude>.*\\-\\d+$</exclude> <!-- exclude pact files where the name ends in a dash followed by a number --> </excludes> </configuration> </plugin> ``` # Publishing verification results to a Pact Broker [version 3.5.4+] For pacts that are loaded from a Pact Broker, the results of running the verification can be published back to the broker against the URL for the pact. You will be able to then see the result on the Pact Broker home screen. To turn on the verification publishing, set the system property `pact.verifier.publishResults` to `true` in the pact maven plugin, not surefire, configuration. # Enabling other verification reports [version 3.5.20+] By default the verification report is written to the console. You can also enable a JSON or Markdown report by setting the `reports` configuration list. ```xml <plugin> <groupId>au.com.dius</groupId> <artifactId>pact-jvm-provider-maven_2.12</artifactId> <version>3.5.20</version> <configuration> <reports> <report>console</report> <report>json</report> <report>markdown</report> </reports> </configuration> </plugin> ``` These reports will be written to `target/reports/pact`.

The Evinced Appium Java SDK integrates with new or existing Appium tests to automatically detect accessibility issues. With the addition of as few as 5 lines of code to your Appium framework, you can begin to analyze your entire application to understand how it can become more accessible. At the conclusion of the test, a rich and comprehensive HTML or JSON report is generated to track issues in any reporting tool.

Heimdall is a Test Reporting Tool which is primarily built to publish your test execution results to the google sheets from

Selenium-Toolkit is a Java based test-toolkit for selenium-testing with testNg. The goal of the toolkit is, to suport you in different things like 'how to manage testdata in the source', parallelisation, Webdriver-managemet, reporting and a lot more.

Heimdall is a Test Reporting Tool which is primarily built to publish your test execution results to the google sheets from

Idaithalam is a low code Test automation Framework, developed using Java and Cucumber. It leverages Behavior Driven Development (BDD). Tester can create test cases/scripts in simple Excel with API Spec. Excel is a simplified way to create Json based test scripts in Idaithalam. Test cases can be created quickly and tested in minutes. As a tester, you need to create Excel and pass it to Idaithalam Framework. First, generate the Json based test scripts(Virtualan Collection) from Excel. During test execution, this test script collection can be directly utilized. Then it generates Feature files from the Virtualan Collection and its executed. Lastly, It generates test report in BDD/Cucumber style. This provide complete testing support for REST APIs, GraphQL, RDBMS DB and Kafka Event messages #idaithalam #apitesting #testautomation #restapitesting #insprintautomation #rdbmstesting #graphqltesting #kafkamessagetesting #eventvalidation #messagevalidation #cucumber #bdd Virtualan Collection is a JSON based TestScripts generated from Excel API testing can be automated using Excel, POSTMAN collection and VIRTUALAN - Utilized for API testing. - Utilized Contract testing. - Utilized Production Checkout. - Agile Testing - Progressive Regression can be done for each sprint. Reduce lot of manual effects.

test report metrics and trend analysis reporting :: reportcard-model

A BDD / Testing framework with pure Groovy DSL, running in a Spark cluster, advanced reporting and clearly define the boundary between BA Tester / developer

Generic runtime Test reporting listeners.

Module "cli" of open-test-reporting

Write Pitest test results to mutation-testing-elements HTML reporter

Stripes web framework parent project.