Testing Framework

A toolkit of support libraries and Ruby core extensions extracted from the Rails framework. Rich support for multibyte strings, internationalization, time zones, and testing.

A toolkit for building modeling frameworks like Active Record. Rich support for attributes, callbacks, validations, serialization, internationalization, and testing.

minitest provides a complete suite of testing facilities supporting TDD, BDD, mocking, and benchmarking. "I had a class with Jim Weirich on testing last week and we were allowed to choose our testing frameworks. Kirk Haines and I were paired up and we cracked open the code for a few test frameworks... I MUST say that minitest is *very* readable / understandable compared to the 'other two' options we looked at. Nicely done and thank you for helping us keep our mental sanity." -- Wayne E. Seguin minitest/test is a small and incredibly fast unit testing framework. It provides a rich set of assertions to make your tests clean and readable. minitest/spec is a functionally complete spec engine. It hooks onto minitest/test and seamlessly bridges test assertions over to spec expectations. minitest/benchmark is an awesome way to assert the performance of your algorithms in a repeatable manner. Now you can assert that your newb co-worker doesn't replace your linear algorithm with an exponential one! minitest/mock by Steven Baker, is a beautifully tiny mock (and stub) object framework. minitest/pride shows pride in testing and adds coloring to your test output. I guess it is an example of how to write IO pipes too. :P minitest/test is meant to have a clean implementation for language implementors that need a minimal set of methods to bootstrap a working test suite. For example, there is no magic involved for test-case discovery. "Again, I can't praise enough the idea of a testing/specing framework that I can actually read in full in one sitting!" -- Piotr Szotkowski Comparing to rspec: rspec is a testing DSL. minitest is ruby. -- Adam Hawkins, "Bow Before MiniTest" minitest doesn't reinvent anything that ruby already provides, like: classes, modules, inheritance, methods. This means you only have to learn ruby to use minitest and all of your regular OO practices like extract-method refactorings still apply.

RSpec's 'test double' framework, with support for stubbing and mocking

Rack::Test is a small, simple testing API for Rack apps. It can be used on its own or as a reusable starting point for Web frameworks and testing libraries to build on.

InSpec provides a framework for creating end-to-end infrastructure tests. You can use it for integration or even compliance testing. Create fully portable test profiles and use them in your workflow to ensure stability and security. Integrate InSpec in your change lifecycle for local testing, CI/CD, and deployment verification. Packaged distributions of Progress® Chef® products obtained from RubyGems are made available pursuant to the Progress Chef EULA at https://www.chef.io/end-user-license-agreement, unless there is an executed agreement in effect between you and Progress that covers the Progress Chef products ("Master Agreement"), in which case the Master Agreement shall govern. Source code obtained from the Chef GitHub repository is made available under Apache-2.0, a copy of which is included.

Test your JavaScript without any framework dependencies, in any environment, and with a nice descriptive syntax. Jasmine for Ruby is deprecated. The direct replacment for the jasmine-core gem is the jasmine-core NPM package. If you are also using the jasmine gem, we recommend using the jasmine-browser-runner NPM package instead. It supports all the same scenarios as the jasmine gem gem plus Webpacker. See https://jasmine.github.io/setup/browser.html for setup instructions, and https://github.com/jasmine/jasmine-gem/blob/main/release_notes/3.9.0.md for other options.

InSpec provides a framework for creating end-to-end infrastructure tests. You can use it for integration or even compliance testing. Create fully portable test profiles and use them in your workflow to ensure stability and security. Integrate InSpec in your change lifecycle for local testing, CI/CD, and deployment verification. This has local support only. See the `inspec` gem for full support. Packaged distributions of Progress® Chef® products obtained from RubyGems are made available pursuant to the Progress Chef EULA at https://www.chef.io/end-user-license-agreement, unless there is an executed agreement in effect between you and Progress that covers the Progress Chef products ("Master Agreement"), in which case the Master Agreement shall govern. Source code obtained from the Chef GitHub repository is made available under Apache-2.0, a copy of which is included.

Webrat lets you quickly write expressive and robust acceptance tests for a Ruby web application. It supports simulating a browser inside a Ruby process to avoid the performance hit and browser dependency of Selenium or Watir, but the same API can also be used to drive real Selenium tests when necessary (eg. for testing AJAX interactions). Most Ruby web frameworks and testing frameworks are supported.

test-unit (Test::Unit) is unit testing framework for Ruby, based on xUnit principles. These were originally designed by Kent Beck, creator of extreme programming software development methodology, for Smalltalk's SUnit. It allows writing tests, checking results and automated testing in Ruby.

Test your JavaScript without any framework dependencies, in any environment, and with a nice descriptive syntax. Jasmine for Ruby is deprecated. We recommend using the jasmine-browser-runner NPM package instead. It supports all the same scenarios as this gem plus Webpacker. See https://jasmine.github.io/setup/browser.html for setup instructions, and https://github.com/jasmine/jasmine-gem/blob/main/release_notes/3.9.0.md for other options.

Extension for popular TDD and BDD frameworks like "Cucumber", "RSpec" and "Minitest", to make testing command line applications meaningful, easy and fun.

ChefSpec is a unit testing and resource coverage (code coverage) framework for testing Chef cookbooks ChefSpec makes it easy to write examples and get fast feedback on cookbook changes without the need for virtual machines or cloud servers.

CI::Reporter is an add-on to Ruby testing frameworks that allows you to generate XML reports of your test runs. The resulting files can be read by a continuous integration system that understands Ant's JUnit report format.

Rack based split testing framework

RSpec driven API testing framework

RR is a test double framework that features a rich selection of double techniques and a terse syntax.

Guard::Minitest automatically run your tests with Minitest framework (much like autotest)

calabash-cucumber drives tests for native iOS apps. You must link your app with calabash-ios-server framework to execute tests.

Konacha is a Rails engine that allows you to test your JavaScript with the mocha test framework and chai assertion library. It is similar to Jasmine and Evergreen, but does not attempt to be framework agnostic. By sticking with Rails, Konacha can take full advantage of features such as the asset pipeline and engines.

Infrastructure Behavior Testing Framework

An extremely fast, expressive, and context-driven unit-testing framework. A replacement for all other testing frameworks. Protest the slow test.

Utilities allowing to implement allure result generation by other test frameworks

A lightweight unit testing framework. Tests (expectations) will be written as follows expect 2 do 1 + 1 end expect(1 + 1) == 2 expect NoMethodError do Object.invalid_method_call end.

This gem provides a set of modules and methods to help quickly and DRYly create a test automation framework using Ruby and Watir (or watir-webdriver).

Ceedling is a build automation tool that helps you create and run C unit test suites. Ceedling provides two core functions: [1] It packages up several tools including the C unit test framework Unity, the Ruby-based mock generation tool CMock, and a C exception library CException. [2] It extends Rake with functionality specific to generating, building, and executing C test suites. Ceedling projects are created with a YAML configuration file. A variety of conventions within the tool simplify generating mocks from C files and assembling suites of unit test functions.

JavaScript BDD Testing Framework

Glimmer DSL for SWT (JRuby Desktop Development Cross-Platform Native GUI Framework) is a native-GUI cross-platform desktop development library written in JRuby, an OS-threaded faster JVM version of Ruby. It includes SWT 4.30 (released on December 1, 2023). Glimmer's main innovation is a declarative Ruby DSL that enables productive and efficient authoring of professional-grade desktop applications by relying on the robust Eclipse SWT library, with the familiar native look, feel, and behavior of GUI on Mac, Windows, and Linux. Glimmer additionally innovates by having built-in data-binding support, which greatly facilitates synchronizing the GUI with domain models, thus achieving true decoupling of object oriented components and enabling developers to solve business problems (test-first) without worrying about GUI concerns, or alternatively drive development GUI-first, and then write clean business models (test-first) afterwards. Not only does Glimmer provide a large set of GUI widgets, but it also supports drawing Canvas Graphics like Shapes and Animations. To get started quickly, Glimmer offers scaffolding options for Apps, Gems, and Custom Widgets. Glimmer also includes native-executable packaging support, sorely lacking in other libraries, thus enabling the delivery of desktop apps written in Ruby as truly native DMG/PKG/APP files on the Mac, MSI/EXE files on Windows, and DEB/RPM files on Linux. Glimmer was the first Ruby gem to bring SWT (Standard Widget Toolkit) to Ruby, thanks to creator Andy Maleh, EclipseCon/EclipseWorld/RubyConf speaker. If you liked Shoes, You'll love Glimmer!

Framework for test-driven infrastructure development.

A diff based testing framework for static sites.

Using standard Gherkin language to define security tests, gauntlt happily wraps cucumber functionality and provides a security testing framework that security engineers, developers and operations teams can collaborate on together.

Protest is a tiny, simple, and easy-to-extend test framework for ruby.

NUnit is a unit-testing framework for all .Net languages. Initially ported from JUnit, the current production release, version 2.5, is the sixth major release of this xUnit based unit testing tool for Microsoft .NET. It is written entirely in C# and has been completely redesigned to take advantage of many .NET language features, for example custom attributes and other reflection related capabilities. NUnit brings xUnit to all .NET languages.

Solano CI runs your test suite simply and quickly in our managed cloud environment. You can run tests by hand, or enable our hosted CI to watch your git repos automatically. Solano CI automatically and safely parallelizes your tests to save you time, and takes care of setting up fresh isolated DB instances for each test thread. Tests have access to a wide variety of databases (postgres, mongo, redis, mysql, memcache), solr, sphinx, selenium/webdriver browsers, webkit and culerity. Solano CI supports all common Ruby test frameworks, including rspec, cucumber, test::unit, and spinach. Solano CI also supports Javascript testing using jasmine, evergreen, and many other frameworks.

A random testing / fuzzer framework for Ruby.

Protractor rails makes it a bit easier to integrate and run angularjs' protractor e2e test framework directly in your rails app.

A Wholesale framework for web, mobile, api, load, accessibility testing. Get you going in a minute. An instant setup of Ruby, Cucumber, Capybara, selenium, REST API Appium, Docker, Gatling and Axe Engine

Guard::PHPUnit automatically run your unit-tests written with the PHPUnit testing framework.

A very light yet powerful test factory framework with an extremely clean syntax that makes it very easy to define your factories.

AXElements is a UI automation library built on top of various OS X frameworks. You can use AXElements to write functional tests for Cocoa apps, script UI interactions, or even to build higher level tools such as screen readers.

Pacto is a judge that arbitrates contract disputes between a service provider and one or more consumers. In other words, it is a framework for Integration Contract Testing, and helps guide service evolution patterns like Consumer-Driven Contracts or Documentation-Driven Contracts.

Testing Framework solely based on plugins. For now only tests using Bash.

A simple test framework for Ruby code that uses introspection to allow defining checks in comments.

Assertion style testing framework.

Inventory Inventory keeps track of the contents of your Ruby¹ projects. Such an inventory can be used to load the project, create gem specifications and gems, run unit tests, compile extensions, and verify that the project’s content is what you think it is. ¹ See http://ruby-lang.org/ § Usage Let’s begin by discussing the project structure that Inventory expects you to use. It’s pretty much exactly the same as the standard Ruby project structure¹: ├── README ├── Rakefile ├── lib │ ├── foo-1.0 │ │ ├── bar.rb │ │ └── version.rb │ └── foo-1.0.rb └── test └── unit ├── foo-1.0 │ ├── bar.rb │ └── version.rb └── foo-1.0.rb Here you see a simplified version of a project called “Foo”’s project structure. The only real difference from the standard is that the main entry point into the library is named “foo-1.0.rb” instead of “foo.rb” and that the root sub-directory of “lib” is similarly named “foo-1.0” instead of “foo”. The difference is the inclusion of the API version. This must be the major version of the project followed by a constant “.0”. The reason for this is that it allows concurrent installations of different major versions of the project and means that the wrong version will never accidentally be loaded with require. There’s a bigger difference in the content of the files. ‹Lib/foo-1.0/version.rb› will contain our inventory instead of a String: require 'inventory-1.0' class Foo Version = Foo.new(1, 4, 0){ authors{ author 'A. U. Thor', 'a.u.thor@example.org' } homepage 'http://example.org/' licenses{ license 'LGPLv3+', 'GNU Lesser General Public License, version 3 or later', 'http://www.gnu.org/licenses/' } def dependencies super + Dependencies.new{ development 'baz', 1, 3, 0 runtime 'goo', 2, 0, 0 optional 'roo-loo', 3, 0, 0, :feature => 'roo-loo' } end def package_libs %w[bar.rb] end } end We’re introducing quite a few concepts at once, and we’ll look into each in greater detail, but we begin by setting the ‹Version› constant to a new instance of an Inventory with major, minor, and patch version atoms 1, 4, and 0. Then we add a couple of dependencies and list the library files that are included in this project. The version numbers shouldn’t come as a surprise. These track the version of the API that we’re shipping using {semantic versioning}². They also allow the Inventory#to_s method to act as if you’d defined Version as ‹'1.4.0'›. Next follows information about the authors of the project, the project’s homepage, and the project’s licenses. Each author has a name and an email address. The homepage is simply a string URL. Licenses have an abbreviation, a name, and a URL where the license text can be found. We then extend the definition of ‹dependencies› by adding another set of dependencies to ‹super›. ‹Super› includes a dependency on the version of the inventory project that’s being used with this project, so you’ll never have to list that yourself. The other three dependencies are all of different kinds: development, runtime, and optional. A development dependency is one that’s required while developing the project, for example, a unit-testing framework, a documentation generator, and so on. Runtime dependencies are requirements of the project to be able to run, both during development and when installed. Finally, optional dependencies are runtime dependencies that may or may not be required during execution. The difference between runtime and optional is that the inventory won’t try to automatically load an optional dependency, instead leaving that up to you to do when and if it becomes necessary. By that logic, runtime dependencies will be automatically loaded, which is a good reason for having dependency information available at runtime. The version numbers of dependencies also use semantic versioning, but note that the patch atom is ignored unless the major atom is 0. You should always only depend on the major and minor atoms. As mentioned, runtime dependencies will be automatically loaded and the feature they try to load is based on the name of the dependency with a “-X.0” tacked on the end, where ‘X’ is the major version of the dependency. Sometimes, this isn’t correct, in which case the :feature option may be given to specify the name of the feature. You may also override other parts of a dependency by passing in a block to the dependency, much like we’re doing for inventories. The rest of an inventory will list the various files included in the project. This project only consists of one additional file to those that an inventory automatically include (Rakefile, README, the main entry point, and the version.rb file that defines the inventory itself), namely the library file ‹bar.rb›. Library files will be loaded automatically when the main entry point file loads the inventory. Library files that shouldn’t be loaded may be listed under a different heading, namely “additional_libs”. Both these sets of files will be used to generate a list of unit test files automatically, so each library file will have a corresponding unit test file in the inventory. We’ll discuss the different headings of an inventory in more detail later on. Now that we’ve written our inventory, let’s set it up so that it’s content gets loaded when our main entry point gets loaded. We add the following piece of code to ‹lib/foo-1.0.rb›: module Foo load File.expand_path('../foo-1.0/version.rb', __FILE__) Version.load end That’s all there’s to it. The inventory can also be used to great effect from a Rakefile using a separate project called Inventory-Rake³. Using it’ll give us tasks for cleaning up our project, compiling extensions, installing dependencies, installing and uninstalling the project itself, and creating and pushing distribution files to distribution points. require 'inventory-rake-1.0' load File.expand_path('../lib/foo-1.0/version.rb', __FILE__) Inventory::Rake::Tasks.define Foo::Version Inventory::Rake::Tasks.unless_installing_dependencies do require 'lookout-rake-3.0' Lookout::Rake::Tasks::Test.new end It’s ‹Inventory::Rake::Tasks.define› that does the heavy lifting. It takes our inventory and sets up the tasks mentioned above. As we want to be able to use our Rakefile to install our dependencies for us, the rest of the Rakefile is inside the conditional #unless_installing_dependencies, which, as the name certainly implies, executes its block unless the task being run is the one that installs our dependencies. This becomes relevant when we set up Travis⁴ integration next. The only conditional set-up we do in our Rakefile is creating our test task via Lookout-Rake⁵, which also uses our inventory to find the unit tests to run when executed. Travis integration is straightforward. Simply put before_script: - gem install inventory-rake -v '~> VERSION' --no-rdoc --no-ri - rake gem:deps:install in the project’s ‹.travis.yml› file, replacing ‹VERSION› with the version of Inventory-Rake that you require. This’ll make sure that Travis installs all development, runtime, and optional dependencies that you’ve listed in your inventory before running any tests. You might also need to put env: - RUBYOPT=rubygems in your ‹.travis.yml› file, depending on how things are set up. ¹ Ruby project structure: http://guides.rubygems.org/make-your-own-gem/ ² Semantic versioning: http://semver.org/ ³ Inventory-Rake: http://disu.se/software/inventory-rake-1.0/ ⁴ Travis: http://travis-ci.org/ ⁵ Lookout-Rake: http://disu.se/software/lookout-rake-3.0/ § API If the guide above doesn’t provide you with all the answers you seek, you may refer to the API¹ for more answers. ¹ See http://disu.se/software/inventory-1.0/api/Inventory/ § Financing Currently, most of my time is spent at my day job and in my rather busy private life. Please motivate me to spend time on this piece of software by donating some of your money to this project. Yeah, I realize that requesting money to develop software is a bit, well, capitalistic of me. But please realize that I live in a capitalistic society and I need money to have other people give me the things that I need to continue living under the rules of said society. So, if you feel that this piece of software has helped you out enough to warrant a reward, please PayPal a donation to now@disu.se¹. Thanks! Your support won’t go unnoticed! ¹ Send a donation: https://www.paypal.com/cgi-bin/webscr?cmd=_donations&business=now@disu.se&item_name=Inventory § Reporting Bugs Please report any bugs that you encounter to the {issue tracker}¹. ¹ See https://github.com/now/inventory/issues § Authors Nikolai Weibull wrote the code, the tests, the documentation, and this README. § Licensing Inventory is free software: you may redistribute it and/or modify it under the terms of the {GNU Lesser General Public License, version 3}¹ or later², as published by the {Free Software Foundation}³. ¹ See http://disu.se/licenses/lgpl-3.0/ ² See http://gnu.org/licenses/ ³ See http://fsf.org/

TLB-Ruby base library that provides common test-load-balancing infrastructure for Ruby testing tools. Core in itself is framework agnostic. It exposes APIs that allow any framework specific libraries to load-balance.

Colored output for Bacon test framework! http://i.imgur.com/EpTpw.png

gizmo is a simple page model testing framework used and sponsored by 'realestate.com.au'. The aim of the project is to DRY up your testing assertions by abstracting code that defines your page resulting in a consistent, easy to maintain test suit

Assertive Expressive is an assertions library specifically designed for reuse by other test frameworks.

Extremely mynymal test framework. Perfect for DIY lovers. Nanotest provides the bare mynymum needed; for everything else, there's ruby.