AQUA: Automated QUality Analysis
About
AQUA improves code quality by surfacing actionable metrics about JavaScript, CSS and C#.
AQUA raises the visibility of code quality and increases awareness within teams by giving immediate feedback about code smells before they become technical debt.
You are free to structure your projects anyway you like, and still take full advantage of AQUA. Simply create an aqua.project.json
file that documents where things are.
Philosophy
The goal of AQUA is to improve code quality, without dictating a strict project structure. Ramp-up gradually by only configuring the the parts of AQUA you want to use. Not ready for end-to-end tests yet? Simply don't configure them. Don't want to type check your JavaScript code? Just leave out the "types" section. Even if you just start by linting your code, that will be a great step in the right direction!
While nothing is stopping you from simply editing a JavaScript file and committing it to source control; doing so will cause you to miss out on: linting, cyclomatic complexity analysis, type checking, running both unit and end-to-end tests, as well as documentation generation, and code coverage reports.
Features
JavaScript
- Lint Source Code for Syntax Errors and Anti-Patterns
- Automatically fix lint errors non-destructively (TODO)
- Analyze Code Against Complexity Thresholds
- Type Checking Source Code (TODO)
- Advanced Minification (TODO)
- Unit Test Runnner
- Code Coverage Reports
- End-to-end Test Runner
- Documentation Generation
CSS
TODO: pull requests welcome :)
- Lint StyleSheets for Syntax Errors and Anti-Patterns
C#
TODO: pull requests welcome :)
- Lint Source Code for Syntax Errors and Anti-Patterns (via FxCop)
- Analyze Code Against Complexity Thresholds (via Code Metrics PowerTool)
- Unit Test Runnner
- Code Coverage Reports
- End-to-end Test Runner
- Documentation Generation
Environments
aqua is designed to play well with the following types of projects.
- Web Projects
- Node.js Projects
Projects Using AQUA
Workflow
AQUA uses Gulp to orchestrate the following suite of NPM Modules.
The Node modules above can the run individually (see tasks below) or chained together.
Run from the Command Line
AQUA dynamically creates gulp tasks for you, based on what is configured in your aqua.project.json
file. Tasks are namespaced with the project id in your config file. Any tasks that are not configured for the project are simply skipped.
task | does |
---|
gulp | List all possible tasks for all projects |
gulp ? | List all possible tasks for all projects |
gulp {id} | Run all project tasks in the correct order |
gulp {id}-? | List all possible tasks for the project |
gulp {id}-all | Run all project tasks in the correct order |
gulp {id}-lint | Lint the source code for syntax issues and anti-patterns |
gulp {id}-lint-fix | Automatically fix lint errors non-destructively |
gulp {id}-gpa | Analyze the source against complexity thresholds |
gulp {id}-unit | Run unit tests against the source code |
gulp {id}-chk | Type check the source code |
gulp {id}-min | Minify the source code and create source map |
gulp {id}-e2e | Run end-to-end tests against the minified code |
gulp {id}-doc | Generate documentation from code annotations |
gulp {id}-wch | Watch for file changes and automatically run tasks |
Console Customization
Avoid a lot needless typing! You can create a shortcut that will open the command prompt at any folder you want.
Use the steps below for Windows.
- Right-click on your desktop and choose
New > Shortcut
- For the location paste in (with the folder you want to open):
C:\Windows\System32\cmd.exe /K "cd /d [Drive/Folder/You/Want/To/Open]"
- Click
Next
, Set a Name and Click Finish
- Leave it on your desktop or drag it to your Taskbar to pin it there.
Also now that you have a shortcut you can customize your command window using Right-click > Properties
:
- Change the font used to something more readable
- Decrease the font size to show more information
- Increase the window size to reduce line wrapping
- Increase the screen buffer to scroll farther back in the task history
Canceling a Task
So you ran gulp aqa-wch
and you want to stop watching for changes to the project source files. Or maybe you accidentally typed gulp aqa-e2e
and don't want to wait for all the end-to-end tests to complete? Pressing Ctrl+c will pause the task and ask if you want to Terminate batch job (Y/N)?
typing y enter will cancel the task.
Quiet Mode
If there is any issues found by the quality checks such as a lint error or failing unit test the system will beep, unless --shh
is included. For example gulp aqa-unit --shh
Run from Chrome Browser
- Download the Gulp Devtools extension for Chrome Developer Tools from the Chrome Web Store.
- Gun
npm install gulp-devtools -g
- Run
gulp-devtools
from the your project root folder (where your gulpfile.js is located) - Open Chrome Dev tools, find the Gulp tab. The AQUA/Gulp tasks are now accessible from Chrome.
Environment Prep
You'll need to have node.js v10+ installed.
- navigate to your project root
- run
npm install
- for windows you may need to add
-msvs_version=2012
- install gulp globally
npm install gulp -g
Troubleshooting
During install some of you may encounter some issues.
Most issues can be solved by one of the following tips, but if are unable to find a solution feel free to an issue via the repository issue tracker.
Update NPM, Bower or Grunt
Sometimes you may find there is a weird error during install like npm's Error: ENOENT. Usually updating those tools to the latest version solves the issue.
- Updating NPM:
npm update -g npm
- Updating Grunt:
npm update -g grunt-cli
- Updating Bower:
npm update -g bower
Cleaning NPM and Bower cache
NPM and Bower has a caching system for holding packages that you already installed.
We found that often cleaning the cache solves some issues.
- NPM Clean Cache:
npm cache clean
- Bower Clean Cache:
bower cache clean
Testing
To improve efficiency and reduce cognitive load, all tests are written in JavaScipt, using a single testing framework (Jasmine). This means you can use the same assertion patterns and mocking strategies whether you're writing end-to-end tests for a web project or unit tests for your Node.js project.
Unit Tests
Unit tests are written in Jasmine and for web projects are run with Karma. Unit tests also have access to the Jasmine JQuery custom matchers and fixture loaders for HTML, CSS and JSON.
Debugging
Unit tests can be run from the commandline with gulp aqa-unit
or in your favorite browser.
Debugging can be done by setting break points in the source code, using console.log
and debugger;
statements.
End-to-end Tests
Page Objects
End to end testing (aka Integration Testing) should be organized using the PageObject Pattern and run by the Protractor API.
Page Objects can be thought of as facing in two directions simultaneously. Facing towards the developer of a test, they represent the services offered by a particular page. Facing away from the developer, they should be the only thing that has a deep knowledge of the structure of the HTML of a page (or part of a page) It's simplest to think of the methods on a Page Object as offering the "services" that a page offers rather than exposing the details and mechanics of the page. As an example, think of the inbox of any web-based email system. Amongst the services that it offers are typically the ability to compose a new email, to choose to read a single email, and to list the subject lines of the emails in the inbox. How these are implemented shouldn't matter to the test.
Page Object Pattern Goals
- The public methods represent the services that the page offers
- Try not to expose the internals of the page
- Generally don't make assertions
- Methods return other PageObjects
- Need not represent an entire page
- Different results for the same action are modelled as different methods
Debugging
You can pause end-to-end tests for debugging by using browser.degugger()
for more, see documentation on Debugging E2E Tests.
Configuring AQUA
Patterns
The AQUA config uses glob patterns similar to those in the various Unix shells to find and filter your files. Glob patterns resemble regular expressions somewhat, but have a much simpler syntax. The following character sequences have special meaning within a glob pattern:
!
negates the pattern?
matches any one character*
matches any number of characters{!glob}
Matches anything that does not match glob{a,b,c}
matches any one of a, b or c[abc]
matches any character in the set a, b or c[^abc]
matches any character not in the set a, b or c[a-z]
matches any character in the range a to z, inclusive. A leading or trailing dash will be interpreted literally
While it's possible to manually include each set of files individually, it's encouraged that you use convention over configuration. For example say you want all your unit tests in a folder called specs, great, then use a wildcard pattern (specs/*.js
) and the correct files will automatically be included as they are added or even renamed over time. All config files paths must be relative to where AQUA is run from.
aqua.config.json
Used to configure AQUA settings that apply to all of your projects
docs
: Required Array; Where to find all source code written for the projectdocs
: Optional Object; all setting related to generating documentation
dir
: Required String; Where to put the generated documentation files for the projectcss
: Optional String; location of a stylesheet file to include into all documentation filesjs
: Optional String; location of a javascript file to include into all documentation filestheme
: Optional String; the name of the docstap theme you want to use
coverage
: Optional Object; all setting related to generating code coverage reports
dir
: Required String; Where to put the generated coverage reports for the project
testing
: Optional Object; all setting related to testing the project
framework
: Required String; Type of testing framework to run the tests inweb
: Optional String; location of the Karma config file used to unit test web projectse2e
: Optional String; location of the protractor config file used to test web projectsnode
: Optional String; location of the config file used to unit test Node.js projects
aqua.project.json
Used to configure AQUA settings that apply to an individual project
id
: Required String; a short but unique id for the projectname
: Required String; a descriptive name of the projectsrc
: Optional Array; Where to find all source code written for the projectby
: Optional String; name and email of the the teams or individuals that worked on the projectreadme
: Optional String; Where to find the markdown file of the project readme (used as the index documentation page)alljs
: Optional Array; Where to find all the JavaScript written for the project, including specs (used for linting)unit
: Optional Object; Everything needed to run unit tests
globals
: Optional Array; Where to find file(s) containing the expected global variables used by your project.
tests
: Required Array; Where to find all of the projects unit testsdeps
: Optional Array; Where to find the scripts the project depends on (such as AngularJS or jQuery)mocks
: Optional Array; Where to find any mocks used by the project unit tests
e2e
: Optional Object; Everything needed to run end-to-end tests
tests
: Required Array; Where to find all of the projects end-to-end testspgobj
: Optional Array; Where to find the page objects used by the project end-to-end tests
types
: Optional Array; Where to find all type definitions used by the projecttype
: Optional String; Used to indicate the type of project (such as Node.js)
Example AQUA Config File
Below are some typical configurations that you can cut-n-paste to jump-start your AQUA setup.
{
"docs": {
"dir": "./docs",
"css": "./css/doc.css",
"js": "./js/doc.js"
},
"coverage": {
"dir": "./test/coverage"
},
"testing": {
"framework": "jasmine",
"web": "./configs/karma.conf.js",
"e2e": "./configs/protractor.conf.js",
"node": "./configs/node.conf.js"
}
}
Example Project Config Files
Below are some typical configurations that you can cut-n-paste to jump-start your project setup.
Web Project
{
"id": "XX",
"name": "YOUR PROJECT NAME",
"by": "TEAM NAME <yourteam@site.com>",
"readme": "./path/to/the/project/markdown/readme.md",
"src" : [
"./any/number/of/paths/to/the/project/sources/files/*.js"
],
"alljs" : [
"./any/number/of/paths/to/the/project/source/specs/e2e/etc/*.js"
],
"unit" : {
"tests": [
"./any/number/of/paths/to/the/project/unit/tests/*.js"
],
"config": "./path/to/the/project/unit/test/karma.conf.js",
"deps": [
"./any/number/of/paths/to/the/project/dependencies/*.js"
],
"mocks": [
"./any/number/of/paths/to/the/unit/test/mocks/*.js"
]
},
"e2e" : {
"tests": [
"./any/number/of/paths/to/the/project/e2e/tests/*.js"
],
"pgobj": [
"./any/number/of/paths/to/the/e2e/test/page/objects/*.js"
]
},
"types": [
"./any/number/of/paths/to/type/definitions/used/by/the/project/*.js"
]
}
Node.js Project
{
"id": "XX",
"name": "YOUR PROJECT NAME",
"by": "TEAM NAME <yourteam@site.com>",
"readme": "./path/to/the/project/markdown/readme.md",
"src": [
"./any/number/of/paths/to/the/project/sources/files/*.js"
],
"type": "nodejs",
"unit": {
"tests": [
"./any/number/of/paths/to/the/project/unit/tests/*.js"
],
"config": "./path/to/the/project/unit/test/jasmine.conf.js",
"mocks": [
"./any/number/of/paths/to/the/unit/test/mocks/*.js"
]
},
"alljs": [
"./any/number/of/paths/to/the/project/source/specs/e2e/etc/*.js"
],
"types": [
"./any/number/of/paths/to/type/definitions/used/by/the/project/*.js"
]
}
License
(The MIT License)
Copyright (c) 2014 Daniel Lamb dlamb.open.source@gmail.com
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
'Software'), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.