metamatcher

metamatcher

Matchers for Jest checking TypeScript code properties via TypeScript and other matchers useful for using tests as a grading tool.

Install

Install with npm:

npm install --save-dev metamatcher

Setup

You need to somehow run the metamatcher module in order to extend expect at runtime. This can be done by either importing the module in your tests (which you will do in order to use locator functions).

Adding the following line to your package config:

"jest": {
  "setupFilesAfterEnv": ["metamatcher/setup"]
}

or the following one in your jest.config.js:

  setupFilesAfterEnv: ["metamatcher/setup"]

does not work... what did I miss?

Usage of TypeScript compiler related matchers

You need to initially load the program somehow. It is recommended to do this in the beforeAll function:

beforeAll(()=>{ loadProgram("src/file.ts") });

In a test you can then use queries for declarations and matchers, e.g.

expect(functionDeclaration("src/someFile.ts", "foo")).toBeRecursive();

Locators

The following locators, i.e. functions returning AST elements or other information, are provided:

• sourceFile(filename: string): ts.SourceFile
• functionDeclaration(sourceFileName: string, functionName: string): ts.FunctionDeclaration
• functionDef(sourceFileName: string, functionName: string): ts.SignatureDeclaration
• returnType(fileName, declName): string
• varDecl(sourceFileName: string, varName: string): ts.VariableDeclaration
  • typeName(varDecl: ts.VariableDeclaration, convertToBaseType = true): string
• allImportModuleSpecifier(sourceFileName: string): string[]
• allExportedNames(sourceFileName: string): string[]
• assignments(func: ts.SignatureDeclaration, lhsString: string): ts.Expression[]

Code and Type Matchers

The following matchers for syntax and convention checks are provided:

• toBeAsync for function
• toBeDirectRecursive for function
• toBeRecursive for function
• toCallFunction for function or whole source file
• toUseArrays for function or whole source file
• toUseAwait for function (nested functions as skipped)
• toUseLoop for function
• toUseRegExp for function
• toContainStringLiteral for function or whole source file
• toContainNumericLiteral for function or whole source file
• toBeType for types, these are declared types!
• toBeBaseType for types, these are declared types!
• toDeclareMember

All these function take an optional argument with a message to be displayed in case of failure.

expect(functionDeclaration("tests/samples.ts", "funcWithRecursion")).toBeRecursive();

Additional Meta-Functions

• collectJestTests: returns all Jests tests of a given module (by mocking test with a collector function and importing the module), even works with each and describe! Note that the module is actually executed (dynamic import, almost as dangerous as eval!). The module is to be specified from the project root folder.

File system

• toBeFileAndExist
• toBeFolderAndExist

Matcher and beforeAll/beforeEach with custom messages

The following matchers are provided for use with custom messages:

• toBe…WithMessage: Similar to toBe…, but with an optional parameter which may be either a string (with a message) or a function producing an message.

Additionally, function

• `` creates a message if the call to the function fails.

The emitted messages are enclosed in §§§ by default. This allows for tools processing the test output to extract the messages. The enclosing tags can be changed via METAMATCHER_MSG_PREFIX and METAMATCHER_MSG_POSTFIX environment variables.

Example:

expect(1).toBeWithMessage(2, (rec,exp) => `Expected ${rec} to be ${exp}`);

This will output a line like this:

§§§Expected 1 to be 2§§§

This allows other tools to extract exactly this message from the test output.

This differs from the solution for custom messages provided by (https://github.com/mattphillips/jest-expect-message)[jest-expect-message]: jest-expect-message can only print out (more or less) constant messages, but the message cannot contain the actual values of the test. This is not sufficient for grading purposes.

Note that toThrow does not work in the async case, since if the promise is rejected, the matcher is not called at all. That is

await expect(asyncCall()).rejects.toThrowWithMessage("message");

will not work. Instead, use

try {
  await asyncCall();
  throw new Error("My message");
} catch (e) {
  // expected
}

in this special case.

runWithFailMessage

If you want to add special messages, in particular for tools analyzing the output later on in your tool chain, you may want to use runWithFailMessage (or short rwfm).

This function encappsulates a call and replaces or modifies the thrown error message.

runWithFailMessage(()=>someFoo(), "Folgefehler");

This will emit §§§Folgefehler§§§ if someFoo() would throw an error.

You can also use a function to generate the message, e.g.

runWithFailMessage(()=>someFoo(), (errMsg) => "Folgefehler: " + errMsg);

This works also with async code:

async function bar(): Promise<string> {…}

const s: string = await rwfm(()=>bar(), (errMsg) => `Folgefehler: ${errMsg}`);

Utils for testing CLIs

In order to test command line interfaces (CLIs), some utility functions are provided:

• findBin: find the bin file defined in a package.json
• callCLI: calls a cli tool and captures output
• callNodeBin: calls a node cli tool and captures output

Development

• Using the TypeScript compiler API: https://github.com/microsoft/TypeScript/wiki/Using-the-Compiler-API#using-the-type-checker
• https://ts-ast-viewer.com/ (for AST)
• https://stackoverflow.com/questions/tagged/typescript-compiler-api?sort=Newest&edited=true

Status

This library is developed on demand. It is not a fully-features code testing library but only contains matchers as far as needed by the author. It is basically made public in order to simplify the author's scripts.

License

This program and the accompanying materials are made available under the terms of the Eclipse Public License v. 2.0 which is available at https://www.eclipse.org/legal/epl-2.0.