Test Reporting

Package ql implements a pure Go embedded SQL database engine. QL is a member of the SQL family of languages. It is less complex and less powerful than SQL (whichever specification SQL is considered to be). 2017-01-10: Release v1.1.0 fixes some bugs and adds a configurable WAL headroom. 2016-07-29: Release v1.0.6 enables alternatively using = instead of == for equality operation. 2016-07-11: Release v1.0.5 undoes vendoring of lldb. QL now uses stable lldb (github.com/cznic/lldb). 2016-07-06: Release v1.0.4 fixes a panic when closing the WAL file. 2016-04-03: Release v1.0.3 fixes a data race. 2016-03-23: Release v1.0.2 vendors github.com/cznic/exp/lldb and github.com/camlistore/go4/lock. 2016-03-17: Release v1.0.1 adjusts for latest goyacc. Parser error messages are improved and changed, but their exact form is not considered a API change. 2016-03-05: The current version has been tagged v1.0.0. 2015-06-15: To improve compatibility with other SQL implementations, the count built-in aggregate function now accepts * as its argument. 2015-05-29: The execution planner was rewritten from scratch. It should use indices in all places where they were used before plus in some additional situations. It is possible to investigate the plan using the newly added EXPLAIN statement. The QL tool is handy for such analysis. If the planner would have used an index, but no such exists, the plan includes hints in form of copy/paste ready CREATE INDEX statements. The planner is still quite simple and a lot of work on it is yet ahead. You can help this process by filling an issue with a schema and query which fails to use an index or indices when it should, in your opinion. Bonus points for including output of `ql 'explain <query>'`. 2015-05-09: The grammar of the CREATE INDEX statement now accepts an expression list instead of a single expression, which was further limited to just a column name or the built-in id(). As a side effect, composite indices are now functional. However, the values in the expression-list style index are not yet used by other statements or the statement/query planner. The composite index is useful while having UNIQUE clause to check for semantically duplicate rows before they get added to the table or when such a row is mutated using the UPDATE statement and the expression-list style index tuple of the row is thus recomputed. 2015-05-02: The Schema field of table __Table now correctly reflects any column constraints and/or defaults. Also, the (*DB).Info method now has that information provided in new ColumInfo fields NotNull, Constraint and Default. 2015-04-20: Added support for {LEFT,RIGHT,FULL} [OUTER] JOIN. 2015-04-18: Column definitions can now have constraints and defaults. Details are discussed in the "Constraints and defaults" chapter below the CREATE TABLE statement documentation. 2015-03-06: New built-in functions formatFloat and formatInt. Thanks urandom! (https://github.com/urandom) 2015-02-16: IN predicate now accepts a SELECT statement. See the updated "Predicates" section. 2015-01-17: Logical operators || and && have now alternative spellings: OR and AND (case insensitive). AND was a keyword before, but OR is a new one. This can possibly break existing queries. For the record, it's a good idea to not use any name appearing in, for example, [7] in your queries as the list of QL's keywords may expand for gaining better compatibility with existing SQL "standards". 2015-01-12: ACID guarantees were tightened at the cost of performance in some cases. The write collecting window mechanism, a formerly used implementation detail, was removed. Inserting rows one by one in a transaction is now slow. I mean very slow. Try to avoid inserting single rows in a transaction. Instead, whenever possible, perform batch updates of tens to, say thousands of rows in a single transaction. See also: http://www.sqlite.org/faq.html#q19, the discussed synchronization principles involved are the same as for QL, modulo minor details. Note: A side effect is that closing a DB before exiting an application, both for the Go API and through database/sql driver, is no more required, strictly speaking. Beware that exiting an application while there is an open (uncommitted) transaction in progress means losing the transaction data. However, the DB will not become corrupted because of not closing it. Nor that was the case before, but formerly failing to close a DB could have resulted in losing the data of the last transaction. 2014-09-21: id() now optionally accepts a single argument - a table name. 2014-09-01: Added the DB.Flush() method and the LIKE pattern matching predicate. 2014-08-08: The built in functions max and min now accept also time values. Thanks opennota! (https://github.com/opennota) 2014-06-05: RecordSet interface extended by new methods FirstRow and Rows. 2014-06-02: Indices on id() are now used by SELECT statements. 2014-05-07: Introduction of Marshal, Schema, Unmarshal. 2014-04-15: Added optional IF NOT EXISTS clause to CREATE INDEX and optional IF EXISTS clause to DROP INDEX. 2014-04-12: The column Unique in the virtual table __Index was renamed to IsUnique because the old name is a keyword. Unfortunately, this is a breaking change, sorry. 2014-04-11: Introduction of LIMIT, OFFSET. 2014-04-10: Introduction of query rewriting. 2014-04-07: Introduction of indices. QL imports zappy[8], a block-based compressor, which speeds up its performance by using a C version of the compression/decompression algorithms. If a CGO-free (pure Go) version of QL, or an app using QL, is required, please include 'purego' in the -tags option of go {build,get,install}. For example: If zappy was installed before installing QL, it might be necessary to rebuild zappy first (or rebuild QL with all its dependencies using the -a option): The syntax is specified using Extended Backus-Naur Form (EBNF) Lower-case production names are used to identify lexical tokens. Non-terminals are in CamelCase. Lexical tokens are enclosed in double quotes "" or back quotes “. The form a … b represents the set of characters from a through b as alternatives. The horizontal ellipsis … is also used elsewhere in the spec to informally denote various enumerations or code snippets that are not further specified. QL source code is Unicode text encoded in UTF-8. The text is not canonicalized, so a single accented code point is distinct from the same character constructed from combining an accent and a letter; those are treated as two code points. For simplicity, this document will use the unqualified term character to refer to a Unicode code point in the source text. Each code point is distinct; for instance, upper and lower case letters are different characters. Implementation restriction: For compatibility with other tools, the parser may disallow the NUL character (U+0000) in the statement. Implementation restriction: A byte order mark is disallowed anywhere in QL statements. The following terms are used to denote specific character classes The underscore character _ (U+005F) is considered a letter. Lexical elements are comments, tokens, identifiers, keywords, operators and delimiters, integer, floating-point, imaginary, rune and string literals and QL parameters. Line comments start with the character sequence // or -- and stop at the end of the line. A line comment acts like a space. General comments start with the character sequence /* and continue through the character sequence */. A general comment acts like a space. Comments do not nest. Tokens form the vocabulary of QL. There are four classes: identifiers, keywords, operators and delimiters, and literals. White space, formed from spaces (U+0020), horizontal tabs (U+0009), carriage returns (U+000D), and newlines (U+000A), is ignored except as it separates tokens that would otherwise combine into a single token. The formal grammar uses semicolons ";" as separators of QL statements. A single QL statement or the last QL statement in a list of statements can have an optional semicolon terminator. (Actually a separator from the following empty statement.) Identifiers name entities such as tables or record set columns. An identifier is a sequence of one or more letters and digits. The first character in an identifier must be a letter. For example No identifiers are predeclared, however note that no keyword can be used as an identifier. Identifiers starting with two underscores are used for meta data virtual tables names. For forward compatibility, users should generally avoid using any identifiers starting with two underscores. For example The following keywords are reserved and may not be used as identifiers. Keywords are not case sensitive. The following character sequences represent operators, delimiters, and other special tokens Operators consisting of more than one character are referred to by names in the rest of the documentation An integer literal is a sequence of digits representing an integer constant. An optional prefix sets a non-decimal base: 0 for octal, 0x or 0X for hexadecimal. In hexadecimal literals, letters a-f and A-F represent values 10 through 15. For example A floating-point literal is a decimal representation of a floating-point constant. It has an integer part, a decimal point, a fractional part, and an exponent part. The integer and fractional part comprise decimal digits; the exponent part is an e or E followed by an optionally signed decimal exponent. One of the integer part or the fractional part may be elided; one of the decimal point or the exponent may be elided. For example An imaginary literal is a decimal representation of the imaginary part of a complex constant. It consists of a floating-point literal or decimal integer followed by the lower-case letter i. For example A rune literal represents a rune constant, an integer value identifying a Unicode code point. A rune literal is expressed as one or more characters enclosed in single quotes. Within the quotes, any character may appear except single quote and newline. A single quoted character represents the Unicode value of the character itself, while multi-character sequences beginning with a backslash encode values in various formats. The simplest form represents the single character within the quotes; since QL statements are Unicode characters encoded in UTF-8, multiple UTF-8-encoded bytes may represent a single integer value. For instance, the literal 'a' holds a single byte representing a literal a, Unicode U+0061, value 0x61, while 'ä' holds two bytes (0xc3 0xa4) representing a literal a-dieresis, U+00E4, value 0xe4. Several backslash escapes allow arbitrary values to be encoded as ASCII text. There are four ways to represent the integer value as a numeric constant: \x followed by exactly two hexadecimal digits; \u followed by exactly four hexadecimal digits; \U followed by exactly eight hexadecimal digits, and a plain backslash \ followed by exactly three octal digits. In each case the value of the literal is the value represented by the digits in the corresponding base. Although these representations all result in an integer, they have different valid ranges. Octal escapes must represent a value between 0 and 255 inclusive. Hexadecimal escapes satisfy this condition by construction. The escapes \u and \U represent Unicode code points so within them some values are illegal, in particular those above 0x10FFFF and surrogate halves. After a backslash, certain single-character escapes represent special values All other sequences starting with a backslash are illegal inside rune literals. For example A string literal represents a string constant obtained from concatenating a sequence of characters. There are two forms: raw string literals and interpreted string literals. Raw string literals are character sequences between back quotes “. Within the quotes, any character is legal except back quote. The value of a raw string literal is the string composed of the uninterpreted (implicitly UTF-8-encoded) characters between the quotes; in particular, backslashes have no special meaning and the string may contain newlines. Carriage returns inside raw string literals are discarded from the raw string value. Interpreted string literals are character sequences between double quotes "". The text between the quotes, which may not contain newlines, forms the value of the literal, with backslash escapes interpreted as they are in rune literals (except that \' is illegal and \" is legal), with the same restrictions. The three-digit octal (\nnn) and two-digit hexadecimal (\xnn) escapes represent individual bytes of the resulting string; all other escapes represent the (possibly multi-byte) UTF-8 encoding of individual characters. Thus inside a string literal \377 and \xFF represent a single byte of value 0xFF=255, while ÿ, \u00FF, \U000000FF and \xc3\xbf represent the two bytes 0xc3 0xbf of the UTF-8 encoding of character U+00FF. For example These examples all represent the same string If the statement source represents a character as two code points, such as a combining form involving an accent and a letter, the result will be an error if placed in a rune literal (it is not a single code point), and will appear as two code points if placed in a string literal. Literals are assigned their values from the respective text representation at "compile" (parse) time. QL parameters provide the same functionality as literals, but their value is assigned at execution time from an expression list passed to DB.Run or DB.Execute. Using '?' or '$' is completely equivalent. For example Keywords 'false' and 'true' (not case sensitive) represent the two possible constant values of type bool (also not case sensitive). Keyword 'NULL' (not case sensitive) represents an untyped constant which is assignable to any type. NULL is distinct from any other value of any type. A type determines the set of values and operations specific to values of that type. A type is specified by a type name. Named instances of the boolean, numeric, and string types are keywords. The names are not case sensitive. Note: The blob type is exchanged between the back end and the API as []byte. On 32 bit platforms this limits the size which the implementation can handle to 2G. A boolean type represents the set of Boolean truth values denoted by the predeclared constants true and false. The predeclared boolean type is bool. A duration type represents the elapsed time between two instants as an int64 nanosecond count. The representation limits the largest representable duration to approximately 290 years. A numeric type represents sets of integer or floating-point values. The predeclared architecture-independent numeric types are The value of an n-bit integer is n bits wide and represented using two's complement arithmetic. Conversions are required when different numeric types are mixed in an expression or assignment. A string type represents the set of string values. A string value is a (possibly empty) sequence of bytes. The case insensitive keyword for the string type is 'string'. The length of a string (its size in bytes) can be discovered using the built-in function len. A time type represents an instant in time with nanosecond precision. Each time has associated with it a location, consulted when computing the presentation form of the time. The following functions are implicitly declared An expression specifies the computation of a value by applying operators and functions to operands. Operands denote the elementary values in an expression. An operand may be a literal, a (possibly qualified) identifier denoting a constant or a function or a table/record set column, or a parenthesized expression. A qualified identifier is an identifier qualified with a table/record set name prefix. For example Primary expression are the operands for unary and binary expressions. For example A primary expression of the form denotes the element of a string indexed by x. Its type is byte. The value x is called the index. The following rules apply - The index x must be of integer type except bigint or duration; it is in range if 0 <= x < len(s), otherwise it is out of range. - A constant index must be non-negative and representable by a value of type int. - A constant index must be in range if the string a is a literal. - If x is out of range at run time, a run-time error occurs. - s[x] is the byte at index x and the type of s[x] is byte. If s is NULL or x is NULL then the result is NULL. Otherwise s[x] is illegal. For a string, the primary expression constructs a substring. The indices low and high select which elements appear in the result. The result has indices starting at 0 and length equal to high - low. For convenience, any of the indices may be omitted. A missing low index defaults to zero; a missing high index defaults to the length of the sliced operand The indices low and high are in range if 0 <= low <= high <= len(a), otherwise they are out of range. A constant index must be non-negative and representable by a value of type int. If both indices are constant, they must satisfy low <= high. If the indices are out of range at run time, a run-time error occurs. Integer values of type bigint or duration cannot be used as indices. If s is NULL the result is NULL. If low or high is not omitted and is NULL then the result is NULL. Given an identifier f denoting a predeclared function, calls f with arguments a1, a2, … an. Arguments are evaluated before the function is called. The type of the expression is the result type of f. In a function call, the function value and arguments are evaluated in the usual order. After they are evaluated, the parameters of the call are passed by value to the function and the called function begins execution. The return value of the function is passed by value when the function returns. Calling an undefined function causes a compile-time error. Operators combine operands into expressions. Comparisons are discussed elsewhere. For other binary operators, the operand types must be identical unless the operation involves shifts or untyped constants. For operations involving constants only, see the section on constant expressions. Except for shift operations, if one operand is an untyped constant and the other operand is not, the constant is converted to the type of the other operand. The right operand in a shift expression must have unsigned integer type or be an untyped constant that can be converted to unsigned integer type. If the left operand of a non-constant shift expression is an untyped constant, the type of the constant is what it would be if the shift expression were replaced by its left operand alone. Expressions of the form yield a boolean value true if expr2, a regular expression, matches expr1 (see also [6]). Both expression must be of type string. If any one of the expressions is NULL the result is NULL. Predicates are special form expressions having a boolean result type. Expressions of the form are equivalent, including NULL handling, to The types of involved expressions must be comparable as defined in "Comparison operators". Another form of the IN predicate creates the expression list from a result of a SelectStmt. The SelectStmt must select only one column. The produced expression list is resource limited by the memory available to the process. NULL values produced by the SelectStmt are ignored, but if all records of the SelectStmt are NULL the predicate yields NULL. The select statement is evaluated only once. If the type of expr is not the same as the type of the field returned by the SelectStmt then the set operation yields false. The type of the column returned by the SelectStmt must be one of the simple (non blob-like) types: Expressions of the form are equivalent, including NULL handling, to The types of involved expressions must be ordered as defined in "Comparison operators". Expressions of the form yield a boolean value true if expr does not have a specific type (case A) or if expr has a specific type (case B). In other cases the result is a boolean value false. Unary operators have the highest precedence. There are five precedence levels for binary operators. Multiplication operators bind strongest, followed by addition operators, comparison operators, && (logical AND), and finally || (logical OR) Binary operators of the same precedence associate from left to right. For instance, x / y * z is the same as (x / y) * z. Note that the operator precedence is reflected explicitly by the grammar. Arithmetic operators apply to numeric values and yield a result of the same type as the first operand. The four standard arithmetic operators (+, -, *, /) apply to integer, rational, floating-point, and complex types; + also applies to strings; +,- also applies to times. All other arithmetic operators apply to integers only. sum integers, rationals, floats, complex values, strings difference integers, rationals, floats, complex values, times product integers, rationals, floats, complex values / quotient integers, rationals, floats, complex values % remainder integers & bitwise AND integers | bitwise OR integers ^ bitwise XOR integers &^ bit clear (AND NOT) integers << left shift integer << unsigned integer >> right shift integer >> unsigned integer Strings can be concatenated using the + operator String addition creates a new string by concatenating the operands. A value of type duration can be added to or subtracted from a value of type time. Times can subtracted from each other producing a value of type duration. For two integer values x and y, the integer quotient q = x / y and remainder r = x % y satisfy the following relationships with x / y truncated towards zero ("truncated division"). As an exception to this rule, if the dividend x is the most negative value for the int type of x, the quotient q = x / -1 is equal to x (and r = 0). If the divisor is a constant expression, it must not be zero. If the divisor is zero at run time, a run-time error occurs. If the dividend is non-negative and the divisor is a constant power of 2, the division may be replaced by a right shift, and computing the remainder may be replaced by a bitwise AND operation The shift operators shift the left operand by the shift count specified by the right operand. They implement arithmetic shifts if the left operand is a signed integer and logical shifts if it is an unsigned integer. There is no upper limit on the shift count. Shifts behave as if the left operand is shifted n times by 1 for a shift count of n. As a result, x << 1 is the same as x*2 and x >> 1 is the same as x/2 but truncated towards negative infinity. For integer operands, the unary operators +, -, and ^ are defined as follows For floating-point and complex numbers, +x is the same as x, while -x is the negation of x. The result of a floating-point or complex division by zero is not specified beyond the IEEE-754 standard; whether a run-time error occurs is implementation-specific. Whenever any operand of any arithmetic operation, unary or binary, is NULL, as well as in the case of the string concatenating operation, the result is NULL. For unsigned integer values, the operations +, -, *, and << are computed modulo 2n, where n is the bit width of the unsigned integer's type. Loosely speaking, these unsigned integer operations discard high bits upon overflow, and expressions may rely on “wrap around”. For signed integers with a finite bit width, the operations +, -, *, and << may legally overflow and the resulting value exists and is deterministically defined by the signed integer representation, the operation, and its operands. No exception is raised as a result of overflow. An evaluator may not optimize an expression under the assumption that overflow does not occur. For instance, it may not assume that x < x + 1 is always true. Integers of type bigint and rationals do not overflow but their handling is limited by the memory resources available to the program. Comparison operators compare two operands and yield a boolean value. In any comparison, the first operand must be of same type as is the second operand, or vice versa. The equality operators == and != apply to operands that are comparable. The ordering operators <, <=, >, and >= apply to operands that are ordered. These terms and the result of the comparisons are defined as follows - Boolean values are comparable. Two boolean values are equal if they are either both true or both false. - Complex values are comparable. Two complex values u and v are equal if both real(u) == real(v) and imag(u) == imag(v). - Integer values are comparable and ordered, in the usual way. Note that durations are integers. - Floating point values are comparable and ordered, as defined by the IEEE-754 standard. - Rational values are comparable and ordered, in the usual way. - String values are comparable and ordered, lexically byte-wise. - Time values are comparable and ordered. Whenever any operand of any comparison operation is NULL, the result is NULL. Note that slices are always of type string. Logical operators apply to boolean values and yield a boolean result. The right operand is evaluated conditionally. The truth tables for logical operations with NULL values Conversions are expressions of the form T(x) where T is a type and x is an expression that can be converted to type T. A constant value x can be converted to type T in any of these cases: - x is representable by a value of type T. - x is a floating-point constant, T is a floating-point type, and x is representable by a value of type T after rounding using IEEE 754 round-to-even rules. The constant T(x) is the rounded value. - x is an integer constant and T is a string type. The same rule as for non-constant x applies in this case. Converting a constant yields a typed constant as result. A non-constant value x can be converted to type T in any of these cases: - x has type T. - x's type and T are both integer or floating point types. - x's type and T are both complex types. - x is an integer, except bigint or duration, and T is a string type. Specific rules apply to (non-constant) conversions between numeric types or to and from a string type. These conversions may change the representation of x and incur a run-time cost. All other conversions only change the type but not the representation of x. A conversion of NULL to any type yields NULL. For the conversion of non-constant numeric values, the following rules apply 1. When converting between integer types, if the value is a signed integer, it is sign extended to implicit infinite precision; otherwise it is zero extended. It is then truncated to fit in the result type's size. For example, if v == uint16(0x10F0), then uint32(int8(v)) == 0xFFFFFFF0. The conversion always yields a valid value; there is no indication of overflow. 2. When converting a floating-point number to an integer, the fraction is discarded (truncation towards zero). 3. When converting an integer or floating-point number to a floating-point type, or a complex number to another complex type, the result value is rounded to the precision specified by the destination type. For instance, the value of a variable x of type float32 may be stored using additional precision beyond that of an IEEE-754 32-bit number, but float32(x) represents the result of rounding x's value to 32-bit precision. Similarly, x + 0.1 may use more than 32 bits of precision, but float32(x + 0.1) does not. In all non-constant conversions involving floating-point or complex values, if the result type cannot represent the value the conversion succeeds but the result value is implementation-dependent. 1. Converting a signed or unsigned integer value to a string type yields a string containing the UTF-8 representation of the integer. Values outside the range of valid Unicode code points are converted to "\uFFFD". 2. Converting a blob to a string type yields a string whose successive bytes are the elements of the blob. 3. Converting a value of a string type to a blob yields a blob whose successive elements are the bytes of the string. 4. Converting a value of a bigint type to a string yields a string containing the decimal decimal representation of the integer. 5. Converting a value of a string type to a bigint yields a bigint value containing the integer represented by the string value. A prefix of “0x” or “0X” selects base 16; the “0” prefix selects base 8, and a “0b” or “0B” prefix selects base 2. Otherwise the value is interpreted in base 10. An error occurs if the string value is not in any valid format. 6. Converting a value of a rational type to a string yields a string containing the decimal decimal representation of the rational in the form "a/b" (even if b == 1). 7. Converting a value of a string type to a bigrat yields a bigrat value containing the rational represented by the string value. The string can be given as a fraction "a/b" or as a floating-point number optionally followed by an exponent. An error occurs if the string value is not in any valid format. 8. Converting a value of a duration type to a string returns a string representing the duration in the form "72h3m0.5s". Leading zero units are omitted. As a special case, durations less than one second format using a smaller unit (milli-, micro-, or nanoseconds) to ensure that the leading digit is non-zero. The zero duration formats as 0, with no unit. 9. Converting a string value to a duration yields a duration represented by the string. A duration string is a possibly signed sequence of decimal numbers, each with optional fraction and a unit suffix, such as "300ms", "-1.5h" or "2h45m". Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". 10. Converting a time value to a string returns the time formatted using the format string When evaluating the operands of an expression or of function calls, operations are evaluated in lexical left-to-right order. For example, in the evaluation of the function calls and evaluation of c happen in the order h(), i(), j(), c. Floating-point operations within a single expression are evaluated according to the associativity of the operators. Explicit parentheses affect the evaluation by overriding the default associativity. In the expression x + (y + z) the addition y + z is performed before adding x. Statements control execution. The empty statement does nothing. Alter table statements modify existing tables. With the ADD clause it adds a new column to the table. The column must not exist. With the DROP clause it removes an existing column from a table. The column must exist and it must be not the only (last) column of the table. IOW, there cannot be a table with no columns. For example When adding a column to a table with existing data, the constraint clause of the ColumnDef cannot be used. Adding a constrained column to an empty table is fine. Begin transactions statements introduce a new transaction level. Every transaction level must be eventually balanced by exactly one of COMMIT or ROLLBACK statements. Note that when a transaction is roll-backed because of a statement failure then no explicit balancing of the respective BEGIN TRANSACTION is statement is required nor permitted. Failure to properly balance any opened transaction level may cause dead locks and/or lose of data updated in the uppermost opened but never properly closed transaction level. For example A database cannot be updated (mutated) outside of a transaction. Statements requiring a transaction A database is effectively read only outside of a transaction. Statements not requiring a transaction The commit statement closes the innermost transaction nesting level. If that's the outermost level then the updates to the DB made by the transaction are atomically made persistent. For example Create index statements create new indices. Index is a named projection of ordered values of a table column to the respective records. As a special case the id() of the record can be indexed. Index name must not be the same as any of the existing tables and it also cannot be the same as of any column name of the table the index is on. For example Now certain SELECT statements may use the indices to speed up joins and/or to speed up record set filtering when the WHERE clause is used; or the indices might be used to improve the performance when the ORDER BY clause is present. The UNIQUE modifier requires the indexed values tuple to be index-wise unique or have all values NULL. The optional IF NOT EXISTS clause makes the statement a no operation if the index already exists. A simple index consists of only one expression which must be either a column name or the built-in id(). A more complex and more general index is one that consists of more than one expression or its single expression does not qualify as a simple index. In this case the type of all expressions in the list must be one of the non blob-like types. Note: Blob-like types are blob, bigint, bigrat, time and duration. Create table statements create new tables. A column definition declares the column name and type. Table names and column names are case sensitive. Neither a table or an index of the same name may exist in the DB. For example The optional IF NOT EXISTS clause makes the statement a no operation if the table already exists. The optional constraint clause has two forms. The first one is found in many SQL dialects. This form prevents the data in column DepartmentName to be NULL. The second form allows an arbitrary boolean expression to be used to validate the column. If the value of the expression is true then the validation succeeded. If the value of the expression is false or NULL then the validation fails. If the value of the expression is not of type bool an error occurs. The optional DEFAULT clause is an expression which, if present, is substituted instead of a NULL value when the colum is assigned a value. Note that the constraint and/or default expressions may refer to other columns by name: When a table row is inserted by the INSERT INTO statement or when a table row is updated by the UPDATE statement, the order of operations is as follows: 1. The new values of the affected columns are set and the values of all the row columns become the named values which can be referred to in default expressions evaluated in step 2. 2. If any row column value is NULL and the DEFAULT clause is present in the column's definition, the default expression is evaluated and its value is set as the respective column value. 3. The values, potentially updated, of row columns become the named values which can be referred to in constraint expressions evaluated during step 4. 4. All row columns which definition has the constraint clause present will have that constraint checked. If any constraint violation is detected, the overall operation fails and no changes to the table are made. Delete from statements remove rows from a table, which must exist. For example If the WHERE clause is not present then all rows are removed and the statement is equivalent to the TRUNCATE TABLE statement. Drop index statements remove indices from the DB. The index must exist. For example The optional IF EXISTS clause makes the statement a no operation if the index does not exist. Drop table statements remove tables from the DB. The table must exist. For example The optional IF EXISTS clause makes the statement a no operation if the table does not exist. Insert into statements insert new rows into tables. New rows come from literal data, if using the VALUES clause, or are a result of select statement. In the later case the select statement is fully evaluated before the insertion of any rows is performed, allowing to insert values calculated from the same table rows are to be inserted into. If the ColumnNameList part is omitted then the number of values inserted in the row must be the same as are columns in the table. If the ColumnNameList part is present then the number of values per row must be same as the same number of column names. All other columns of the record are set to NULL. The type of the value assigned to a column must be the same as is the column's type or the value must be NULL. For example If any of the columns of the table were defined using the optional constraints clause or the optional defaults clause then those are processed on a per row basis. The details are discussed in the "Constraints and defaults" chapter below the CREATE TABLE statement documentation. Explain statement produces a recordset consisting of lines of text which describe the execution plan of a statement, if any. For example, the QL tool treats the explain statement specially and outputs the joined lines: The explanation may aid in uderstanding how a statement/query would be executed and if indices are used as expected - or which indices may possibly improve the statement performance. The create index statements above were directly copy/pasted in the terminal from the suggestions provided by the filter recordset pipeline part returned by the explain statement. If the statement has nothing special in its plan, the result is the original statement. To get an explanation of the select statement of the IN predicate, use the EXPLAIN statement with that particular select statement. The rollback statement closes the innermost transaction nesting level discarding any updates to the DB made by it. If that's the outermost level then the effects on the DB are as if the transaction never happened. For example The (temporary) record set from the last statement is returned and can be processed by the client. In this case the rollback is the same as 'DROP TABLE tmp;' but it can be a more complex operation. Select from statements produce recordsets. The optional DISTINCT modifier ensures all rows in the result recordset are unique. Either all of the resulting fields are returned ('*') or only those named in FieldList. RecordSetList is a list of table names or parenthesized select statements, optionally (re)named using the AS clause. The result can be filtered using a WhereClause and orderd by the OrderBy clause. For example If Recordset is a nested, parenthesized SelectStmt then it must be given a name using the AS clause if its field are to be accessible in expressions. A field is an named expression. Identifiers, not used as a type in conversion or a function name in the Call clause, denote names of (other) fields, values of which should be used in the expression. The expression can be named using the AS clause. If the AS clause is not present and the expression consists solely of a field name, then that field name is used as the name of the resulting field. Otherwise the field is unnamed. For example The SELECT statement can optionally enumerate the desired/resulting fields in a list. No two identical field names can appear in the list. When more than one record set is used in the FROM clause record set list, the result record set field names are rewritten to be qualified using the record set names. If a particular record set doesn't have a name, its respective fields became unnamed. The optional JOIN clause, for example is mostly equal to except that the rows from a which, when they appear in the cross join, never made expr to evaluate to true, are combined with a virtual row from b, containing all nulls, and added to the result set. For the RIGHT JOIN variant the discussed rules are used for rows from b not satisfying expr == true and the virtual, all-null row "comes" from a. The FULL JOIN adds the respective rows which would be otherwise provided by the separate executions of the LEFT JOIN and RIGHT JOIN variants. For more thorough OUTER JOIN discussion please see the Wikipedia article at [10]. Resultins rows of a SELECT statement can be optionally ordered by the ORDER BY clause. Collating proceeds by considering the expressions in the expression list left to right until a collating order is determined. Any possibly remaining expressions are not evaluated. All of the expression values must yield an ordered type or NULL. Ordered types are defined in "Comparison operators". Collating of elements having a NULL value is different compared to what the comparison operators yield in expression evaluation (NULL result instead of a boolean value). Below, T denotes a non NULL value of any QL type. NULL collates before any non NULL value (is considered smaller than T). Two NULLs have no collating order (are considered equal). The WHERE clause restricts records considered by some statements, like SELECT FROM, DELETE FROM, or UPDATE. It is an error if the expression evaluates to a non null value of non bool type. The GROUP BY clause is used to project rows having common values into a smaller set of rows. For example Using the GROUP BY without any aggregate functions in the selected fields is in certain cases equal to using the DISTINCT modifier. The last two examples above produce the same resultsets. The optional OFFSET clause allows to ignore first N records. For example The above will produce only rows 11, 12, ... of the record set, if they exist. The value of the expression must a non negative integer, but not bigint or duration. The optional LIMIT clause allows to ignore all but first N records. For example The above will return at most the first 10 records of the record set. The value of the expression must a non negative integer, but not bigint or duration. The LIMIT and OFFSET clauses can be combined. For example Considering table t has, say 10 records, the above will produce only records 4 - 8. After returning record #8, no more result rows/records are computed. 1. The FROM clause is evaluated, producing a Cartesian product of its source record sets (tables or nested SELECT statements). 2. If present, the JOIN cluase is evaluated on the result set of the previous evaluation and the recordset specified by the JOIN clause. (... JOIN Recordset ON ...) 3. If present, the WHERE clause is evaluated on the result set of the previous evaluation. 4. If present, the GROUP BY clause is evaluated on the result set of the previous evaluation(s). 5. The SELECT field expressions are evaluated on the result set of the previous evaluation(s). 6. If present, the DISTINCT modifier is evaluated on the result set of the previous evaluation(s). 7. If present, the ORDER BY clause is evaluated on the result set of the previous evaluation(s). 8. If present, the OFFSET clause is evaluated on the result set of the previous evaluation(s). The offset expression is evaluated once for the first record produced by the previous evaluations. 9. If present, the LIMIT clause is evaluated on the result set of the previous evaluation(s). The limit expression is evaluated once for the first record produced by the previous evaluations. Truncate table statements remove all records from a table. The table must exist. For example Update statements change values of fields in rows of a table. For example Note: The SET clause is optional. If any of the columns of the table were defined using the optional constraints clause or the optional defaults clause then those are processed on a per row basis. The details are discussed in the "Constraints and defaults" chapter below the CREATE TABLE statement documentation. To allow to query for DB meta data, there exist specially named tables, some of them being virtual. Note: Virtual system tables may have fake table-wise unique but meaningless and unstable record IDs. Do not apply the built-in id() to any system table. The table __Table lists all tables in the DB. The schema is The Schema column returns the statement to (re)create table Name. This table is virtual. The table __Colum lists all columns of all tables in the DB. The schema is The Ordinal column defines the 1-based index of the column in the record. This table is virtual. The table __Colum2 lists all columns of all tables in the DB which have the constraint NOT NULL or which have a constraint expression defined or which have a default expression defined. The schema is It's possible to obtain a consolidated recordset for all properties of all DB columns using The Name column is the column name in TableName. The table __Index lists all indices in the DB. The schema is The IsUnique columns reflects if the index was created using the optional UNIQUE clause. This table is virtual. Built-in functions are predeclared. The built-in aggregate function avg returns the average of values of an expression. Avg ignores NULL values, but returns NULL if all values of a column are NULL or if avg is applied to an empty record set. The column values must be of a numeric type. The built-in function contains returns true if substr is within s. If any argument to contains is NULL the result is NULL. The built-in aggregate function count returns how many times an expression has a non NULL values or the number of rows in a record set. Note: count() returns 0 for an empty record set. For example Date returns the time corresponding to in the appropriate zone for that time in the given location. The month, day, hour, min, sec, and nsec values may be outside their usual ranges and will be normalized during the conversion. For example, October 32 converts to November 1. A daylight savings time transition skips or repeats times. For example, in the United States, March 13, 2011 2:15am never occurred, while November 6, 2011 1:15am occurred twice. In such cases, the choice of time zone, and therefore the time, is not well-defined. Date returns a time that is correct in one of the two zones involved in the transition, but it does not guarantee which. A location maps time instants to the zone in use at that time. Typically, the location represents the collection of time offsets in use in a geographical area, such as "CEST" and "CET" for central Europe. "local" represents the system's local time zone. "UTC" represents Universal Coordinated Time (UTC). The month specifies a month of the year (January = 1, ...). If any argument to date is NULL the result is NULL. The built-in function day returns the day of the month specified by t. If the argument to day is NULL the result is NULL. The built-in function formatTime returns a textual representation of the time value formatted according to layout, which defines the format by showing how the reference time, would be displayed if it were the value; it serves as an example of the desired output. The same display rules will then be applied to the time value. If any argument to formatTime is NULL the result is NULL. NOTE: The string value of the time zone, like "CET" or "ACDT", is dependent on the time zone of the machine the function is run on. For example, if the t value is in "CET", but the machine is in "ACDT", instead of "CET" the result is "+0100". This is the same what Go (time.Time).String() returns and in fact formatTime directly calls t.String(). returns on a machine in the CET time zone, but may return on a machine in the ACDT zone. The time value is in both cases the same so its ordering and comparing is correct. Only the display value can differ. The built-in functions formatFloat and formatInt format numbers to strings using go's number format functions in the `strconv` package. For all three functions, only the first argument is mandatory. The default values of the rest are shown in the examples. If the first argument is NULL, the result is NULL. returns returns returns Unlike the `strconv` equivalent, the formatInt function handles all integer types, both signed and unsigned. The built-in function hasPrefix tests whether the string s begins with prefix. If any argument to hasPrefix is NULL the result is NULL. The built-in function hasSuffix tests whether the string s ends with suffix. If any argument to hasSuffix is NULL the result is NULL. The built-in function hour returns the hour within the day specified by t, in the range [0, 23]. If the argument to hour is NULL the result is NULL. The built-in function hours returns the duration as a floating point number of hours. If the argument to hours is NULL the result is NULL. The built-in function id takes zero or one arguments. If no argument is provided, id() returns a table-unique automatically assigned numeric identifier of type int. Ids of deleted records are not reused unless the DB becomes completely empty (has no tables). For example If id() without arguments is called for a row which is not a table record then the result value is NULL. For example If id() has one argument it must be a table name of a table in a cross join. For example The built-in function len takes a string argument and returns the lentgh of the string in bytes. The expression len(s) is constant if s is a string constant. If the argument to len is NULL the result is NULL. The built-in aggregate function max returns the largest value of an expression in a record set. Max ignores NULL values, but returns NULL if all values of a column are NULL or if max is applied to an empty record set. The expression values must be of an ordered type. For example The built-in aggregate function min returns the smallest value of an expression in a record set. Min ignores NULL values, but returns NULL if all values of a column are NULL or if min is applied to an empty record set. For example The column values must be of an ordered type. The built-in function minute returns the minute offset within the hour specified by t, in the range [0, 59]. If the argument to minute is NULL the result is NULL. The built-in function minutes returns the duration as a floating point number of minutes. If the argument to minutes is NULL the result is NULL. The built-in function month returns the month of the year specified by t (January = 1, ...). If the argument to month is NULL the result is NULL. The built-in function nanosecond returns the nanosecond offset within the second specified by t, in the range [0, 999999999]. If the argument to nanosecond is NULL the result is NULL. The built-in function nanoseconds returns the duration as an integer nanosecond count. If the argument to nanoseconds is NULL the result is NULL. The built-in function now returns the current local time. The built-in function parseTime parses a formatted string and returns the time value it represents. The layout defines the format by showing how the reference time, would be interpreted if it were the value; it serves as an example of the input format. The same interpretation will then be made to the input string. Elements omitted from the value are assumed to be zero or, when zero is impossible, one, so parsing "3:04pm" returns the time corresponding to Jan 1, year 0, 15:04:00 UTC (note that because the year is 0, this time is before the zero Time). Years must be in the range 0000..9999. The day of the week is checked for syntax but it is otherwise ignored. In the absence of a time zone indicator, parseTime returns a time in UTC. When parsing a time with a zone offset like -0700, if the offset corresponds to a time zone used by the current location, then parseTime uses that location and zone in the returned time. Otherwise it records the time as being in a fabricated location with time fixed at the given zone offset. When parsing a time with a zone abbreviation like MST, if the zone abbreviation has a defined offset in the current location, then that offset is used. The zone abbreviation "UTC" is recognized as UTC regardless of location. If the zone abbreviation is unknown, Parse records the time as being in a fabricated location with the given zone abbreviation and a zero offset. This choice means that such a time can be parses and reformatted with the same layout losslessly, but the exact instant used in the representation will differ by the actual zone offset. To avoid such problems, prefer time layouts that use a numeric zone offset. If any argument to parseTime is NULL the result is NULL. The built-in function second returns the second offset within the minute specified by t, in the range [0, 59]. If the argument to second is NULL the result is NULL. The built-in function seconds returns the duration as a floating point number of seconds. If the argument to seconds is NULL the result is NULL. The built-in function since returns the time elapsed since t. It is shorthand for now()-t. If the argument to since is NULL the result is NULL. The built-in aggregate function sum returns the sum of values of an expression for all rows of a record set. Sum ignores NULL values, but returns NULL if all values of a column are NULL or if sum is applied to an empty record set. The column values must be of a numeric type. The built-in function timeIn returns t with the location information set to loc. For discussion of the loc argument please see date(). If any argument to timeIn is NULL the result is NULL. The built-in function weekday returns the day of the week specified by t. Sunday == 0, Monday == 1, ... If the argument to weekday is NULL the result is NULL. The built-in function year returns the year in which t occurs. If the argument to year is NULL the result is NULL. The built-in function yearDay returns the day of the year specified by t, in the range [1,365] for non-leap years, and [1,366] in leap years. If the argument to yearDay is NULL the result is NULL. Three functions assemble and disassemble complex numbers. The built-in function complex constructs a complex value from a floating-point real and imaginary part, while real and imag extract the real and imaginary parts of a complex value. The type of the arguments and return value correspond. For complex, the two arguments must be of the same floating-point type and the return type is the complex type with the corresponding floating-point constituents: complex64 for float32, complex128 for float64. The real and imag functions together form the inverse, so for a complex value z, z == complex(real(z), imag(z)). If the operands of these functions are all constants, the return value is a constant. If any argument to any of complex, real, imag functions is NULL the result is NULL. For the numeric types, the following sizes are guaranteed Portions of this specification page are modifications based on work[2] created and shared by Google[3] and used according to terms described in the Creative Commons 3.0 Attribution License[4]. This specification is licensed under the Creative Commons Attribution 3.0 License, and code is licensed under a BSD license[5]. Links from the above documentation This section is not part of the specification. WARNING: The implementation of indices is new and it surely needs more time to become mature. Indices are used currently used only by the WHERE clause. The following expression patterns of 'WHERE expression' are recognized and trigger index use. The relOp is one of the relation operators <, <=, ==, >=, >. For the equality operator both operands must be of comparable types. For all other operators both operands must be of ordered types. The constant expression is a compile time constant expression. Some constant folding is still a TODO. Parameter is a QL parameter ($1 etc.). Consider tables t and u, both with an indexed field f. The WHERE expression doesn't comply with the above simple detected cases. However, such query is now automatically rewritten to which will use both of the indices. The impact of using the indices can be substantial (cf. BenchmarkCrossJoin*) if the resulting rows have low "selectivity", ie. only few rows from both tables are selected by the respective WHERE filtering. Note: Existing QL DBs can be used and indices can be added to them. However, once any indices are present in the DB, the old QL versions cannot work with such DB anymore. Running a benchmark with -v (-test.v) outputs information about the scale used to report records/s and a brief description of the benchmark. For example Running the full suite of benchmarks takes a lot of time. Use the -timeout flag to avoid them being killed after the default time limit (10 minutes).

Pact Go enables consumer driven contract testing, providing a mock service and DSL for the consumer project, and interaction playback and verification for the service provider project. Consumer side Pact testing is an isolated test that ensures a given component is able to collaborate with another (remote) component. Pact will automatically start a Mock server in the background that will act as the collaborators' test double. This implies that any interactions expected on the Mock server will be validated, meaning a test will fail if all interactions were not completed, or if unexpected interactions were found: A typical consumer-side test would look something like this: If this test completed successfully, a Pact file should have been written to ./pacts/my_consumer-my_provider.json containing all of the interactions expected to occur between the Consumer and Provider. In addition to verbatim value matching, you have 3 useful matching functions in the `dsl` package that can increase expressiveness and reduce brittle test cases. Here is a complex example that shows how all 3 terms can be used together: colour := Term("red", "red|green|blue") match := EachLike( This example will result in a response body from the mock server that looks like: See the examples in the dsl package and the matcher tests (https://github.com/pact-foundation/pact-go/blob/master/dsl/matcher_test.go) for more matching examples. NOTE: You will need to use valid Ruby regular expressions (http://ruby-doc.org/core-2.1.5/Regexp.html) and double escape backslashes. Read more about flexible matching (https://github.com/pact-foundation/pact-ruby/wiki/Regular-expressions-and-type-matching-with-Pact. Provider side Pact testing, involves verifying that the contract - the Pact file - can be satisfied by the Provider. A typical Provider side test would like something like: The `VerifyProvider` will handle all verifications, treating them as subtests and giving you granular test reporting. If you don't like this behaviour, you may call `VerifyProviderRaw` directly and handle the errors manually. Note that `PactURLs` may be a list of local pact files or remote based urls (possibly from a Pact Broker - http://docs.pact.io/documentation/sharings_pacts.html). Pact reads the specified pact files (from remote or local sources) and replays the interactions against a running Provider. If all of the interactions are met we can say that both sides of the contract are satisfied and the test passes. When validating a Provider, you have 3 options to provide the Pact files: 1. Use "PactURLs" to specify the exact set of pacts to be replayed: 2. Use "PactBroker" to automatically find all of the latest consumers: 3. Use "PactBroker" and "Tags" to automatically find all of the latest consumers: Options 2 and 3 are particularly useful when you want to validate that your Provider is able to meet the contracts of what's in Production and also the latest in development. See this [article](http://rea.tech/enter-the-pact-matrix-or-how-to-decouple-the-release-cycles-of-your-microservices/) for more on this strategy. Each interaction in a pact should be verified in isolation, with no context maintained from the previous interactions. So how do you test a request that requires data to exist on the provider? Provider states are how you achieve this using Pact. Provider states also allow the consumer to make the same request with different expected responses (e.g. different response codes, or the same resource with a different subset of data). States are configured on the consumer side when you issue a dsl.Given() clause with a corresponding request/response pair. Configuring the provider is a little more involved, and (currently) requires running an API endpoint to configure any [provider states](http://docs.pact.io/documentation/provider_states.html) during the verification process. The option you must provide to the dsl.VerifyRequest is: An example route using the standard Go http package might look like this: See the examples or read more at http://docs.pact.io/documentation/provider_states.html. See the Pact Broker (http://docs.pact.io/documentation/sharings_pacts.html) documentation for more details on the Broker and this article (http://rea.tech/enter-the-pact-matrix-or-how-to-decouple-the-release-cycles-of-your-microservices/) on how to make it work for you. Publishing using Go code: Publishing from the CLI: Use a cURL request like the following to PUT the pact to the right location, specifying your consumer name, provider name and consumer version. The following flags are required to use basic authentication when publishing or retrieving Pact files to/from a Pact Broker: Pact Go uses a simple log utility (logutils - https://github.com/hashicorp/logutils) to filter log messages. The CLI already contains flags to manage this, should you want to control log level in your tests, you can set it like so:

Command pigeon generates parsers in Go from a PEG grammar. From Wikipedia [0]: Its features and syntax are inspired by the PEG.js project [1], while the implementation is loosely based on [2]. Formal presentation of the PEG theory by Bryan Ford is also an important reference [3]. An introductory blog post can be found at [4]. The pigeon tool must be called with PEG input as defined by the accepted PEG syntax below. The grammar may be provided by a file or read from stdin. The generated parser is written to stdout by default. The following options can be specified: The tool makes no attempt to format the code, nor to detect the required imports. It is recommended to use goimports to properly generate the output code: The goimports tool can be installed with: If the code blocks in the grammar (see below, section "Code block") are golint- and go vet-compliant, then the resulting generated code will also be golint- and go vet-compliant. The generated code doesn't use any third-party dependency unless code blocks in the grammar require such a dependency. The accepted syntax for the grammar is formally defined in the grammar/pigeon.peg file, using the PEG syntax. What follows is an informal description of this syntax. Identifiers, whitespace, comments and literals follow the same notation as the Go language, as defined in the language specification (http://golang.org/ref/spec#Source_code_representation): The grammar must be Unicode text encoded in UTF-8. New lines are identified by the \n character (U+000A). Space (U+0020), horizontal tabs (U+0009) and carriage returns (U+000D) are considered whitespace and are ignored except to separate tokens. A PEG grammar consists of a set of rules. A rule is an identifier followed by a rule definition operator and an expression. An optional display name - a string literal used in error messages instead of the rule identifier - can be specified after the rule identifier. E.g.: The rule definition operator can be any one of those: A rule is defined by an expression. The following sections describe the various expression types. Expressions can be grouped by using parentheses, and a rule can be referenced by its identifier in place of an expression. The choice expression is a list of expressions that will be tested in the order they are defined. The first one that matches will be used. Expressions are separated by the forward slash character "/". E.g.: Because the first match is used, it is important to think about the order of expressions. For example, in this rule, "<=" would never be used because the "<" expression comes first: The sequence expression is a list of expressions that must all match in that same order for the sequence expression to be considered a match. Expressions are separated by whitespace. E.g.: A labeled expression consists of an identifier followed by a colon ":" and an expression. A labeled expression introduces a variable named with the label that can be referenced in the code blocks in the same scope. The variable will have the value of the expression that follows the colon. E.g.: The variable is typed as an empty interface, and the underlying type depends on the following: For terminals (character and string literals, character classes and the any matcher), the value is []byte. E.g.: For predicates (& and !), the value is always nil. E.g.: For a sequence, the value is a slice of empty interfaces, one for each expression value in the sequence. The underlying types of each value in the slice follow the same rules described here, recursively. E.g.: For a repetition (+ and *), the value is a slice of empty interfaces, one for each repetition. The underlying types of each value in the slice follow the same rules described here, recursively. E.g.: For a choice expression, the value is that of the matching choice. E.g.: For the optional expression (?), the value is nil or the value of the expression. E.g.: Of course, the type of the value can be anything once an action code block is used. E.g.: An expression prefixed with the ampersand "&" is the "and" predicate expression: it is considered a match if the following expression is a match, but it does not consume any input. An expression prefixed with the exclamation point "!" is the "not" predicate expression: it is considered a match if the following expression is not a match, but it does not consume any input. E.g.: The expression following the & and ! operators can be a code block. In that case, the code block must return a bool and an error. The operator's semantic is the same, & is a match if the code block returns true, ! is a match if the code block returns false. The code block has access to any labeled value defined in its scope. E.g.: An expression followed by "*", "?" or "+" is a match if the expression occurs zero or more times ("*"), zero or one time "?" or one or more times ("+") respectively. The match is greedy, it will match as many times as possible. E.g. A literal matcher tries to match the input against a single character or a string literal. The literal may be a single-quoted single character, a double-quoted string or a backtick-quoted raw string. The same rules as in Go apply regarding the allowed characters and escapes. The literal may be followed by a lowercase "i" (outside the ending quote) to indicate that the match is case-insensitive. E.g.: A character class matcher tries to match the input against a class of characters inside square brackets "[...]". Inside the brackets, characters represent themselves and the same escapes as in string literals are available, except that the single- and double-quote escape is not valid, instead the closing square bracket "]" must be escaped to be used. Character ranges can be specified using the "[a-z]" notation. Unicode classes can be specified using the "[\pL]" notation, where L is a single-letter Unicode class of characters, or using the "[\p{Class}]" notation where Class is a valid Unicode class (e.g. "Latin"). As for string literals, a lowercase "i" may follow the matcher (outside the ending square bracket) to indicate that the match is case-insensitive. A "^" as first character inside the square brackets indicates that the match is inverted (it is a match if the input does not match the character class matcher). E.g.: The any matcher is represented by the dot ".". It matches any character except the end of file, thus the "!." expression is used to indicate "match the end of file". E.g.: Code blocks can be added to generate custom Go code. There are three kinds of code blocks: the initializer, the action and the predicate. All code blocks appear inside curly braces "{...}". The initializer must appear first in the grammar, before any rule. It is copied as-is (minus the wrapping curly braces) at the top of the generated parser. It may contain function declarations, types, variables, etc. just like any Go file. Every symbol declared here will be available to all other code blocks. Although the initializer is optional in a valid grammar, it is usually required to generate a valid Go source code file (for the package clause). E.g.: Action code blocks are code blocks declared after an expression in a rule. Those code blocks are turned into a method on the "*current" type in the generated source code. The method receives any labeled expression's value as argument (as interface{}) and must return two values, the first being the value of the expression (an interface{}), and the second an error. If a non-nil error is returned, it is added to the list of errors that the parser will return. E.g.: Predicate code blocks are code blocks declared immediately after the and "&" or the not "!" operators. Like action code blocks, predicate code blocks are turned into a method on the "*current" type in the generated source code. The method receives any labeled expression's value as argument (as interface{}) and must return two values, the first being a bool and the second an error. If a non-nil error is returned, it is added to the list of errors that the parser will return. E.g.: The current type is a struct that provides two useful fields that can be accessed in action and predicate code blocks: "pos" and "text". The "pos" field indicates the current position of the parser in the source input. It is itself a struct with three fields: "line", "col" and "offset". Line is a 1-based line number, col is a 1-based column number that counts runes from the start of the line, and offset is a 0-based byte offset. The "text" field is the slice of bytes of the current match. It is empty in a predicate code block. The parser generated by pigeon exports a few symbols so that it can be used as a package with public functions to parse input text. The exported API is: See the godoc page of the generated parser for the test/predicates grammar for an example documentation page of the exported API: http://godoc.org/github.com/mna/pigeon/test/predicates. Like the grammar used to generate the parser, the input text must be UTF-8-encoded Unicode. The start rule of the parser is the first rule in the PEG grammar used to generate the parser. A call to any of the Parse* functions returns the value generated by executing the grammar on the provided input text, and an optional error. Typically, the grammar should generate some kind of abstract syntax tree (AST), but for simple grammars it may evaluate the result immediately, such as in the examples/calculator example. There are no constraints imposed on the author of the grammar, it can return whatever is needed. When the parser returns a non-nil error, the error is always of type errList, which is defined as a slice of errors ([]error). Each error in the list is of type *parserError. This is a struct that has an "Inner" field that can be used to access the original error. So if a code block returns some well-known error like: The original error can be accessed this way: By defaut the parser will continue after an error is returned and will cumulate all errors found during parsing. If the grammar reaches a point where it shouldn't continue, a panic statement can be used to terminate parsing. The panic will be caught at the top-level of the Parse* call and will be converted into a *parserError like any error, and an errList will still be returned to the caller. The divide by zero error in the examples/calculator grammar leverages this feature (no special code is needed to handle division by zero, if it happens, the runtime panics and it is recovered and returned as a parsing error). Providing good error reporting in a parser is not a trivial task. Part of it is provided by the pigeon tool, by offering features such as filename, position and rule name in the error message, but an important part of good error reporting needs to be done by the grammar author. For example, many programming languages use double-quotes for string literals. Usually, if the opening quote is found, the closing quote is expected, and if none is found, there won't be any other rule that will match, there's no need to backtrack and try other choices, an error should be added to the list and the match should be consumed. In order to do this, the grammar can look something like this: This is just one example, but it illustrates the idea that error reporting needs to be thought out when designing the grammar. Generated parsers have user-provided code mixed with pigeon code in the same package, so there is no package boundary in the resulting code to prevent access to unexported symbols. What is meant to be implementation details in pigeon is also available to user code - which doesn't mean it should be used. For this reason, it is important to precisely define what is intended to be the supported API of pigeon, the parts that will be stable in future versions. The "stability" of the API attempts to make a similar guarantee as the Go 1 compatibility [5]. The following lists what part of the current pigeon code falls under that guarantee (features may be added in the future): The pigeon command-line flags and arguments: those will not be removed and will maintain the same semantics. The explicitly exported API generated by pigeon. See [6] for the documentation of this API on a generated parser. The PEG syntax, as documented above. The code blocks (except the initializer) will always be generated as methods on the *current type, and this type is guaranteed to have the fields pos (type position) and text (type []byte). There are no guarantees on other fields and methods of this type. The position type will always have the fields line, col and offset, all defined as int. There are no guarantees on other fields and methods of this type. The type of the error value returned by the Parse* functions, when not nil, will always be errList defined as a []error. There are no guarantees on methods of this type, other than the fact it implements the error interface. Individual errors in the errList will always be of type *parserError, and this type is guaranteed to have an Inner field that contains the original error value. There are no guarantees on other fields and methods of this type. References:

Command mox is a modern, secure, full-featured, open source mail server for low-maintenance self-hosted email. Mox is started with the "serve" subcommand, but mox also has many other subcommands. Many of those commands talk to a running mox instance, through the ctl file in the data directory. Specify the configuration file (that holds the path to the data directory) through the -config flag or MOXCONF environment variable. Commands that don't talk to a running mox instance are often for testing/debugging email functionality. For example for parsing an email message, or looking up SPF/DKIM/DMARC records. Below is the usage information as printed by the command when started without any parameters. Followed by the help and usage information for each command. Start mox, serving SMTP/IMAP/HTTPS. Incoming email is accepted over SMTP. Email can be retrieved by users using IMAP. HTTP listeners are started for the admin/account web interfaces, and for automated TLS configuration. Missing essential TLS certificates are immediately requested, other TLS certificates are requested on demand. Only implemented on unix systems, not Windows. Quickstart generates configuration files and prints instructions to quickly set up a mox instance. Quickstart writes configuration files, prints initial admin and account passwords, DNS records you should create. If you run it on Linux it writes a systemd service file and prints commands to enable and start mox as service. The user or uid is optional, defaults to "mox", and is the user or uid/gid mox will run as after initialization. Quickstart assumes mox will run on the machine you run quickstart on and uses its host name and public IPs. On many systems the hostname is not a fully qualified domain name, but only the first dns "label", e.g. "mail" in case of "mail.example.org". If so, quickstart does a reverse DNS lookup to find the hostname, and as fallback uses the label plus the domain of the email address you specified. Use flag -hostname to explicitly specify the hostname mox will run on. Mox is by far easiest to operate if you let it listen on port 443 (HTTPS) and 80 (HTTP). TLS will be fully automatic with ACME with Let's Encrypt. You can run mox along with an existing webserver, but because of MTA-STS and autoconfig, you'll need to forward HTTPS traffic for two domains to mox. Run "mox quickstart -existing-webserver ..." to generate configuration files and instructions for configuring mox along with an existing webserver. But please first consider configuring mox on port 443. It can itself serve domains with HTTP/HTTPS, including with automatic TLS with ACME, is easily configured through both configuration files and admin web interface, and can act as a reverse proxy (and static file server for that matter), so you can forward traffic to your existing backend applications. Look for "WebHandlers:" in the output of "mox config describe-domains" and see the output of "mox config example webhandlers". Shut mox down, giving connections maximum 3 seconds to stop before closing them. While shutting down, new IMAP and SMTP connections will get a status response indicating temporary unavailability. Existing connections will get a 3 second period to finish their transaction and shut down. Under normal circumstances, only IMAP has long-living connections, with the IDLE command to get notified of new mail deliveries. Set new password an account. The password is read from stdin. Secrets derived from the password, but not the password itself, are stored in the account database. The stored secrets are for authentication with: scram-sha-256, scram-sha-1, cram-md5, plain text (bcrypt hash). The parameter is an account name, as configured under Accounts in domains.conf and as present in the data/accounts/ directory, not a configured email address for an account. Set a new admin password, for the web interface. The password is read from stdin. Its bcrypt hash is stored in a file named "adminpasswd" in the configuration directory. Print the log levels, or set a new default log level, or a level for the given package. By default, a single log level applies to all logging in mox. But for each "pkg", an overriding log level can be configured. Examples of packages: smtpserver, smtpclient, queue, imapserver, spf, dkim, dmarc, junk, message, etc. Specify a pkg and an empty level to clear the configured level for a package. Valid labels: error, info, debug, trace, traceauth, tracedata. List hold rules for the delivery queue. Messages submitted to the queue that match a hold rule will be marked as on hold and not scheduled for delivery. Add hold rule for the delivery queue. Add a hold rule to mark matching newly submitted messages as on hold. Set the matching rules with the flags. Don't specify any flags to match all submitted messages. Remove hold rule for the delivery queue. Remove a hold rule by its id. List matching messages in the delivery queue. Prints the message with its ID, last and next delivery attempts, last error. Mark matching messages on hold. Messages that are on hold are not delivered until marked as off hold again, or otherwise handled by the admin. Mark matching messages off hold. Once off hold, messages can be delivered according to their current next delivery attempt. See the "queue schedule" command. Change next delivery attempt for matching messages. The next delivery attempt is adjusted by the duration parameter. If the -now flag is set, the new delivery attempt is set to the duration added to the current time, instead of added to the current scheduled time. Schedule immediate delivery with "mox queue schedule -now 0". Set transport for matching messages. By default, the routing rules determine how a message is delivered. The default and common case is direct delivery with SMTP. Messages can get a previously configured transport assigned to use for delivery, e.g. using submission to another mail server or with connections over a SOCKS proxy. Set TLS requirements for delivery of matching messages. Value "yes" is handled as if the RequireTLS extension was specified during submission. Value "no" is handled as if the message has a header "TLS-Required: No". This header is not added by the queue. If messages without this header are relayed through other mail servers they will apply their own default TLS policy. Value "default" is the default behaviour, currently for unverified opportunistic TLS. Fail delivery of matching messages, delivering DSNs. Failing a message is handled similar to how delivery is given up after all delivery attempts failed. The DSN (delivery status notification) message contains a line saying the message was canceled by the admin. Remove matching messages from the queue. Dangerous operation, this completely removes the message. If you want to store the message, use "queue dump" before removing. Dump a message from the queue. The message is printed to stdout and is in standard internet mail format. List matching messages in the retired queue. Prints messages with their ID and results. Print a message from the retired queue. Prints a JSON representation of the information from the retired queue. Print addresses in suppression list. Add address to suppression list for account. Remove address from suppression list for account. Check if address is present in suppression list, for any or specific account. List matching webhooks in the queue. Prints list of webhooks, their IDs and basic information. Change next delivery attempt for matching webhooks. The next delivery attempt is adjusted by the duration parameter. If the -now flag is set, the new delivery attempt is set to the duration added to the current time, instead of added to the current scheduled time. Schedule immediate delivery with "mox queue schedule -now 0". Fail delivery of matching webhooks. Print details of a webhook from the queue. The webhook is printed to stdout as JSON. List matching webhooks in the retired queue. Prints list of retired webhooks, their IDs and basic information. Print details of a webhook from the retired queue. The retired webhook is printed to stdout as JSON. Import a maildir into an account. The mbox/maildir archive is accessed and imported by the running mox process, so it must have access to the archive files. The default suggested systemd service file isolates mox from most of the file system, with only the "data/" directory accessible, so you may want to put the mbox/maildir archive files in a directory like "data/import/" to make it available to mox. By default, messages will train the junk filter based on their flags and, if "automatic junk flags" configuration is set, based on mailbox naming. If the destination mailbox is the Sent mailbox, the recipients of the messages are added to the message metadata, causing later incoming messages from these recipients to be accepted, unless other reputation signals prevent that. Users can also import mailboxes/messages through the account web page by uploading a zip or tgz file with mbox and/or maildirs. Messages are imported even if already present. Importing messages twice will result in duplicate messages. Mailbox flags, like "seen", "answered", will be imported. An optional dovecot-keywords file can specify additional flags, like Forwarded/Junk/NotJunk. Import an mbox into an account. Using mbox is not recommended, maildir is a better defined format. The mbox/maildir archive is accessed and imported by the running mox process, so it must have access to the archive files. The default suggested systemd service file isolates mox from most of the file system, with only the "data/" directory accessible, so you may want to put the mbox/maildir archive files in a directory like "data/import/" to make it available to mox. By default, messages will train the junk filter based on their flags and, if "automatic junk flags" configuration is set, based on mailbox naming. If the destination mailbox is the Sent mailbox, the recipients of the messages are added to the message metadata, causing later incoming messages from these recipients to be accepted, unless other reputation signals prevent that. Users can also import mailboxes/messages through the account web page by uploading a zip or tgz file with mbox and/or maildirs. Messages are imported even if already present. Importing messages twice will result in duplicate messages. Export one or all mailboxes from an account in maildir format. Export bypasses a running mox instance. It opens the account mailbox/message database file directly. This may block if a running mox instance also has the database open, e.g. for IMAP connections. To export from a running instance, use the accounts web page or webmail. Export messages from one or all mailboxes in an account in mbox format. Using mbox is not recommended. Maildir is a better format. Export bypasses a running mox instance. It opens the account mailbox/message database file directly. This may block if a running mox instance also has the database open, e.g. for IMAP connections. To export from a running instance, use the accounts web page or webmail. For mbox export, "mboxrd" is used where message lines starting with the magic "From " string are escaped by prepending a >. All ">*From " are escaped, otherwise reconstructing the original could lose a ">". Start a local SMTP/IMAP server that accepts all messages, useful when testing/developing software that sends email. Localserve starts mox with a configuration suitable for local email-related software development/testing. It listens for SMTP/Submission(s), IMAP(s) and HTTP(s), on the regular port numbers + 1000. Data is stored in the system user's configuration directory under "mox-localserve", e.g. $HOME/.config/mox-localserve/ on linux, but can be overridden with the -dir flag. If the directory does not yet exist, it is automatically initialized with configuration files, an account with email address mox@localhost and password moxmoxmox, and a newly generated self-signed TLS certificate. Incoming messages are delivered as normal, falling back to accepting and delivering to the mox account for unknown addresses. Submitted messages are added to the queue, which delivers by ignoring the destination servers, always connecting to itself instead. Recipient addresses with the following localpart suffixes are handled specially: - "temperror": fail with a temporary error code - "permerror": fail with a permanent error code - [45][0-9][0-9]: fail with the specific error code - "timeout": no response (for an hour) If the localpart begins with "mailfrom" or "rcptto", the error is returned during those commands instead of during "data". Prints help about matching commands. If multiple commands match, they are listed along with the first line of their help text. If a single command matches, its usage and full help text is printed. Creates a backup of the data directory. Backup creates consistent snapshots of the databases and message files and copies other files in the data directory. Empty directories are not copied. These files can then be stored elsewhere for long-term storage, or used to fall back to should an upgrade fail. Simply copying files in the data directory while mox is running can result in unusable database files. Message files never change (they are read-only, though can be removed) and are hard-linked so they don't consume additional space. If hardlinking fails, for example when the backup destination directory is on a different file system, a regular copy is made. Using a destination directory like "data/tmp/backup" increases the odds hardlinking succeeds: the default systemd service file specifically mounts the data directory, causing attempts to hardlink outside it to fail with an error about cross-device linking. All files in the data directory that aren't recognized (i.e. other than known database files, message files, an acme directory, the "tmp" directory, etc), are stored, but with a warning. Remove files in the destination directory before doing another backup. The backup command will not overwrite files, but print and return errors. Exit code 0 indicates the backup was successful. A clean successful backup does not print any output, but may print warnings. Use the -verbose flag for details, including timing. To restore a backup, first shut down mox, move away the old data directory and move an earlier backed up directory in its place, run "mox verifydata", possibly with the "-fix" option, and restart mox. After the restore, you may also want to run "mox bumpuidvalidity" for each account for which messages in a mailbox changed, to force IMAP clients to synchronize mailbox state. Before upgrading, to check if the upgrade will likely succeed, first make a backup, then use the new mox binary to run "mox verifydata" on the backup. This can change the backup files (e.g. upgrade database files, move away unrecognized message files), so you should make a new backup before actually upgrading. Verify the contents of a data directory, typically of a backup. Verifydata checks all database files to see if they are valid BoltDB/bstore databases. It checks that all messages in the database have a corresponding on-disk message file and there are no unrecognized files. If option -fix is specified, unrecognized message files are moved away. This may be needed after a restore, because messages enqueued or delivered in the future may get those message sequence numbers assigned and writing the message file would fail. Consistency of message/mailbox UID, UIDNEXT and UIDVALIDITY is verified as well. Because verifydata opens the database files, schema upgrades may automatically be applied. This can happen if you use a new mox release. It is useful to run "mox verifydata" with a new binary before attempting an upgrade, but only on a copy of the database files, as made with "mox backup". Before upgrading, make a new backup again since "mox verifydata" may have upgraded the database files, possibly making them potentially no longer readable by the previous version. Print licenses of mox source code and dependencies. Parses and validates the configuration files. If valid, the command exits with status 0. If not valid, all errors encountered are printed. Check the DNS records with the configuration for the domain, and print any errors/warnings. Prints annotated DNS records as zone file that should be created for the domain. The zone file can be imported into existing DNS software. You should review the DNS records, especially if your domain previously/currently has email configured. Prints an annotated empty configuration for use as domains.conf. The domains configuration file contains the domains and their configuration, and accounts and their configuration. This includes the configured email addresses. The mox admin web interface, and the mox command line interface, can make changes to this file. Mox automatically reloads this file when it changes. Like the static configuration, the example domains.conf printed by this command needs modifications to make it valid. Prints an annotated empty configuration for use as mox.conf. The static configuration file cannot be reloaded while mox is running. Mox has to be restarted for changes to the static configuration file to take effect. This configuration file needs modifications to make it valid. For example, it may contain unfinished list items. Add an account with an email address and reload the configuration. Email can be delivered to this address/account. A password has to be configured explicitly, see the setaccountpassword command. Remove an account and reload the configuration. Email addresses for this account will also be removed, and incoming email for these addresses will be rejected. All data for the account will be removed. Adds an address to an account and reloads the configuration. If address starts with a @ (i.e. a missing localpart), this is a catchall address for the domain. Remove an address and reload the configuration. Incoming email for this address will be rejected after removing an address. Adds a new domain to the configuration and reloads the configuration. The account is used for the postmaster mailboxes the domain, including as DMARC and TLS reporting. Localpart is the "username" at the domain for this account. If must be set if and only if account does not yet exist. Remove a domain from the configuration and reload the configuration. This is a dangerous operation. Incoming email delivery for this domain will be rejected. List aliases for domain. Print settings and members of alias. Add new alias with one or more addresses. Update alias configuration. Remove alias. Add addresses to alias. Remove addresses from alias. Describe configuration for mox when invoked as sendmail. Prints a systemd unit service file for mox. This is the same file as generated using quickstart. If the systemd service file has changed with a newer version of mox, use this command to generate an up to date version. Ensure host private keys exist for TLS listeners with ACME. In mox.conf, each listener can have TLS configured. Long-lived private key files can be specified, which will be used when requesting ACME certificates. Configuring these private keys makes it feasible to publish DANE TLSA records for the corresponding public keys in DNS, protected with DNSSEC, allowing TLS certificate verification without depending on a list of Certificate Authorities (CAs). Previous versions of mox did not pre-generate private keys for use with ACME certificates, but would generate private keys on-demand. By explicitly configuring private keys, they will not change automatedly with new certificates, and the DNS TLSA records stay valid. This command looks for listeners in mox.conf with TLS with ACME configured. For each missing host private key (of type rsa-2048 and ecdsa-p256) a key is written to config/hostkeys/. If a certificate exists in the ACME "cache", its private key is copied. Otherwise a new private key is generated. Snippets for manually updating/editing mox.conf are printed. After running this command, and updating mox.conf, run "mox config dnsrecords" for a domain and create the TLSA DNS records it suggests to enable DANE. List available config examples, or print a specific example. Check if a newer version of mox is available. A single DNS TXT lookup to _updates.xmox.nl tells if a new version is available. If so, a changelog is fetched from https://updates.xmox.nl, and the individual entries verified with a builtin public key. The changelog is printed. Turn an ID from a Received header into a cid, for looking up in logs. A cid is essentially a connection counter initialized when mox starts. Each log line contains a cid. Received headers added by mox contain a unique ID that can be decrypted to a cid by admin of a mox instance only. Print the configuration for email clients for a domain. Sending email is typically not done on the SMTP port 25, but on submission ports 465 (with TLS) and 587 (without initial TLS, but usually added to the connection with STARTTLS). For IMAP, the port with TLS is 993 and without is 143. Without TLS/STARTTLS, passwords are sent in clear text, which should only be configured over otherwise secured connections, like a VPN. Dial the address using TLS with certificate verification using DANE. Data is copied between connection and stdin/stdout until either side closes the connection. Connect to MX server for domain using STARTTLS verified with DANE. If no destination host is specified, regular delivery logic is used to find the hosts to attempt delivery too. This involves following CNAMEs for the domain, looking up MX records, and possibly falling back to the domain name itself as host. If a destination host is specified, that is the only candidate host considered for dialing. With a list of destinations gathered, each is dialed until a successful SMTP session verified with DANE has been initialized, including EHLO and STARTTLS commands. Once connected, data is copied between connection and stdin/stdout, until either side closes the connection. This command follows the same logic as delivery attempts made from the queue, sharing most of its code. Print TLSA record for given certificate/key and parameters. Valid values: - usage: pkix-ta (0), pkix-ee (1), dane-ta (2), dane-ee (3) - selector: cert (0), spki (1) - matchtype: full (0), sha2-256 (1), sha2-512 (2) Common DANE TLSA record parameters are: dane-ee spki sha2-256, or 3 1 1, followed by a sha2-256 hash of the DER-encoded "SPKI" (subject public key info) from the certificate. An example DNS zone file entry: The first usable information from the pem file is used to compose the TLSA record. In case of selector "cert", a certificate is required. Otherwise the "subject public key info" (spki) of the first certificate or public or private key (pkcs#8, pkcs#1 or ec private key) is used. Lookup DNS name of given type. Lookup always prints whether the response was DNSSEC-protected. Examples: mox dns lookup ptr 1.1.1.1 mox dns lookup mx xmox.nl mox dns lookup txt _dmarc.xmox.nl. mox dns lookup tlsa _25._tcp.xmox.nl Generate a new ed25519 key for use with DKIM. Ed25519 keys are much smaller than RSA keys of comparable cryptographic strength. This is convenient because of maximum DNS message sizes. At the time of writing, not many mail servers appear to support ed25519 DKIM keys though, so it is recommended to sign messages with both RSA and ed25519 keys. Generate a new 2048 bit RSA private key for use with DKIM. The generated file is in PEM format, and has a comment it is generated for use with DKIM, by mox. Lookup and print the DKIM record for the selector at the domain. Print a DKIM DNS TXT record with the public key derived from the private key read from stdin. The DNS should be configured as a TXT record at $selector._domainkey.$domain. Verify the DKIM signatures in a message and print the results. The message is parsed, and the DKIM-Signature headers are validated. Validation of older messages may fail because the DNS records have been removed or changed by now, or because the signature header may have specified an expiration time that was passed. Sign a message, adding DKIM-Signature headers based on the domain in the From header. The message is parsed, the domain looked up in the configuration files, and DKIM-Signature headers generated. The message is printed with the DKIM-Signature headers prepended. Lookup dmarc policy for domain, a DNS TXT record at _dmarc.<domain>, validate and print it. Parse a DMARC report from an email message, and print its extracted details. DMARC reports are periodically mailed, if requested in the DMARC DNS record of a domain. Reports are sent by mail servers that received messages with our domain in a From header. This may or may not be legatimate email. DMARC reports contain summaries of evaluations of DMARC and DKIM/SPF, which can help understand email deliverability problems. Parse an email message and evaluate it against the DMARC policy of the domain in the From-header. mailfromaddress and helodomain are used for SPF validation. If both are empty, SPF validation is skipped. mailfromaddress should be the address used as MAIL FROM in the SMTP session. For DSN messages, that address may be empty. The helo domain was specified at the beginning of the SMTP transaction that delivered the message. These values can be found in message headers. For each reporting address in the domain's DMARC record, check if it has opted into receiving reports (if needed). A DMARC record can request reports about DMARC evaluations to be sent to an email/http address. If the organizational domains of that of the DMARC record and that of the report destination address do not match, the destination address must opt-in to receiving DMARC reports by creating a DMARC record at <dmarcdomain>._report._dmarc.<reportdestdomain>. Test if IP is in the DNS blocklist of the zone, e.g. bl.spamcop.net. If the IP is in the blocklist, an explanation is printed. This is typically a URL with more information. Check the health of the DNS blocklist represented by zone, e.g. bl.spamcop.net. The health of a DNS blocklist can be checked by querying for 127.0.0.1 and 127.0.0.2. The second must and the first must not be present. Lookup the MTASTS record and policy for the domain. MTA-STS is a mechanism for a domain to specify if it requires TLS connections for delivering email. If a domain has a valid MTA-STS DNS TXT record at _mta-sts.<domain> it signals it implements MTA-STS. A policy can then be fetched at https://mta-sts.<domain>/.well-known/mta-sts.txt. The policy specifies the mode (enforce, testing, none), which MX servers support TLS and should be used, and how long the policy can be cached. Recreate and retrain the junk filter for the account. Useful after having made changes to the junk filter configuration, or if the implementation has changed. Sendmail is a drop-in replacement for /usr/sbin/sendmail to deliver emails sent by unix processes like cron. If invoked as "sendmail", it will act as sendmail for sending messages. Its intention is to let processes like cron send emails. Messages are submitted to an actual mail server over SMTP. The destination mail server and credentials are configured in /etc/moxsubmit.conf, see mox config describe-sendmail. The From message header is rewritten to the configured address. When the addressee appears to be a local user, because without @, the message is sent to the configured default address. If submitting an email fails, it is added to a directory moxsubmit.failures in the user's home directory. Most flags are ignored to fake compatibility with other sendmail implementations. A single recipient or the -t flag with a To-header is required. With the -t flag, Cc and Bcc headers are not handled specially, so Bcc is not removed and the addresses do not receive the email. /etc/moxsubmit.conf should be group-readable and not readable by others and this binary should be setgid that group: Check the status of IP for the policy published in DNS for the domain. IPs may be allowed to send for a domain, or disallowed, and several shades in between. If not allowed, an explanation may be provided by the policy. If so, the explanation is printed. The SPF mechanism that matched (if any) is also printed. Lookup the SPF record for the domain and print it. Parse the record as SPF record. If valid, nothing is printed. Lookup the TLSRPT record for the domain. A TLSRPT record typically contains an email address where reports about TLS connectivity should be sent. Mail servers attempting delivery to our domain should attempt to use TLS. TLSRPT lets them report how many connection successfully used TLS, and how what kind of errors occurred otherwise. Parse and print the TLSRPT in the message. The report is printed in formatted JSON. Prints this mox version. Lists available methods, prints request/response parameters for method, or calls a method with a request read from standard input. List available examples, or print a specific example. Change the IMAP UID validity of the mailbox, causing IMAP clients to refetch messages. This can be useful after manually repairing metadata about the account/mailbox. Opens account database file directly. Ensure mox does not have the account open, or is not running. Reassign UIDs in one mailbox or all mailboxes in an account and bump UID validity, causing IMAP clients to refetch messages. Opens account database file directly. Ensure mox does not have the account open, or is not running. Fix inconsistent UIDVALIDITY and UIDNEXT in messages/mailboxes/account. The next UID to use for a message in a mailbox should always be higher than any existing message UID in the mailbox. If it is not, the mailbox UIDNEXT is updated. Each mailbox has a UIDVALIDITY sequence number, which should always be lower than the per-account next UIDVALIDITY to use. If it is not, the account next UIDVALIDITY is updated. Opens account database file directly. Ensure mox does not have the account open, or is not running. Ensure message sizes in the database matching the sum of the message prefix length and on-disk file size. Messages with an inconsistent size are also parsed again. If an inconsistency is found, you should probably also run "mox bumpuidvalidity" on the mailboxes or entire account to force IMAP clients to refetch messages. Parse all messages in the account or all accounts again. Can be useful after upgrading mox with improved message parsing. Messages are parsed in batches, so other access to the mailboxes/messages are not blocked while reparsing all messages. Ensure messages in the database have a pre-parsed MIME form in the database. Recalculate message counts for all mailboxes in the account, and total message size for quota. When a message is added to/removed from a mailbox, or when message flags change, the total, unread, unseen and deleted messages are accounted, the total size of the mailbox, and the total message size for the account. In case of a bug in this accounting, the numbers could become incorrect. This command will find, fix and print them. Parse message, print JSON representation. Reassign message threads. For all accounts, or optionally only the specified account. Threading for all messages in an account is first reset, and new base subject and normalized message-id saved with the message. Then all messages are evaluated and matched against their parents/ancestors. Messages are matched based on the References header, with a fall-back to an In-Reply-To header, and if neither is present/valid, based only on base subject. A References header typically points to multiple previous messages in a hierarchy. From oldest ancestor to most recent parent. An In-Reply-To header would have only a message-id of the parent message. A message is only linked to a parent/ancestor if their base subject is the same. This ensures unrelated replies, with a new subject, are placed in their own thread. The base subject is lower cased, has whitespace collapsed to a single space, and some components removed: leading "Re:", "Fwd:", "Fw:", or bracketed tag (that mailing lists often add, e.g. "[listname]"), trailing "(fwd)", or enclosing "[fwd: ...]". Messages are linked to all their ancestors. If an intermediate parent/ancestor message is deleted in the future, the message can still be linked to the earlier ancestors. If the direct parent already wasn't available while matching, this is stored as the message having a "missing link" to its stored ancestors.

Package main is the UBNT edgeos-dnsmasq-blacklist dnsmasq DNS Blacklisting and Redirection. View the software license here (https://github.com/britannic/blacklist/blob/master/LICENSE.txt)Latest versionVersion (https://github.com/britannic/blacklist)Go documentationGoDoc (https://godoc.org/github.com/britannic/blacklist)Build status for this versionBuild Status (https://travis-ci.org/britannic/blacklist)Test coverage status for this versionCoverage Status (https://coveralls.io/github/britannic/blacklist?branch=master)Quality of Go code for this versionGo Report Card (https://goreportcard.com/report/github.com/britannic/blacklist) Follow the conversation @ community.ubnt.com (https://community.ubnt.com/t5/EdgeRouter/DNS-Adblocking-amp-Blacklisting-dnsmasq-Configuration/td-p/2215008/jump-to/first-unread-message "Follow the conversation about this software in the EdgeRouter forum (https://community.ubnt.com/t5/EdgeRouter/)") Please show your thanks by donating to the project using Securely send and receive cash without fees using Square CashSquare Cash (https://cash.me/$HelmRockSecurity/) or PayPal (https://www.paypal.me/helmrocksecurity/) Donate (https://cash.me/$HelmRockSecurity/5 "Give $5 using Square Cash (free money transfer)") Donate (https://cash.me/$HelmRockSecurity/10 "Give $10 using Square Cash (free money transfer)") Donate (https://cash.me/$HelmRockSecurity/15 "Give $15 using Square Cash (free money transfer)") Donate (https://cash.me/$HelmRockSecurity/20 "Give $20 using Square Cash (free money transfer)") Donate (https://cash.me/$HelmRockSecurity/25 "Give $25 using Square Cash (free money transfer)") Donate (https://cash.me/$HelmRockSecurity/50 "Give $50 using Square Cash (free money transfer)") Donate (https://cash.me/$HelmRockSecurity/100 "Give $100 using Square Cash (free money transfer)") Donate (https://cash.me/$HelmRockSecurity/ "Choose your own donation amount using Square Cash (free money transfer)") Donate (https://paypal.me/helmrocksecurity/5 "Give $5 using PayPal (PayPal money transfer)") Donate (https://paypal.me/helmrocksecurity/10 "Give $10 using PayPal (PayPal money transfer)") Donate (https://paypal.me/helmrocksecurity/15 "Give $15 using PayPal (PayPal money transfer)") Donate (https://paypal.me/helmrocksecurity/20 "Give $20 using PayPal (PayPal money transfer)") Donate (https://paypal.me/helmrocksecurity/25 "Give $25 using PayPal (PayPal money transfer)") Donate (https://paypal.me/helmrocksecurity/50 "Give $50 using PayPal (PayPal money transfer)") Donate (https://paypal.me/helmrocksecurity/100 "Give $100 using PayPal (PayPal money transfer)") Donate (https://paypal.me/helmrocksecurity/ "Choose your own donation amount using PayPal (PayPal money transfer)") We greatly appreciate any and all donations - Thank you! Funds go to maintaining development servers and networks. Note: This is 3rd party software and isn't supported or endorsed by Ubiquiti Networks® • Overview (#overview) • Donate (#donations-and-sponsorship) • Copyright (#copyright) • Licenses (#licenses) • Latest Version (#latest-version) • Change Log (https://github.com/britannic/blacklist/blob/master/CHANGELOG.md) • Features (#features) • Compatibility (#compatibility) • Installation (#installation) • Using apt-get (#apt-get-installation---erlite-3-erpoe-5-er-x-er-x-sfp--unifi-gateway-3) • Using dpkg (#dpkg-installation---best-for-disk-space-constrained-routers) • Upgrade (#upgrade) • Removal (#removal) • Frequently Asked Questions (#frequently-asked-questions) • Can I donate to project? (#donations-and-sponsorship) • Does the install backup my blacklist configuration before deleting it? (#does-the-install-backup-my-blacklist-configuration-before-deleting-it) • Does update-dnsmasq run automatically? (#does-update-dnsmasq-run-automatically) • How do I add or delete sources? (#how-do-i-add-or-delete-sources) • How do I back up my blacklist configuration and restore it later? (#how-do-i-back-up-my-blacklist-configuration-and-restore-it-later) • How do I configure dnsmasq? (#how-do-i-configure-dnsmasq) • How do I configure local file sources instead of internet based ones? (#how-do-i-configure-local-file-sources-instead-of-internet-based-ones) • How do I disable/enable dnsmasq blacklisting? (#how-do-i-disableenable-dnsmasq-blacklisting) • How do I exclude or include a host or a domain? (#how-do-i-exclude-or-include-a-host-or-a-domain) • How do I globally exclude or include hosts or a domains? (#how-do-i-globally-exclude-or-include-hosts-or-a-domains) • How do I use the command line switches? (#how-do-i-use-the-command-line-switches) • How do can keep my USG configuration after an upgrade, provision or reboot? (#how-do-can-keep-my-usg-configuration-after-an-upgrade-provision-or-reboot) • How does whitelisting work? (#how-does-whitelisting-work) • What is the difference between blocking domains and hosts? (#what-is-the-difference-between-blocking-domains-and-hosts) • Which blacklist sources are installed by default? (#which-blacklist-sources-are-installed-by-default) EdgeMax dnsmasq DNS blacklisting and redirection is inspired by the users at EdgeMAX Community (https://community.ubnt.com/t5/EdgeMAX/bd-p/EdgeMAX/) [Top] (#contents) • Copyright © Visit Helm Rock Consulting at https://www.helmrock.com/2019 Helm Rock Consulting (https://www.helmrock.com/) [Top] (#contents) Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: • Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. • Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. The views and conclusions contained in the software and documentation are those of the authors and should not be interpreted as representing official policies, either expressed or implied, of the FreeBSD Project. [Top] (#contents) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Latest versionLatest (https://github.com/britannic/blacklist/releases/latest) Release v1.1.6.2 (April 24, 2018) • Code refactor • Global whitelist and blacklist configuration files now have their own prefix: "roots" i.e. [Top] (#contents) • See changelog (https://github.com/britannic/blacklist/blob/master/CHANGELOG.md) for details. [Top] (#contents) • Adds DNS blacklisting integration to the EdgeRouter configuration • Generates configuration files used directly by dnsmasq to redirect dns lookups • Integrated with the EdgeMax OS CLI • Any FQDN in the blacklist will force dnsmasq to return the configured dns redirect IP address [Top] (#contents) • edgeos-dnsmasq-blacklist has been tested on the EdgeRouter ERLite-3, ERPoe-5, ER-X and UniFi Security Gateway USG-3 routers • EdgeMAX versions: v1.9.7+hotfix.4-v1.10.1, UniFi: v4.4.12-v4.4.18 • integration could be adapted to work on VyOS and Vyatta derived ports, since EdgeOS is a fork and port of Vyatta 6.3 [Top] (#contents) • Using apt-get (#apt-get-installation---erlite-3-erpoe-5-er-x-er-x-sfp--unifi-gateway-3) - works for all routers • Using dpkg (#dpkg-installation---best-for-disk-space-constrained-routers) - best for disk space constrained routers [Top] (#contents) apt-get Installation - ERLite-3, ERPoe-5, ER-X, ER-X-SFP & UniFi-Gateway-3 • Add the blacklist debian package repository using the router's CLI shell • Add the GPG signing key • Update the system repositorities and install edgeos-dnsmasq-blacklist [Top] (#contents) dpkg Installation - best for disk space constrained routers EdgeRouter ERLite-3, ERPoe-5 & UniFi-Gateway-3 [Top] (#contents) EdgeRouter ER-X & ER-X-SFP • Ensure the router has enough space, by removing unnecessary files • Now download and install the edgeos-dnsmasq-blacklist package [Top] (#contents) • If the repository is set up and you are using apt-get: • Note, if you are using dpkg, it cannot upgrade packages, so follow these instructions (#dpkg-installation---best-for-disk-space-constrained-routers) and the previous package version will be automatically removed before the new package version is installed [Top] (#contents) EdgeMAX - All Platforms [Top] (#contents) How do I disable/enable dnsmasq blacklisting? • Use these CLI configure commands: • Disable: • Enable: [Top] (#contents) Does the install backup my blacklist configuration before deleting it? • If a blacklist configuration already exists, the install routine will automatically back it up to /config/user-data/blacklist.$(date +'%FT%H%M%S').cmds [Top] (#contents) How do I back up my blacklist configuration and restore it later? • use the following commands (make a note of the file name): • After installing the latest version, you can merge your backed up configuration: • If you prefer to delete the default configuration and restore your previous configuration, run these commands: [Top] (#contents) Which blacklist sources are installed by default? • You can use this command in the CLI shell to view the current sources after installation or view the log and see previous downloads: [Top] (#contents) How do I configure local file sources instead of internet based ones? • Use these commands to configure a local file source • File contents example for /config/user-data/blist.hosts.src: [Top] (#contents) How do can keep my USG configuration after an upgrade, provision or reboot? • Follow these instructions (https://britannic.github.io/install-edgeos-packages/) on how to automatically install edgeos-dnsmasq-blacklist • Create a config.gateway.json file following these instructions (https://help.ubnt.com/hc/en-us/articles/215458888-UniFi-How-to-further-customize-USG-configuration-with-config-gateway-json) • Here's a sample config.gateway.json (https://raw.githubusercontent.com/britannic/blacklist/master/config.gateway.json) [Top] (#contents) How do I add or delete sources? • Using the CLI configure command, to delete domains and hosts sources: • To add a source, first check it can serve a text list and also note the prefix (if any) before the hosts or domains, e.g. http://www.malwaredomainlist.com/ (http://www.malwaredomainlist.com/) has this format: • So the prefix is "127.0.0.1 " • Here's how to creating the source in the CLI: [Top] (#contents) How do I globally exclude or include hosts or a domains? • Use these example commands to globally include or exclude blacklisted entries: [Top] (#contents) How do I exclude or include a host or a domain? • Use these example commands to include or exclude blacklisted entries: [Top] (#contents) How does whitelisting work? *dnsmasq will whitelist any entries in the configuration file domains and hosts (servers) with a hash in place of an IP address (the "#" force dnsmasq to forward the DNS request to the router's configured nameservers) • i.e. servers (hosts) • i.e. domains [Top] (#contents) Does update-dnsmasq run automatically? • Yes, a scheduled task is created and run daily at midnight with a random start delay is used ensure other routers in the same time zone won't overload the source servers. • The random start delay window is configured in seconds using this command - this example sets the start delay between 1-10800 seconds (0-3 hours): • It can be reconfigured using these CLI configuration commands: • For example, to change the execution interval to every 6 hours, use this command: • In daily use, no additional interaction with update-dnsmasq is required. By default, cron will run update-dnsmasq at midnight each day to download the blacklist sources and update the dnsmasq configuration files in /etc/dnsmasq.d. dnsmasq will automatically be reloaded after the configuration file update is completed. [Top] (#contents) How do I use the command line switches? • update-dnsmasq has the following commandline switches available: [Top] (#contents) How do I configure dnsmasq? • dnsmasq may need to be configured to ensure blacklisting works correctly • Here is an example using the EdgeOS configuration shell [Top] (#contents) What is the difference between blocking domains and hosts? • The difference lies in the order of update-dnsmasq's processing algorithm. Domains are processed first and take precedence over hosts, so that a blacklisted domain will force update-dnsmasq's source parser to exclude subsequent hosts from the same domain. This reduces dnsmasq's list of lookups, since it will automatically redirect hosts for a blacklisted domain. [Top] (#contents) blacklist

Package forest has functions for REST Api testing in Go This package provides a few simple helper types and functions to create functional tests that call HTTP services. A test uses a forest Client which encapsulates a standard http.Client and a base url. Such a client can be created inside a function or by initializing a package variable for shared access. Using a client, you can send http requests and call multiple expectation functions on each response. Most functions of the forest package take the *testing.T variable as an argument to report any error. Example To compose http requests, you create a RequestConfig value which as a Builder interface for setting the path,query,header and body parameters. The ProcessTemplate function can be useful to create textual payloads. To inspect http responses, you use the Expect functions that perform the unmarshalling or use XMLPath or JSONPath functions directly on the response. If needed, implement the standard TestMain to do global setup and teardown. In contrast to the standard behavior, the Body of a http.Response is made re-readable. This means one can apply expectations to a response as well as dump the full contents. The function XMLPath provides XPath expression support. It uses the [https://godoc.org/launchpad.net/xmlpath] package. The similar function JSONPath can be used on JSON documents. Colorizes error output (can be configured using package vars). All functions can also be used in a setup and teardown as part of TestMain. (c) 2015, http://ernestmicklei.com. MIT License

Package gofpdf implements a PDF document generator with high level support for text, drawing and images. - UTF-8 support - Choice of measurement unit, page format and margins - Page header and footer management - Automatic page breaks, line breaks, and text justification - Inclusion of JPEG, PNG, GIF, TIFF and basic path-only SVG images - Colors, gradients and alpha channel transparency - Outline bookmarks - Internal and external links - TrueType, Type1 and encoding support - Page compression - Lines, Bézier curves, arcs, and ellipses - Rotation, scaling, skewing, translation, and mirroring - Clipping - Document protection - Layers - Templates - Barcodes - Charting facility - Import PDFs as templates gofpdf has no dependencies other than the Go standard library. All tests pass on Linux, Mac and Windows platforms. gofpdf supports UTF-8 TrueType fonts and “right-to-left” languages. Note that Chinese, Japanese, and Korean characters may not be included in many general purpose fonts. For these languages, a specialized font (for example, NotoSansSC for simplified Chinese) can be used. Also, support is provided to automatically translate UTF-8 runes to code page encodings for languages that have fewer than 256 glyphs. This repository will not be maintained, at least for some unknown duration. But it is hoped that gofpdf has a bright future in the open source world. Due to Go’s promise of compatibility, gofpdf should continue to function without modification for a longer time than would be the case with many other languages. Forks should be based on the last viable commit. Tools such as active-forks can be used to select a fork that looks promising for your needs. If a particular fork looks like it has taken the lead in attracting followers, this README will be updated to point people in that direction. The efforts of all contributors to this project have been deeply appreciated. Best wishes to all of you. To install the package on your system, run Later, to receive updates, run The following Go code generates a simple PDF file. See the functions in the fpdf_test.go file (shown as examples in this documentation) for more advanced PDF examples. If an error occurs in an Fpdf method, an internal error field is set. After this occurs, Fpdf method calls typically return without performing any operations and the error state is retained. This error management scheme facilitates PDF generation since individual method calls do not need to be examined for failure; it is generally sufficient to wait until after Output() is called. For the same reason, if an error occurs in the calling application during PDF generation, it may be desirable for the application to transfer the error to the Fpdf instance by calling the SetError() method or the SetErrorf() method. At any time during the life cycle of the Fpdf instance, the error state can be determined with a call to Ok() or Err(). The error itself can be retrieved with a call to Error(). This package is a relatively straightforward translation from the original FPDF library written in PHP (despite the caveat in the introduction to Effective Go). The API names have been retained even though the Go idiom would suggest otherwise (for example, pdf.GetX() is used rather than simply pdf.X()). The similarity of the two libraries makes the original FPDF website a good source of information. It includes a forum and FAQ. However, some internal changes have been made. Page content is built up using buffers (of type bytes.Buffer) rather than repeated string concatenation. Errors are handled as explained above rather than panicking. Output is generated through an interface of type io.Writer or io.WriteCloser. A number of the original PHP methods behave differently based on the type of the arguments that are passed to them; in these cases additional methods have been exported to provide similar functionality. Font definition files are produced in JSON rather than PHP. A side effect of running go test ./... is the production of a number of example PDFs. These can be found in the gofpdf/pdf directory after the tests complete. Please note that these examples run in the context of a test. In order run an example as a standalone application, you’ll need to examine fpdf_test.go for some helper routines, for example exampleFilename() and summary(). Example PDFs can be compared with reference copies in order to verify that they have been generated as expected. This comparison will be performed if a PDF with the same name as the example PDF is placed in the gofpdf/pdf/reference directory and if the third argument to ComparePDFFiles() in internal/example/example.go is true. (By default it is false.) The routine that summarizes an example will look for this file and, if found, will call ComparePDFFiles() to check the example PDF for equality with its reference PDF. If differences exist between the two files they will be printed to standard output and the test will fail. If the reference file is missing, the comparison is considered to succeed. In order to successfully compare two PDFs, the placement of internal resources must be consistent and the internal creation timestamps must be the same. To do this, the methods SetCatalogSort() and SetCreationDate() need to be called for both files. This is done automatically for all examples. Nothing special is required to use the standard PDF fonts (courier, helvetica, times, zapfdingbats) in your documents other than calling SetFont(). You should use AddUTF8Font() or AddUTF8FontFromBytes() to add a TrueType UTF-8 encoded font. Use RTL() and LTR() methods switch between “right-to-left” and “left-to-right” mode. In order to use a different non-UTF-8 TrueType or Type1 font, you will need to generate a font definition file and, if the font will be embedded into PDFs, a compressed version of the font file. This is done by calling the MakeFont function or using the included makefont command line utility. To create the utility, cd into the makefont subdirectory and run “go build”. This will produce a standalone executable named makefont. Select the appropriate encoding file from the font subdirectory and run the command as in the following example. In your PDF generation code, call AddFont() to load the font and, as with the standard fonts, SetFont() to begin using it. Most examples, including the package example, demonstrate this method. Good sources of free, open-source fonts include Google Fonts and DejaVu Fonts. The draw2d package is a two dimensional vector graphics library that can generate output in different forms. It uses gofpdf for its document production mode. gofpdf is a global community effort and you are invited to make it even better. If you have implemented a new feature or corrected a problem, please consider contributing your change to the project. A contribution that does not directly pertain to the core functionality of gofpdf should be placed in its own directory directly beneath the contrib directory. Here are guidelines for making submissions. Your change should - be compatible with the MIT License - be properly documented - be formatted with go fmt - include an example in fpdf_test.go if appropriate - conform to the standards of golint and go vet, that is, golint . and go vet . should not generate any warnings - not diminish test coverage Pull requests are the preferred means of accepting your changes. gofpdf is released under the MIT License. It is copyrighted by Kurt Jung and the contributors acknowledged below. This package’s code and documentation are closely derived from the FPDF library created by Olivier Plathey, and a number of font and image resources are copied directly from it. Bruno Michel has provided valuable assistance with the code. Drawing support is adapted from the FPDF geometric figures script by David Hernández Sanz. Transparency support is adapted from the FPDF transparency script by Martin Hall-May. Support for gradients and clipping is adapted from FPDF scripts by Andreas Würmser. Support for outline bookmarks is adapted from Olivier Plathey by Manuel Cornes. Layer support is adapted from Olivier Plathey. Support for transformations is adapted from the FPDF transformation script by Moritz Wagner and Andreas Würmser. PDF protection is adapted from the work of Klemen Vodopivec for the FPDF product. Lawrence Kesteloot provided code to allow an image’s extent to be determined prior to placement. Support for vertical alignment within a cell was provided by Stefan Schroeder. Ivan Daniluk generalized the font and image loading code to use the Reader interface while maintaining backward compatibility. Anthony Starks provided code for the Polygon function. Robert Lillack provided the Beziergon function and corrected some naming issues with the internal curve function. Claudio Felber provided implementations for dashed line drawing and generalized font loading. Stani Michiels provided support for multi-segment path drawing with smooth line joins, line join styles, enhanced fill modes, and has helped greatly with package presentation and tests. Templating is adapted by Marcus Downing from the FPDF_Tpl library created by Jan Slabon and Setasign. Jelmer Snoeck contributed packages that generate a variety of barcodes and help with registering images on the web. Jelmer Snoek and Guillermo Pascual augmented the basic HTML functionality with aligned text. Kent Quirk implemented backwards-compatible support for reading DPI from images that support it, and for setting DPI manually and then having it properly taken into account when calculating image size. Paulo Coutinho provided support for static embedded fonts. Dan Meyers added support for embedded JavaScript. David Fish added a generic alias-replacement function to enable, among other things, table of contents functionality. Andy Bakun identified and corrected a problem in which the internal catalogs were not sorted stably. Paul Montag added encoding and decoding functionality for templates, including images that are embedded in templates; this allows templates to be stored independently of gofpdf. Paul also added support for page boxes used in printing PDF documents. Wojciech Matusiak added supported for word spacing. Artem Korotkiy added support of UTF-8 fonts. Dave Barnes added support for imported objects and templates. Brigham Thompson added support for rounded rectangles. Joe Westcott added underline functionality and optimized image storage. Benoit KUGLER contributed support for rectangles with corners of unequal radius, modification times, and for file attachments and annotations. - Remove all legacy code page font support; use UTF-8 exclusively - Improve test coverage as reported by the coverage tool. Example demonstrates the generation of a simple PDF document. Note that since only core fonts are used (in this case Arial, a synonym for Helvetica), an empty string can be specified for the font directory in the call to New(). Note also that the example.Filename() and example.Summary() functions belong to a separate, internal package and are not part of the gofpdf library. If an error occurs at some point during the construction of the document, subsequent method calls exit immediately and the error is finally retrieved with the output call where it can be handled by the application.

Package gohg is a Go client library for using the Mercurial dvcs via it's Command Server. For Mercurial see: http://mercurial.selenic.com. For the Hg Command Server see: http://mercurial.selenic.com/wiki/CommandServer. ▪ Mercurial For Mercurial any version starting from 1.9 should be ok, cause that's the one where the Command Server was introduced. If you send wrong options to it through gohg, or commands or options not yet supported (or obsolete) in your Hg version, you'll simply get back an error from Hg itself, as gohg does not check them. But on the other hand gohg allows issuing new commands, not yet implemented by gohg; see further. ▪ Go Currently gohg is currently developed with Go1.2.1. Though I started with the Go1.0 versions, I can't remember having had to change one or two minor things when moving to Go1.1.1. Updating to Go1.1.2 required no changes at all. I had an issue though with Go1.2, on Windows only, causing some tests using os.exec.Command to fail. I'll have to look into that further, to find out if I should report a bug. ▪ Platform I'm developing and testing both on Windows 7 and Ubuntu 12.04/13.04/13.10. But I suppose it should work on any other platform that supports Hg and Go. Only Go and it's standard library. And Mercurial should be installed of course. At the commandline type: to have gohg available in your GOPATH. Start with importing the gohg package. Examples: All interaction with the Mercurial Command Server (Hg CS from now on) happens through the HgClient type, of which you have to create an instance: Then you can connect the Hg CS as follows: 1. The Hg executable: The first parameter is the Mercurial command to use (which 'hg'). You can leave it blanc to let the gohg tool use the default Mercurial command on the system. Having a parameter for the Hg command allows for using a different Hg version, for testing purposes for instance. 2. The repository path: The second parameter is the path to the repository you want to work on. You can leave it blanc to have gohg use the repository it can find for the current path you are running the program in (searching upward in the folder tree eventually). 3. The config for the session: The third parameter allows to provide extra configuration for the session. Though this is currently not implemented yet. 4. Should gohg create a new repo before connecting? This fourth parameter allows you to indicate that you want gohg to first create a new Mercurial repo if it does not already exist in the path given by the second parameter. See the documentation for more detailed info. 5. The returnvalue: The HgClient.Connect() method eventually returns an error, so you can check if the connection succeeded, and if it is safe to go on. Once the work is done, you can disconnect the Hg CS using a typical Go idiom: The gohg tool sets some environment variables for the Hg CS session, to ensure it's good working: Once we have a connection to a Hg CS we can do some work with the repository. This is done with commands, and gohg offers 3 ways to use them. 1. The command methods of the HgClient type. 2. The HgCmd type. 3. The ExecCmd() method of the HgClient type. Each of which has its own reason of existence. Commands return a byte slice containing the resulting data, and eventually an error. But there are a few exceptions (see api docs). If a command fails, the returned error contains 5 elements: 1) the name of the internal routine where the error was trapped, 2) the name of the HgClient command that was run, 3) the returncode by Mercurial, 4) the full command that was passed to the Hg CS, and 5) the eventual error message returned by Mercurial. So the command could return something like the following in the err variable when it fails: The command aliases (like 'id' for 'identify') are not implemented. But there are examples in identify.go and showconfig.go of how you can easily implement them yourself. This is the easiest way, a kind of convenience. And the most readable too. A con is that as a user you cannot know the exact command that was passed to Hg, without some extra mechanics. Each command has the same name as the corresponding Hg command, except it starts with a capital letter of course. An example (also see examples/example1.go): Note that these methods all use the HgCmd type internally. As such they are convenience wrappers around that type. You could also consider them as a kind of syntactic sugar. If you just want to simply issue a command, nothing more, they are the way to go. The only way to obtain the commandstring sent to Hg when using these command methods, is by calling the HgClient.ShowLastCmd() method afterwards before issuing any other commands: Using the HgCmd type is kind of the standard way. It is a struct that you can instantiate for any command, and for which you can set elements Name, Options and Params (see the api docs for more details). It allows for building the command step by step, and also to query the exact command that will be sent to the Hg CS. A pro of this method is that it allows you to obtain the exact command string that will be passed to Mercurial before it is performed, by calling the CmdLine() method of HgCmd. This could be handy for logging, or for showing feedback to the user in a GUI program. (You could even call CmdLine() several times, and show the building of the command step by step.) An example (also see examples/example2.go): As you can see, this way requires some more coding. The source code will also show you that the HgCmd type is indeed used as the underlying type for the convenience HgClient commands, in all the New<hg-command>Cmd() constructors. The HgClient type has an extra method ExecCmd(), allowing you to pass a fully custom built command to Hg. It accepts a string slice that is supposed to contain all the elements of the complete command, as you would type it at the command line. It could be a convenient way for performing commands that are not yet implemented in gohg, or to make use of extensions to Hg (for which gohg offers no support (yet?)). An example (also see examples/example3.go): Just like on the commandline, options come before parameters. Options to commands use the same name as the long form of the Mercurial option they represent, but start with the necessary capital letter. An options value can be of type bool, int or string. You just pass the value as the parameter to the option (= type conversion of the value to the option type). You can pass any number of options, as the elements of a slice. Options can occur more than once if appropriate (see the ones marked with '[+]' in the Mercurial help). Parameters are used to provide any arguments for a command that are not options. They are passed in as a string or a slice of strings, depending on the command. These parameters typically contain revisions, paths or filenames and so. The gohg tool only checks if the options the caller gives are valid for that command. It does not check if the values are valid for the combination of that command and that option, as that is done by Mercurial. No need to implement that again. If an option is not valid for a command, it is silently ignored, so it is not passed to the Hg CS. A few options are not implemented, as they seemed not relevant for use with this tool (for instance: the global --color option, or the --print0 option for status). The gohg tool only returns errors, with an as clear as possible message, and never uses log.Fatal() nor panics, even if those may seem appropriate. It leaves it up to the caller to do that eventually. It's not up to this library to decide whether to do a retry or to abort the complete application. ▪ The following config settings are fixated in the code (at least for now): ▪ As mentioned earlier, passing config info is not implemented yet. ▪ Currently the only support for extensions to Mercurial is through the ExecCmd method. ▪ If multiple Hg CSs are used against the same repo, it is up to Mercurial to handle this correctly. ▪ Mercurial is always run in english. Internationalization is not necessary here, as the conversation with Hg is internal to the application. Please note that this tool is still in it's very early stages. If you have suggestions or requests, or experience any problems, please use the issue tracker at https://bitbucket.org/gohg/gohg/issues?status=new&status=open. Or you could send a patch or a pull request. Copyright 2012-2014, The gohg Authors. All rights reserved. Use of this source code is governed by a BSD style license that can be found in the LICENSE.md file.

Package gofpdf implements a PDF document generator with high level support for text, drawing and images. - UTF-8 support - Choice of measurement unit, page format and margins - Page header and footer management - Automatic page breaks, line breaks, and text justification - Inclusion of JPEG, PNG, GIF, TIFF and basic path-only SVG images - Colors, gradients and alpha channel transparency - Outline bookmarks - Internal and external links - TrueType, Type1 and encoding support - Page compression - Lines, Bézier curves, arcs, and ellipses - Rotation, scaling, skewing, translation, and mirroring - Clipping - Document protection - Layers - Templates - Barcodes - Charting facility - Import PDFs as templates gofpdf has no dependencies other than the Go standard library. All tests pass on Linux, Mac and Windows platforms. gofpdf supports UTF-8 TrueType fonts and “right-to-left” languages. Note that Chinese, Japanese, and Korean characters may not be included in many general purpose fonts. For these languages, a specialized font (for example, NotoSansSC for simplified Chinese) can be used. Also, support is provided to automatically translate UTF-8 runes to code page encodings for languages that have fewer than 256 glyphs. This repository will not be maintained, at least for some unknown duration. But it is hoped that gofpdf has a bright future in the open source world. Due to Go’s promise of compatibility, gofpdf should continue to function without modification for a longer time than would be the case with many other languages. Forks should be based on the last viable commit. Tools such as active-forks can be used to select a fork that looks promising for your needs. If a particular fork looks like it has taken the lead in attracting followers, this README will be updated to point people in that direction. The efforts of all contributors to this project have been deeply appreciated. Best wishes to all of you. To install the package on your system, run Later, to receive updates, run The following Go code generates a simple PDF file. See the functions in the fpdf_test.go file (shown as examples in this documentation) for more advanced PDF examples. If an error occurs in an Fpdf method, an internal error field is set. After this occurs, Fpdf method calls typically return without performing any operations and the error state is retained. This error management scheme facilitates PDF generation since individual method calls do not need to be examined for failure; it is generally sufficient to wait until after Output() is called. For the same reason, if an error occurs in the calling application during PDF generation, it may be desirable for the application to transfer the error to the Fpdf instance by calling the SetError() method or the SetErrorf() method. At any time during the life cycle of the Fpdf instance, the error state can be determined with a call to Ok() or Err(). The error itself can be retrieved with a call to Error(). This package is a relatively straightforward translation from the original FPDF library written in PHP (despite the caveat in the introduction to Effective Go). The API names have been retained even though the Go idiom would suggest otherwise (for example, pdf.GetX() is used rather than simply pdf.X()). The similarity of the two libraries makes the original FPDF website a good source of information. It includes a forum and FAQ. However, some internal changes have been made. Page content is built up using buffers (of type bytes.Buffer) rather than repeated string concatenation. Errors are handled as explained above rather than panicking. Output is generated through an interface of type io.Writer or io.WriteCloser. A number of the original PHP methods behave differently based on the type of the arguments that are passed to them; in these cases additional methods have been exported to provide similar functionality. Font definition files are produced in JSON rather than PHP. A side effect of running go test ./... is the production of a number of example PDFs. These can be found in the gofpdf/pdf directory after the tests complete. Please note that these examples run in the context of a test. In order run an example as a standalone application, you’ll need to examine fpdf_test.go for some helper routines, for example exampleFilename() and summary(). Example PDFs can be compared with reference copies in order to verify that they have been generated as expected. This comparison will be performed if a PDF with the same name as the example PDF is placed in the gofpdf/pdf/reference directory and if the third argument to ComparePDFFiles() in internal/example/example.go is true. (By default it is false.) The routine that summarizes an example will look for this file and, if found, will call ComparePDFFiles() to check the example PDF for equality with its reference PDF. If differences exist between the two files they will be printed to standard output and the test will fail. If the reference file is missing, the comparison is considered to succeed. In order to successfully compare two PDFs, the placement of internal resources must be consistent and the internal creation timestamps must be the same. To do this, the methods SetCatalogSort() and SetCreationDate() need to be called for both files. This is done automatically for all examples. Nothing special is required to use the standard PDF fonts (courier, helvetica, times, zapfdingbats) in your documents other than calling SetFont(). You should use AddUTF8Font() or AddUTF8FontFromBytes() to add a TrueType UTF-8 encoded font. Use RTL() and LTR() methods switch between “right-to-left” and “left-to-right” mode. In order to use a different non-UTF-8 TrueType or Type1 font, you will need to generate a font definition file and, if the font will be embedded into PDFs, a compressed version of the font file. This is done by calling the MakeFont function or using the included makefont command line utility. To create the utility, cd into the makefont subdirectory and run “go build”. This will produce a standalone executable named makefont. Select the appropriate encoding file from the font subdirectory and run the command as in the following example. In your PDF generation code, call AddFont() to load the font and, as with the standard fonts, SetFont() to begin using it. Most examples, including the package example, demonstrate this method. Good sources of free, open-source fonts include Google Fonts and DejaVu Fonts. The draw2d package is a two dimensional vector graphics library that can generate output in different forms. It uses gofpdf for its document production mode. gofpdf is a global community effort and you are invited to make it even better. If you have implemented a new feature or corrected a problem, please consider contributing your change to the project. A contribution that does not directly pertain to the core functionality of gofpdf should be placed in its own directory directly beneath the contrib directory. Here are guidelines for making submissions. Your change should - be compatible with the MIT License - be properly documented - be formatted with go fmt - include an example in fpdf_test.go if appropriate - conform to the standards of golint and go vet, that is, golint . and go vet . should not generate any warnings - not diminish test coverage Pull requests are the preferred means of accepting your changes. gofpdf is released under the MIT License. It is copyrighted by Kurt Jung and the contributors acknowledged below. This package’s code and documentation are closely derived from the FPDF library created by Olivier Plathey, and a number of font and image resources are copied directly from it. Bruno Michel has provided valuable assistance with the code. Drawing support is adapted from the FPDF geometric figures script by David Hernández Sanz. Transparency support is adapted from the FPDF transparency script by Martin Hall-May. Support for gradients and clipping is adapted from FPDF scripts by Andreas Würmser. Support for outline bookmarks is adapted from Olivier Plathey by Manuel Cornes. Layer support is adapted from Olivier Plathey. Support for transformations is adapted from the FPDF transformation script by Moritz Wagner and Andreas Würmser. PDF protection is adapted from the work of Klemen Vodopivec for the FPDF product. Lawrence Kesteloot provided code to allow an image’s extent to be determined prior to placement. Support for vertical alignment within a cell was provided by Stefan Schroeder. Ivan Daniluk generalized the font and image loading code to use the Reader interface while maintaining backward compatibility. Anthony Starks provided code for the Polygon function. Robert Lillack provided the Beziergon function and corrected some naming issues with the internal curve function. Claudio Felber provided implementations for dashed line drawing and generalized font loading. Stani Michiels provided support for multi-segment path drawing with smooth line joins, line join styles, enhanced fill modes, and has helped greatly with package presentation and tests. Templating is adapted by Marcus Downing from the FPDF_Tpl library created by Jan Slabon and Setasign. Jelmer Snoeck contributed packages that generate a variety of barcodes and help with registering images on the web. Jelmer Snoek and Guillermo Pascual augmented the basic HTML functionality with aligned text. Kent Quirk implemented backwards-compatible support for reading DPI from images that support it, and for setting DPI manually and then having it properly taken into account when calculating image size. Paulo Coutinho provided support for static embedded fonts. Dan Meyers added support for embedded JavaScript. David Fish added a generic alias-replacement function to enable, among other things, table of contents functionality. Andy Bakun identified and corrected a problem in which the internal catalogs were not sorted stably. Paul Montag added encoding and decoding functionality for templates, including images that are embedded in templates; this allows templates to be stored independently of gofpdf. Paul also added support for page boxes used in printing PDF documents. Wojciech Matusiak added supported for word spacing. Artem Korotkiy added support of UTF-8 fonts. Dave Barnes added support for imported objects and templates. Brigham Thompson added support for rounded rectangles. Joe Westcott added underline functionality and optimized image storage. Benoit KUGLER contributed support for rectangles with corners of unequal radius, modification times, and for file attachments and annotations. - Remove all legacy code page font support; use UTF-8 exclusively - Improve test coverage as reported by the coverage tool. Example demonstrates the generation of a simple PDF document. Note that since only core fonts are used (in this case Arial, a synonym for Helvetica), an empty string can be specified for the font directory in the call to New(). Note also that the example.Filename() and example.Summary() functions belong to a separate, internal package and are not part of the gofpdf library. If an error occurs at some point during the construction of the document, subsequent method calls exit immediately and the error is finally retrieved with the output call where it can be handled by the application.

Package gofpdf implements a PDF document generator with high level support for text, drawing and images. - UTF-8 support - Choice of measurement unit, page format and margins - Page header and footer management - Automatic page breaks, line breaks, and text justification - Inclusion of JPEG, PNG, GIF, TIFF and basic path-only SVG images - Colors, gradients and alpha channel transparency - Outline bookmarks - Internal and external links - TrueType, Type1 and encoding support - Page compression - Lines, Bézier curves, arcs, and ellipses - Rotation, scaling, skewing, translation, and mirroring - Clipping - Document protection - Layers - Templates - Barcodes - Charting facility - Import PDFs as templates gofpdf has no dependencies other than the Go standard library. All tests pass on Linux, Mac and Windows platforms. gofpdf supports UTF-8 TrueType fonts and “right-to-left” languages. Note that Chinese, Japanese, and Korean characters may not be included in many general purpose fonts. For these languages, a specialized font (for example, NotoSansSC for simplified Chinese) can be used. Also, support is provided to automatically translate UTF-8 runes to code page encodings for languages that have fewer than 256 glyphs. This repository will not be maintained, at least for some unknown duration. But it is hoped that gofpdf has a bright future in the open source world. Due to Go’s promise of compatibility, gofpdf should continue to function without modification for a longer time than would be the case with many other languages. Forks should be based on the last viable commit. Tools such as active-forks can be used to select a fork that looks promising for your needs. If a particular fork looks like it has taken the lead in attracting followers, this README will be updated to point people in that direction. The efforts of all contributors to this project have been deeply appreciated. Best wishes to all of you. To install the package on your system, run Later, to receive updates, run The following Go code generates a simple PDF file. See the functions in the fpdf_test.go file (shown as examples in this documentation) for more advanced PDF examples. If an error occurs in an Fpdf method, an internal error field is set. After this occurs, Fpdf method calls typically return without performing any operations and the error state is retained. This error management scheme facilitates PDF generation since individual method calls do not need to be examined for failure; it is generally sufficient to wait until after Output() is called. For the same reason, if an error occurs in the calling application during PDF generation, it may be desirable for the application to transfer the error to the Fpdf instance by calling the SetError() method or the SetErrorf() method. At any time during the life cycle of the Fpdf instance, the error state can be determined with a call to Ok() or Err(). The error itself can be retrieved with a call to Error(). This package is a relatively straightforward translation from the original FPDF library written in PHP (despite the caveat in the introduction to Effective Go). The API names have been retained even though the Go idiom would suggest otherwise (for example, pdf.GetX() is used rather than simply pdf.X()). The similarity of the two libraries makes the original FPDF website a good source of information. It includes a forum and FAQ. However, some internal changes have been made. Page content is built up using buffers (of type bytes.Buffer) rather than repeated string concatenation. Errors are handled as explained above rather than panicking. Output is generated through an interface of type io.Writer or io.WriteCloser. A number of the original PHP methods behave differently based on the type of the arguments that are passed to them; in these cases additional methods have been exported to provide similar functionality. Font definition files are produced in JSON rather than PHP. A side effect of running go test ./... is the production of a number of example PDFs. These can be found in the gofpdf/pdf directory after the tests complete. Please note that these examples run in the context of a test. In order run an example as a standalone application, you’ll need to examine fpdf_test.go for some helper routines, for example exampleFilename() and summary(). Example PDFs can be compared with reference copies in order to verify that they have been generated as expected. This comparison will be performed if a PDF with the same name as the example PDF is placed in the gofpdf/pdf/reference directory and if the third argument to ComparePDFFiles() in internal/example/example.go is true. (By default it is false.) The routine that summarizes an example will look for this file and, if found, will call ComparePDFFiles() to check the example PDF for equality with its reference PDF. If differences exist between the two files they will be printed to standard output and the test will fail. If the reference file is missing, the comparison is considered to succeed. In order to successfully compare two PDFs, the placement of internal resources must be consistent and the internal creation timestamps must be the same. To do this, the methods SetCatalogSort() and SetCreationDate() need to be called for both files. This is done automatically for all examples. Nothing special is required to use the standard PDF fonts (courier, helvetica, times, zapfdingbats) in your documents other than calling SetFont(). You should use AddUTF8Font() or AddUTF8FontFromBytes() to add a TrueType UTF-8 encoded font. Use RTL() and LTR() methods switch between “right-to-left” and “left-to-right” mode. In order to use a different non-UTF-8 TrueType or Type1 font, you will need to generate a font definition file and, if the font will be embedded into PDFs, a compressed version of the font file. This is done by calling the MakeFont function or using the included makefont command line utility. To create the utility, cd into the makefont subdirectory and run “go build”. This will produce a standalone executable named makefont. Select the appropriate encoding file from the font subdirectory and run the command as in the following example. In your PDF generation code, call AddFont() to load the font and, as with the standard fonts, SetFont() to begin using it. Most examples, including the package example, demonstrate this method. Good sources of free, open-source fonts include Google Fonts and DejaVu Fonts. The draw2d package is a two dimensional vector graphics library that can generate output in different forms. It uses gofpdf for its document production mode. gofpdf is a global community effort and you are invited to make it even better. If you have implemented a new feature or corrected a problem, please consider contributing your change to the project. A contribution that does not directly pertain to the core functionality of gofpdf should be placed in its own directory directly beneath the contrib directory. Here are guidelines for making submissions. Your change should - be compatible with the MIT License - be properly documented - be formatted with go fmt - include an example in fpdf_test.go if appropriate - conform to the standards of golint and go vet, that is, golint . and go vet . should not generate any warnings - not diminish test coverage Pull requests are the preferred means of accepting your changes. gofpdf is released under the MIT License. It is copyrighted by Kurt Jung and the contributors acknowledged below. This package’s code and documentation are closely derived from the FPDF library created by Olivier Plathey, and a number of font and image resources are copied directly from it. Bruno Michel has provided valuable assistance with the code. Drawing support is adapted from the FPDF geometric figures script by David Hernández Sanz. Transparency support is adapted from the FPDF transparency script by Martin Hall-May. Support for gradients and clipping is adapted from FPDF scripts by Andreas Würmser. Support for outline bookmarks is adapted from Olivier Plathey by Manuel Cornes. Layer support is adapted from Olivier Plathey. Support for transformations is adapted from the FPDF transformation script by Moritz Wagner and Andreas Würmser. PDF protection is adapted from the work of Klemen Vodopivec for the FPDF product. Lawrence Kesteloot provided code to allow an image’s extent to be determined prior to placement. Support for vertical alignment within a cell was provided by Stefan Schroeder. Ivan Daniluk generalized the font and image loading code to use the Reader interface while maintaining backward compatibility. Anthony Starks provided code for the Polygon function. Robert Lillack provided the Beziergon function and corrected some naming issues with the internal curve function. Claudio Felber provided implementations for dashed line drawing and generalized font loading. Stani Michiels provided support for multi-segment path drawing with smooth line joins, line join styles, enhanced fill modes, and has helped greatly with package presentation and tests. Templating is adapted by Marcus Downing from the FPDF_Tpl library created by Jan Slabon and Setasign. Jelmer Snoeck contributed packages that generate a variety of barcodes and help with registering images on the web. Jelmer Snoek and Guillermo Pascual augmented the basic HTML functionality with aligned text. Kent Quirk implemented backwards-compatible support for reading DPI from images that support it, and for setting DPI manually and then having it properly taken into account when calculating image size. Paulo Coutinho provided support for static embedded fonts. Dan Meyers added support for embedded JavaScript. David Fish added a generic alias-replacement function to enable, among other things, table of contents functionality. Andy Bakun identified and corrected a problem in which the internal catalogs were not sorted stably. Paul Montag added encoding and decoding functionality for templates, including images that are embedded in templates; this allows templates to be stored independently of gofpdf. Paul also added support for page boxes used in printing PDF documents. Wojciech Matusiak added supported for word spacing. Artem Korotkiy added support of UTF-8 fonts. Dave Barnes added support for imported objects and templates. Brigham Thompson added support for rounded rectangles. Joe Westcott added underline functionality and optimized image storage. Benoit KUGLER contributed support for rectangles with corners of unequal radius, modification times, and for file attachments and annotations. - Remove all legacy code page font support; use UTF-8 exclusively - Improve test coverage as reported by the coverage tool. Example demonstrates the generation of a simple PDF document. Note that since only core fonts are used (in this case Arial, a synonym for Helvetica), an empty string can be specified for the font directory in the call to New(). Note also that the example.Filename() and example.Summary() functions belong to a separate, internal package and are not part of the gofpdf library. If an error occurs at some point during the construction of the document, subsequent method calls exit immediately and the error is finally retrieved with the output call where it can be handled by the application.

Package stream provides a set of generic functions for working concurrent design patterns in Go. [](https://github.com/devnw/stream/actions) [](https://goreportcard.com/report/go.atomizer.io/stream) [](https://codecov.io/gh/devnw/stream) [](https://pkg.go.dev/go.atomizer.io/stream) [](https://opensource.org/licenses/Apache-2.0) [](http://makeapullrequest.com) ## Installation To install the package, run: ## Usage ## Benchmarks To execute the benchmarks, run the following command: To view benchmarks over time for the `main` branch of the repository they can be seen on our Benchmark Report Card.

Vet examines Go source code and reports suspicious constructs, such as Printf calls whose arguments do not align with the format string. Vet uses heuristics that do not guarantee all reports are genuine problems, but it can find errors not caught by the compilers. Vet is normally invoked using the go command by running "go vet": vets the package in the current directory. vets the package whose path is provided. Use "go help packages" to see other ways of specifying which packages to vet. Vet's exit code is 2 for erroneous invocation of the tool, 1 if a problem was reported, and 0 otherwise. Note that the tool does not check every possible problem and depends on unreliable heuristics so it should be used as guidance only, not as a firm indicator of program correctness. By default the -all flag is set so all checks are performed. If any flags are explicitly set to true, only those tests are run. Conversely, if any flag is explicitly set to false, only those tests are disabled. Thus -printf=true runs the printf check, -printf=false runs all checks except the printf check. By default vet uses the object files generated by 'go install some/pkg' to typecheck the code. If the -source flag is provided, vet uses only source code. Available checks: Flag: -asmdecl Mismatches between assembly files and Go function declarations. Flag: -assign Check for useless assignments. Flag: -atomic Common mistaken usages of the sync/atomic package. Flag: -bool Mistakes involving boolean operators. Flag: -buildtags Badly formed or misplaced +build tags. Flag: -cgocall Detect some violations of the cgo pointer passing rules. Flag: -composites Composite struct literals that do not use the field-keyed syntax. Flag: -copylocks Locks that are erroneously passed by value. Flag: -httpresponse Mistakes deferring a function call on an HTTP response before checking whether the error returned with the response was nil. Flag: -lostcancel The cancelation function returned by context.WithCancel, WithTimeout, and WithDeadline must be called or the new context will remain live until its parent context is cancelled. (The background context is never cancelled.) Flag: -methods Non-standard signatures for methods with familiar names, including: Flag: -nilfunc Comparisons between functions and nil. Flag: -printf Suspicious calls to functions in the Printf family, including any functions with these names, disregarding case: The -printfuncs flag can be used to redefine this list. If the function name ends with an 'f', the function is assumed to take a format descriptor string in the manner of fmt.Printf. If not, vet complains about arguments that look like format descriptor strings. It also checks for errors such as using a Writer as the first argument of Printf. Flag: -rangeloops Incorrect uses of range loop variables in closures. Flag: -shadow=false (experimental; must be set explicitly) Variables that may have been unintentionally shadowed. Flag: -shift Shifts equal to or longer than the variable's length. Flag: -structtags Struct tags that do not follow the format understood by reflect.StructTag.Get. Well-known encoding struct tags (json, xml) used with unexported fields. Flag: -tests Mistakes involving tests including functions with incorrect names or signatures and example tests that document identifiers not in the package. Flag: -unreachable Unreachable code. Flag: -unsafeptr Likely incorrect uses of unsafe.Pointer to convert integers to pointers. A conversion from uintptr to unsafe.Pointer is invalid if it implies that there is a uintptr-typed word in memory that holds a pointer value, because that word will be invisible to stack copying and to the garbage collector. Flag: -unusedresult Calls to well-known functions and methods that return a value that is discarded. By default, this includes functions like fmt.Errorf and fmt.Sprintf and methods like String and Error. The flags -unusedfuncs and -unusedstringmethods control the set. These flags configure the behavior of vet: For testing and debugging vet can be run directly by invoking "go tool vet" or just running the binary. Run this way, vet might not have up to date information for imported packages. vets the files named, all of which must be in the same package. recursively descends the directory, vetting each package it finds. Vet is a simple checker for static errors in Go source code. See doc.go for more information.

Package validate provides methods to validate a swagger specification, as well as tools to validate data against their schema. This package follows Swagger 2.0. specification (aka OpenAPI 2.0). Reference can be found here: https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md. Validates a spec document (from JSON or YAML) against the JSON schema for swagger, then checks a number of extra rules that can't be expressed in JSON schema. Entry points: Reported as errors: Reported as warnings: The schema validation toolkit validates data against JSON-schema-draft 04 schema. It is tested against the full json-schema-testing-suite (https://github.com/json-schema-org/JSON-Schema-Test-Suite), except for the optional part (bignum, ECMA regexp, ...). It supports the complete JSON-schema vocabulary, including keywords not supported by Swagger (e.g. additionalItems, ...) Entry points: With the current version of this package, the following aspects of swagger are not yet supported:

Package workpool implements a pool of go routines that are dedicated to processing work that is posted into the pool. The following is a list of parameters for creating a TraceLog: Go routines are used to manage and process all the work. A single Queue routine provides the safe queuing of work. The Queue routine keeps track of the amount of work in the queue and reports an error if the queue is full. The concurrencyLevel parameter defines the number of work routines to create. These work routines will process work subbmitted to the queue. The work routines keep track of the number of active work routines for reporting. The PostWork method is used to post work into the ThreadPool. This call will block until the Queue routine reports back success or failure that the work is in queue. The following shows a simple test application The following shows some sample output

Package goBolt implements drivers for the Neo4J Bolt Protocol Versions 1-4. There are some limitations to the types of collections the internalDriver supports. Specifically, maps should always be of type map[string]interface{} and lists should always be of type []interface{}. It doesn't seem that the Bolt protocol supports uint64 either, so the biggest number it can send right now is the int64 max. The URL format is: `bolt://(user):(password)@(host):(port)` Schema must be `bolt`. User and password is only necessary if you are authenticating. TLS is supported by using query parameters on the connection string, like so: `bolt://host:port?tls=true&tls_no_verify=false` The supported query params are: * timeout - the number of seconds to set the connection timeout to. Defaults to 60 seconds. * tls - Set to 'true' or '1' if you want to use TLS encryption * tls_no_verify - Set to 'true' or '1' if you want to accept any server certificate (for testing, not secure) * tls_ca_cert_file - path to a custom ca cert for a self-signed TLS cert * tls_cert_file - path to a cert file for this client (need to verify this is processed by Neo4j) * tls_key_file - path to a key file for this client (need to verify this is processed by Neo4j) Errors returned from the API support wrapping, so if you receive an error from the library, it might be wrapping other errors. You can get the innermost error by using the `InnerMost` method. Failure messages from Neo4J are reported, along with their metadata, as an error. In order to get the failure message metadata from a wrapped error, you can do so by calling `err.(*errors.Error).InnerMost().(messages.FailureMessage).Metadata` If there is an error with the database connection, you should get a sql/internalDriver ErrBadConn as per the best practice recommendations of the Golang SQL Driver. However, this error may be wrapped, so you might have to call `InnerMost` to get it, as specified above.

Package fakeio is a small library to fake stdout/stderr/stdin mainly for unit testing. Following example fakes stdout/stderr/stdin. Set 'from stdin!' as stdin and get output from both stdout and stderr as string. Please read example/example_test.go to see live examples. If you find some bugs, please report it to repository page: https://github.com/rhysd/go-fakeio

Package httpexpect helps with end-to-end HTTP and REST API testing. See example directory: There are two common ways to test API with httpexpect: The second approach works only if the server is a Go module and its handler can be imported in tests. Concrete behaviour is determined by Client implementation passed to Config struct. If you're using http.Client, set its Transport field (http.RoundTriper) to one of the following: Note that http handler can be usually obtained from http framework you're using. E.g., echo framework provides either http.Handler. You can also provide your own implementation of RequestFactory (creates http.Request), or Client (gets http.Request and returns http.Response). If you're starting server from tests, it's very handy to use net/http/httptest. Whenever values are checked for equality in httpexpect, they are converted to "canonical form": This is equivalent to subsequently json.Marshal() and json.Unmarshal() the value and currently is implemented so. When some check fails, failure is reported. If non-fatal failures are used (see Reporter interface), execution is continued and instance that was checked is marked as failed. If specific instance is marked as failed, all subsequent checks are ignored for this instance and for any child instances retrieved after failure. Example: If you want to be informed about every asserion made, successful or failed, you can use AssertionHandler interface. Default implementation of this interface ignores successful assertions and reports failed assertions using Formatter and Reporter objects. Custom AssertionHandler can handle all assertions (e.g. dump them in JSON format) and is free to use or not to use Formatter and Reporter in its sole discretion.

Package httpexpect helps with end-to-end HTTP and REST API testing. See example directory: There are two common ways to test API with httpexpect: The second approach works only if the server is a Go module and its handler can be imported in tests. Concrete behaviour is determined by Client implementation passed to Config struct. If you're using http.Client, set its Transport field (http.RoundTriper) to one of the following: Note that http handler can be usually obtained from http framework you're using. E.g., echo framework provides either http.Handler or fasthttp.RequestHandler. You can also provide your own implementation of RequestFactory (creates http.Request), or Client (gets http.Request and returns http.Response). If you're starting server from tests, it's very handy to use net/http/httptest. Whenever values are checked for equality in httpexpect, they are converted to "canonical form": This is equivalent to subsequently json.Marshal() and json.Unmarshal() the value and currently is implemented so. When some check fails, failure is reported. If non-fatal failures are used (see Reporter interface), execution is continued and instance that was checked is marked as failed. If specific instance is marked as failed, all subsequent checks are ignored for this instance and for any child instances retrieved after failure. Example:

Package forest has functions for REST Api testing in Go This package provides a few simple helper types and functions to create functional tests that call HTTP services. A test uses a forest Client which encapsulates a standard http.Client and a base url. Such a client can be created inside a function or by initializing a package variable for shared access. Using a client, you can send http requests and call multiple expectation functions on each response. Most functions of the forest package take the *testing.T variable as an argument to report any error. Example To compose http requests, you create a RequestConfig value which as a Builder interface for setting the path,query,header and body parameters. The ProcessTemplate function can be useful to create textual payloads. To inspect http responses, you use the Expect functions that perform the unmarshalling or use XMLPath or JSONPath functions directly on the response. If needed, implement the standard TestMain to do global setup and teardown. In contrast to the standard behavior, the Body of a http.Response is made re-readable. This means one can apply expectations to a response as well as dump the full contents. The function XMLPath provides XPath expression support. It uses the [https://godoc.org/launchpad.net/xmlpath] package. The similar function JSONPath can be used on JSON documents. Colorizes error output (can be configured using package vars). All functions can also be used in a setup and teardown as part of TestMain. (c) 2015, http://ernestmicklei.com. MIT License

Package cgi implements the common gateway interface (CGI) for Caddy 2, a modern, full-featured, easy-to-use web server. It has been forked from the fantastic work of Kurt Jung who wrote that plugin for Caddy 1. This plugin lets you generate dynamic content on your website by means of command line scripts. To collect information about the inbound HTTP request, your script examines certain environment variables such as PATH_INFO and QUERY_STRING. Then, to return a dynamically generated web page to the client, your script simply writes content to standard output. In the case of POST requests, your script reads additional inbound content from standard input. The advantage of CGI is that you do not need to fuss with server startup and persistence, long term memory management, sockets, and crash recovery. Your script is called when a request matches one of the patterns that you specify in your Caddyfile. As soon as your script completes its response, it terminates. This simplicity makes CGI a perfect complement to the straightforward operation and configuration of Caddy. The benefits of Caddy, including HTTPS by default, basic access authentication, and lots of middleware options extend easily to your CGI scripts. CGI has some disadvantages. For one, Caddy needs to start a new process for each request. This can adversely impact performance and, if resources are shared between CGI applications, may require the use of some interprocess synchronization mechanism such as a file lock. Your server's responsiveness could in some circumstances be affected, such as when your web server is hit with very high demand, when your script's dependencies require a long startup, or when concurrently running scripts take a long time to respond. However, in many cases, such as using a pre-compiled CGI application like fossil or a Lua script, the impact will generally be insignificant. Another restriction of CGI is that scripts will be run with the same permissions as Caddy itself. This can sometimes be less than ideal, for example when your script needs to read or write files associated with a different owner. Serving dynamic content exposes your server to more potential threats than serving static pages. There are a number of considerations of which you should be aware when using CGI applications. CGI scripts should be located outside of Caddy's document root. Otherwise, an inadvertent misconfiguration could result in Caddy delivering the script as an ordinary static resource. At best, this could merely confuse the site visitor. At worst, it could expose sensitive internal information that should not leave the server. Mistrust the contents of PATH_INFO, QUERY_STRING and standard input. Most of the environment variables available to your CGI program are inherently safe because they originate with Caddy and cannot be modified by external users. This is not the case with PATH_INFO, QUERY_STRING and, in the case of POST actions, the contents of standard input. Be sure to validate and sanitize all inbound content. If you use a CGI library or framework to process your scripts, make sure you understand its limitations. An error in a CGI application is generally handled within the application itself and reported in the headers it returns. Your CGI application can be executed directly or indirectly. In the direct case, the application can be a compiled native executable or it can be a shell script that contains as its first line a shebang that identifies the interpreter to which the file's name should be passed. Caddy must have permission to execute the application. On Posix systems this will mean making sure the application's ownership and permission bits are set appropriately; on Windows, this may involve properly setting up the filename extension association. In the indirect case, the name of the CGI script is passed to an interpreter such as lua, perl or python. - This module needs to be installed (obviously). - The directive needs to be registered in the Caddyfile: The basic cgi directive lets you add a handler in the current caddy router location with a given script and optional arguments. The matcher is a default caddy matcher that is used to restrict the scope of this directive. The directive can be repeated any reasonable number of times. Here is the basic syntax: For example: When a request such as https://example.com/report or https://example.com/report/weekly arrives, the cgi middleware will detect the match and invoke the script named /usr/local/cgi-bin/report. The current working directory will be the same as Caddy itself. Here, it is assumed that the script is self-contained, for example a pre-compiled CGI application or a shell script. Here is an example of a standalone script, similar to one used in the cgi plugin's test suite: The environment variables PATH_INFO and QUERY_STRING are populated and passed to the script automatically. There are a number of other standard CGI variables included that are described below. If you need to pass any special environment variables or allow any environment variables that are part of Caddy's process to pass to your script, you will need to use the advanced directive syntax described below. Beware that in Caddy v2 it is (currently) not possible to separate the path left of the matcher from the full URL. Therefore if you require your CGI program to know the SCRIPT_NAME, make sure to pass that explicitly: In order to specify custom environment variables, pass along one or more environment variables known to Caddy, or specify more than one match pattern for a given rule, you will need to use the advanced directive syntax. That looks like this: For example, The script_name subdirective helps the cgi module to separate the path to the script from the (virtual) path afterwards (which shall be passed to the script). env can be used to define a list of key=value environment variable pairs that shall be passed to the script. pass_env can be used to define a list of environment variables of the Caddy process that shall be passed to the script. If your CGI application runs properly at the command line but fails to run from Caddy it is possible that certain environment variables may be missing. For example, the ruby gem loader evidently requires the HOME environment variable to be set; you can do this with the subdirective pass_env HOME. Another class of problematic applications require the COMPUTERNAME variable. The pass_all_env subdirective instructs Caddy to pass each environment variable it knows about to the CGI excutable. This addresses a common frustration that is caused when an executable requires an environment variable and fails without a descriptive error message when the variable cannot be found. These applications often run fine from the command prompt but fail when invoked with CGI. The risk with this subdirective is that a lot of server information is shared with the CGI executable. Use this subdirective only with CGI applications that you trust not to leak this information. buffer_limit is used when a http request has Transfer-Endcoding: chunked. The Go CGI Handler refused to handle these kinds of requests, see https://github.com/golang/go/issues/5613. In order to work around this the chunked request is buffered by caddy and sent to the CGI application as a whole with the correct CONTENT_LENGTH set. The buffer_limit setting marks a threshold between buffering in memory and using a temporary file. Every request body smaller than the buffer_limit is buffered in-memory. It accepts all formats supported by go-humanize. Default: 4MiB. (An example of this is git push if the objects to push are larger than the http.postBuffer) With the unbuffered_output subdirective it is possible to instruct the CGI handler to flush output from the CGI script as soon as possible. By default, the output is buffered into chunks before it is being written to optimize the network usage and allow to determine the Content-Length. When unbuffered, bytes will be written as soon as possible. This will also force the response to be written in chunked encoding. If you run into unexpected results with the CGI plugin, you are able to examine the environment in which your CGI application runs. To enter inspection mode, add the subdirective inspect to your CGI configuration block. This is a development option that should not be used in production. When in inspection mode, the plugin will respond to matching requests with a page that displays variables of interest. In particular, it will show the replacement value of {match} and the environment variables to which your CGI application has access. For example, consider this example CGI block: When you request a matching URL, for example, the Caddy server will deliver a text page similar to the following. The CGI application (in this case, wapptclsh) will not be called. This information can be used to diagnose problems with how a CGI application is called. To return to operation mode, remove or comment out the inspect subdirective. In this example, the Caddyfile looks like this: Note that a request for /show gets mapped to a script named /usr/local/cgi-bin/report/gen. There is no need for any element of the script name to match any element of the match pattern. The contents of /usr/local/cgi-bin/report/gen are: The purpose of this script is to show how request information gets communicated to a CGI script. Note that POST data must be read from standard input. In this particular case, posted data gets stored in the variable POST_DATA. Your script may use a different method to read POST content. Secondly, the SCRIPT_EXEC variable is not a CGI standard. It is provided by this middleware and contains the entire command line, including all arguments, with which the CGI script was executed. When a browser requests the response looks like When a client makes a POST request, such as with the following command the response looks the same except for the following lines: This small example demonstrates how to write a CGI program in Go. The use of a bytes.Buffer makes it easy to report the content length in the CGI header. When this program is compiled and installed as /usr/local/bin/servertime, the following directive in your Caddy file will make it available: The module is written in a way that it expects the scripts you want it to execute to actually exist. A non-existing or non-executable file is considered a setup error and will yield a HTTP 500. If you want to make sure, only existing scripts are executed, use a more specific matcher, as explained in the Caddy docs. Example: When calling a url like /cgi/foo/bar.pl it will check if the local file ./app/foo/bar.pl exists and only then it will proceed with calling the CGI.

Package fs provides a filesystem-backed Kivik driver. This driver is very much a work in progress. Please refer to the GitHub page for current status and ongoing changes. https://github.com/go-kivik/fsdb Bug reports, feature requests, and pull requests are always welcome. Current development is primarily focused around using fsdb for testing of CouchDB applications, and bootstraping CouchDB applications. Use the `fs` driver name when using this driver. The DSN should be an existing directory on the local filesystem. Access control is managed by your filesystem permissions. Database names represent directories under the path provided to `kivik.New`. For example: would look for document files in `/home/usr/some/path/foo`. This driver supports three types of connection strings to the New() method: CouchDB allows databases and document IDs to contain a slash (/) character. This is not permitted in most operating systems/filenames, to be stored directly on the filesystem this way. Therefore, it is necessary for this package to escape certain characters in filenames. This is done as conservatively as possible. The escaping rules are:

Pact Go enables consumer driven contract testing, providing a mock service and DSL for the consumer project, and interaction playback and verification for the service provider project. Consumer side Pact testing is an isolated test that ensures a given component is able to collaborate with another (remote) component. Pact will automatically start a Mock server in the background that will act as the collaborators' test double. This implies that any interactions expected on the Mock server will be validated, meaning a test will fail if all interactions were not completed, or if unexpected interactions were found: A typical consumer-side test would look something like this: If this test completed successfully, a Pact file should have been written to ./pacts/my_consumer-my_provider.json containing all of the interactions expected to occur between the Consumer and Provider. In addition to verbatim value matching, you have 3 useful matching functions in the `dsl` package that can increase expressiveness and reduce brittle test cases. Here is a complex example that shows how all 3 terms can be used together: This example will result in a response body from the mock server that looks like: See the examples in the dsl package and the matcher tests (https://github.com/pact-foundation/pact-go/blob/master/dsl/matcher_test.go) for more matching examples. NOTE: You will need to use valid Ruby regular expressions (http://ruby-doc.org/core-2.1.5/Regexp.html) and double escape backslashes. Read more about flexible matching (https://github.com/pact-foundation/pact-ruby/wiki/Regular-expressions-and-type-matching-with-Pact. Provider side Pact testing, involves verifying that the contract - the Pact file - can be satisfied by the Provider. A typical Provider side test would like something like: The `VerifyProvider` will handle all verifications, treating them as subtests and giving you granular test reporting. If you don't like this behaviour, you may call `VerifyProviderRaw` directly and handle the errors manually. Note that `PactURLs` may be a list of local pact files or remote based urls (possibly from a Pact Broker - http://docs.pact.io/documentation/sharings_pacts.html). Pact reads the specified pact files (from remote or local sources) and replays the interactions against a running Provider. If all of the interactions are met we can say that both sides of the contract are satisfied and the test passes. When validating a Provider, you have 3 options to provide the Pact files: 1. Use "PactURLs" to specify the exact set of pacts to be replayed: Options 2 and 3 are particularly useful when you want to validate that your Provider is able to meet the contracts of what's in Production and also the latest in development. See this [article](http://rea.tech/enter-the-pact-matrix-or-how-to-decouple-the-release-cycles-of-your-microservices/) for more on this strategy. Each interaction in a pact should be verified in isolation, with no context maintained from the previous interactions. So how do you test a request that requires data to exist on the provider? Provider states are how you achieve this using Pact. Provider states also allow the consumer to make the same request with different expected responses (e.g. different response codes, or the same resource with a different subset of data). States are configured on the consumer side when you issue a dsl.Given() clause with a corresponding request/response pair. Configuring the provider is a little more involved, and (currently) requires running an API endpoint to configure any [provider states](http://docs.pact.io/documentation/provider_states.html) during the verification process. The option you must provide to the dsl.VerifyRequest is: An example route using the standard Go http package might look like this: See the examples or read more at http://docs.pact.io/documentation/provider_states.html. See the Pact Broker (http://docs.pact.io/documentation/sharings_pacts.html) documentation for more details on the Broker and this article (http://rea.tech/enter-the-pact-matrix-or-how-to-decouple-the-release-cycles-of-your-microservices/) on how to make it work for you. Publishing using Go code: Publishing from the CLI: Use a cURL request like the following to PUT the pact to the right location, specifying your consumer name, provider name and consumer version. The following flags are required to use basic authentication when publishing or retrieving Pact files to/from a Pact Broker: Pact Go uses a simple log utility (logutils - https://github.com/hashicorp/logutils) to filter log messages. The CLI already contains flags to manage this, should you want to control log level in your tests, you can set it like so:

Pact Go enables consumer driven contract testing, providing a mock service and DSL for the consumer project, and interaction playback and verification for the service provider project. Consumer side Pact testing is an isolated test that ensures a given component is able to collaborate with another (remote) component. Pact will automatically start a Mock server in the background that will act as the collaborators' test double. This implies that any interactions expected on the Mock server will be validated, meaning a test will fail if all interactions were not completed, or if unexpected interactions were found: A typical consumer-side test would look something like this: If this test completed successfully, a Pact file should have been written to ./pacts/my_consumer-my_provider.json containing all of the interactions expected to occur between the Consumer and Provider. In addition to verbatim value matching, you have 3 useful matching functions in the `dsl` package that can increase expressiveness and reduce brittle test cases. Here is a complex example that shows how all 3 terms can be used together: This example will result in a response body from the mock server that looks like: See the examples in the dsl package and the matcher tests (https://github.com/you54f/pact-go/v2/blob/master/dsl/matcher_test.go) for more matching examples. NOTE: You will need to use valid Ruby regular expressions (http://ruby-doc.org/core-2.1.5/Regexp.html) and double escape backslashes. Read more about flexible matching (https://github.com/pact-foundation/pact-ruby/wiki/Regular-expressions-and-type-matching-with-Pact. Provider side Pact testing, involves verifying that the contract - the Pact file - can be satisfied by the Provider. A typical Provider side test would like something like: The `VerifyProvider` will handle all verifications, treating them as subtests and giving you granular test reporting. If you don't like this behaviour, you may call `VerifyProviderRaw` directly and handle the errors manually. Note that `PactURLs` may be a list of local pact files or remote based urls (possibly from a Pact Broker - http://docs.pact.io/documentation/sharings_pacts.html). Pact reads the specified pact files (from remote or local sources) and replays the interactions against a running Provider. If all of the interactions are met we can say that both sides of the contract are satisfied and the test passes. When validating a Provider, you have 3 options to provide the Pact files: 1. Use "PactURLs" to specify the exact set of pacts to be replayed: Options 2 and 3 are particularly useful when you want to validate that your Provider is able to meet the contracts of what's in Production and also the latest in development. See this [article](http://rea.tech/enter-the-pact-matrix-or-how-to-decouple-the-release-cycles-of-your-microservices/) for more on this strategy. Each interaction in a pact should be verified in isolation, with no context maintained from the previous interactions. So how do you test a request that requires data to exist on the provider? Provider states are how you achieve this using Pact. Provider states also allow the consumer to make the same request with different expected responses (e.g. different response codes, or the same resource with a different subset of data). States are configured on the consumer side when you issue a dsl.Given() clause with a corresponding request/response pair. Configuring the provider is a little more involved, and (currently) requires running an API endpoint to configure any [provider states](http://docs.pact.io/documentation/provider_states.html) during the verification process. The option you must provide to the dsl.VerifyRequest is: An example route using the standard Go http package might look like this: See the examples or read more at http://docs.pact.io/documentation/provider_states.html. See the Pact Broker (http://docs.pact.io/documentation/sharings_pacts.html) documentation for more details on the Broker and this article (http://rea.tech/enter-the-pact-matrix-or-how-to-decouple-the-release-cycles-of-your-microservices/) on how to make it work for you. Publishing using Go code: Publishing from the CLI: Use a cURL request like the following to PUT the pact to the right location, specifying your consumer name, provider name and consumer version. The following flags are required to use basic authentication when publishing or retrieving Pact files to/from a Pact Broker: Pact Go uses a simple log utility (logutils - https://github.com/hashicorp/logutils) to filter log messages. The CLI already contains flags to manage this, should you want to control log level in your tests, you can set it like so:

Package saml contains a partial implementation of the SAML standard in golang. SAML is a standard for identity federation, i.e. either allowing a third party to authenticate your users or allowing third parties to rely on us to authenticate their users. In SAML parlance an Identity Provider (IDP) is a service that knows how to authenticate users. A Service Provider (SP) is a service that delegates authentication to an IDP. If you are building a service where users log in with someone else's credentials, then you are a Service Provider. This package supports implementing both service providers and identity providers. The core package contains the implementation of SAML. The package samlsp provides helper middleware suitable for use in Service Provider applications. The package samlidp provides a rudimentary IDP service that is useful for testing or as a starting point for other integrations. Version 0.4.0 introduces a few breaking changes to the _samlsp_ package in order to make the package more extensible, and to clean up the interfaces a bit. The default behavior remains the same, but you can now provide interface implementations of _RequestTracker_ (which tracks pending requests), _Session_ (which handles maintaining a session) and _OnError_ which handles reporting errors. Public fields of _samlsp.Middleware_ have changed, so some usages may require adjustment. See [issue 231](https://github.com/crewjam/saml/issues/231) for details. The option to provide an IDP metadata URL has been deprecated. Instead, we recommend that you use the `FetchMetadata()` function, or fetch the metadata yourself and use the new `ParseMetadata()` function, and pass the metadata in _samlsp.Options.IDPMetadata_. Similarly, the _HTTPClient_ field is now deprecated because it was only used for fetching metdata, which is no longer directly implemented. The fields that manage how cookies are set are deprecated as well. To customize how cookies are managed, provide custom implementation of _RequestTracker_ and/or _Session_, perhaps by extending the default implementations. The deprecated fields have not been removed from the Options structure, but will be in future. In particular we have deprecated the following fields in _samlsp.Options_: - `Logger` - This was used to emit errors while validating, which is an anti-pattern. - `IDPMetadataURL` - Instead use `FetchMetadata()` - `HTTPClient` - Instead pass httpClient to FetchMetadata - `CookieMaxAge` - Instead assign a custom CookieRequestTracker or CookieSessionProvider - `CookieName` - Instead assign a custom CookieRequestTracker or CookieSessionProvider - `CookieDomain` - Instead assign a custom CookieRequestTracker or CookieSessionProvider - `CookieDomain` - Instead assign a custom CookieRequestTracker or CookieSessionProvider Let us assume we have a simple web application to protect. We'll modify this application so it uses SAML to authenticate users. ```golang package main import ( ) ``` Each service provider must have an self-signed X.509 key pair established. You can generate your own with something like this: We will use `samlsp.Middleware` to wrap the endpoint we want to protect. Middleware provides both an `http.Handler` to serve the SAML specific URLs and a set of wrappers to require the user to be logged in. We also provide the URL where the service provider can fetch the metadata from the IDP at startup. In our case, we'll use [samltest.id](https://samltest.id/), an identity provider designed for testing. ```golang package main import ( ) ``` Next we'll have to register our service provider with the identity provider to establish trust from the service provider to the IDP. For [samltest.id](https://samltest.id/), you can do something like: Navigate to https://samltest.id/upload.php and upload the file you fetched. Now you should be able to authenticate. The flow should look like this: 1. You browse to `localhost:8000/hello` 1. The middleware redirects you to `https://samltest.id/idp/profile/SAML2/Redirect/SSO` 1. samltest.id prompts you for a username and password. 1. samltest.id returns you an HTML document which contains an HTML form setup to POST to `localhost:8000/saml/acs`. The form is automatically submitted if you have javascript enabled. 1. The local service validates the response, issues a session cookie, and redirects you to the original URL, `localhost:8000/hello`. 1. This time when `localhost:8000/hello` is requested there is a valid session and so the main content is served. Please see `example/idp/` for a substantially complete example of how to use the library and helpers to be an identity provider. The SAML standard is huge and complex with many dark corners and strange, unused features. This package implements the most commonly used subset of these features required to provide a single sign on experience. The package supports at least the subset of SAML known as [interoperable SAML](http://saml2int.org). This package supports the Web SSO profile. Message flows from the service provider to the IDP are supported using the HTTP Redirect binding and the HTTP POST binding. Message flows from the IDP to the service provider are supported via the HTTP POST binding. The package can produce signed SAML assertions, and can validate both signed and encrypted SAML assertions. It does not support signed or encrypted requests. The _RelayState_ parameter allows you to pass user state information across the authentication flow. The most common use for this is to allow a user to request a deep link into your site, be redirected through the SAML login flow, and upon successful completion, be directed to the originally requested link, rather than the root. Unfortunately, _RelayState_ is less useful than it could be. Firstly, it is not authenticated, so anything you supply must be signed to avoid XSS or CSRF. Secondly, it is limited to 80 bytes in length, which precludes signing. (See section 3.6.3.1 of SAMLProfiles.) The SAML specification is a collection of PDFs (sadly): - [SAMLCore](http://docs.oasis-open.org/security/saml/v2.0/saml-core-2.0-os.pdf) defines data types. - [SAMLBindings](http://docs.oasis-open.org/security/saml/v2.0/saml-bindings-2.0-os.pdf) defines the details of the HTTP requests in play. - [SAMLProfiles](http://docs.oasis-open.org/security/saml/v2.0/saml-profiles-2.0-os.pdf) describes data flows. - [SAMLConformance](http://docs.oasis-open.org/security/saml/v2.0/saml-conformance-2.0-os.pdf) includes a support matrix for various parts of the protocol. [SAMLtest](https://samltest.id/) is a testing ground for SAML service and identity providers. Please do not report security issues in the issue tracker. Rather, please contact me directly at ross@kndr.org ([PGP Key `78B6038B3B9DFB88`](https://keybase.io/crewjam)).

shortinette is the core framework for managing and automating the process of grading coding bootcamps (Shorts). It provides a comprehensive set of tools for running and testing student submissions across various programming languages. The shortinette package is composed of several sub-packages, each responsible for a specific aspect of the grading pipeline: `logger`: Handles logging for the framework, including general informational messages, error reporting, and trace logging for feedback on individual submissions. This package ensures that all important events and errors are captured for debugging and auditing purposes. `requirements`: Validates the necessary environment variables and dependencies required by the framework. This includes checking for essential configuration values in a `.env` file and ensuring that all necessary tools (e.g., Docker images) are available before grading begins. `testutils`: Provides utility functions for compiling and running code submissions. This includes functions for compiling Rust code, running executables with various options (such as timeouts and real-time output), and manipulating files. The utility functions are designed to handle the intricacies of running untrusted student code safely and efficiently. `git`: Manages interactions with GitHub, including cloning repositories, managing collaborators, and uploading files. This package abstracts the GitHub API to simplify common tasks such as adding collaborators to repositories, creating branches, and pushing code or data to specific locations in a repository. `exercise`: Defines the structure and behavior of individual coding exercises. This includes specifying the files that students are allowed to submit, the expected output, and the functions to be tested. The `exercise` package provides the framework for setting up exercises, running tests, and reporting results. `module`: Organizes exercises into modules, allowing for the grouping of related exercises into a coherent curriculum. The `module` package handles the execution of all exercises within a module, aggregates results, and manages the overall grading process. `webhook`: Enables automatic grading triggered by GitHub webhooks. This allows for a fully automated workflow where student submissions are graded as soon as they are pushed to a specific branch in a GitHub repository. `short`: The central orchestrator of the grading process, integrating all sub-packages into a cohesive system. The `short` package handles the setup and teardown of grading environments, manages the execution of modules and exercises, and ensures that all results are properly recorded and reported.

Package be is a minimalist test assertion helper library. Tests usually should not fail. When they do fail, the failure should be repeatable. Therefore, it doesn't make sense to spend a lot of time writing good test messages. (This is unlike error messages, which should happen fairly often, and in production, irrepeatably.) Package be is designed to simply fail a test quickly and quietly if a condition is not met with a reference to the line number of the failing test. If the reason for having the test is not immediately clear from context, you can write a comment, like normal code. If you do need more extensive reporting to figure out why a test is failing, use be.DebugLog or be.Debug to capture more information. Most tests just need simple equality testing, which is handled by be.Equal (for comparable types), be.AllEqual (for slices of comparable types), and be.DeepEqual (which relies on reflect.DeepEqual). Another common test is that a string or byte slice should contain or not some substring, which is handled by be.In and be.NotIn. Rather than package be providing every possible test helper, you are encouraged to write your own advanced helpers for use with be.True, while package be takes away the drudgery of writing yet another simple func nilErr(t *testing.T, err) { ... }. The github.com/carlmjohnson/be/testfile subpackage has functions that make it easy to write file-based tests that ensure that the output of some transformation matches a golden file. Subtests can automatically be run for all files matching a glob pattern, such as testfile.Run(t, "testdata/*/input.txt", ...). If the test fails, the failure output will be written to a file, such as "testdata/basic-test/-failed-output.txt", and then the output can be examined via diff testing with standard tools. Set the environmental variable TESTFILE_UPDATE to update the golden file. Every test in the be package requires a testing.TB as its first argument. There are various clever ways to get the testing.TB implicitly,* but package be is designed to be simple and explicit, so it's easiest to just always pass in a testing.TB the boring way.

Package gofpdf implements a PDF document generator with high level support for text, drawing and images. - UTF-8 support - Choice of measurement unit, page format and margins - Page header and footer management - Automatic page breaks, line breaks, and text justification - Inclusion of JPEG, PNG, GIF, TIFF and basic path-only SVG images - Colors, gradients and alpha channel transparency - Outline bookmarks - Internal and external links - TrueType, Type1 and encoding support - Page compression - Lines, Bézier curves, arcs, and ellipses - Rotation, scaling, skewing, translation, and mirroring - Clipping - Document protection - Layers - Templates - Barcodes - Charting facility - Import PDFs as templates gofpdf has no dependencies other than the Go standard library. All tests pass on Linux, Mac and Windows platforms. gofpdf supports UTF-8 TrueType fonts and “right-to-left” languages. Note that Chinese, Japanese, and Korean characters may not be included in many general purpose fonts. For these languages, a specialized font (for example, NotoSansSC for simplified Chinese) can be used. Also, support is provided to automatically translate UTF-8 runes to code page encodings for languages that have fewer than 256 glyphs. To install the package on your system, run Later, to receive updates, run The following Go code generates a simple PDF file. See the functions in the fpdf_test.go file (shown as examples in this documentation) for more advanced PDF examples. If an error occurs in an Fpdf method, an internal error field is set. After this occurs, Fpdf method calls typically return without performing any operations and the error state is retained. This error management scheme facilitates PDF generation since individual method calls do not need to be examined for failure; it is generally sufficient to wait until after Output() is called. For the same reason, if an error occurs in the calling application during PDF generation, it may be desirable for the application to transfer the error to the Fpdf instance by calling the SetError() method or the SetErrorf() method. At any time during the life cycle of the Fpdf instance, the error state can be determined with a call to Ok() or Err(). The error itself can be retrieved with a call to Error(). This package is a relatively straightforward translation from the original FPDF library written in PHP (despite the caveat in the introduction to Effective Go). The API names have been retained even though the Go idiom would suggest otherwise (for example, pdf.GetX() is used rather than simply pdf.X()). The similarity of the two libraries makes the original FPDF website a good source of information. It includes a forum and FAQ. However, some internal changes have been made. Page content is built up using buffers (of type bytes.Buffer) rather than repeated string concatenation. Errors are handled as explained above rather than panicking. Output is generated through an interface of type io.Writer or io.WriteCloser. A number of the original PHP methods behave differently based on the type of the arguments that are passed to them; in these cases additional methods have been exported to provide similar functionality. Font definition files are produced in JSON rather than PHP. A side effect of running go test ./... is the production of a number of example PDFs. These can be found in the gofpdf/pdf directory after the tests complete. Please note that these examples run in the context of a test. In order run an example as a standalone application, you’ll need to examine fpdf_test.go for some helper routines, for example exampleFilename() and summary(). Example PDFs can be compared with reference copies in order to verify that they have been generated as expected. This comparison will be performed if a PDF with the same name as the example PDF is placed in the gofpdf/pdf/reference directory and if the third argument to ComparePDFFiles() in internal/example/example.go is true. (By default it is false.) The routine that summarizes an example will look for this file and, if found, will call ComparePDFFiles() to check the example PDF for equality with its reference PDF. If differences exist between the two files they will be printed to standard output and the test will fail. If the reference file is missing, the comparison is considered to succeed. In order to successfully compare two PDFs, the placement of internal resources must be consistent and the internal creation timestamps must be the same. To do this, the methods SetCatalogSort() and SetCreationDate() need to be called for both files. This is done automatically for all examples. Nothing special is required to use the standard PDF fonts (courier, helvetica, times, zapfdingbats) in your documents other than calling SetFont(). You should use AddUTF8Font() or AddUTF8FontFromBytes() to add a TrueType UTF-8 encoded font. Use RTL() and LTR() methods switch between “right-to-left” and “left-to-right” mode. In order to use a different non-UTF-8 TrueType or Type1 font, you will need to generate a font definition file and, if the font will be embedded into PDFs, a compressed version of the font file. This is done by calling the MakeFont function or using the included makefont command line utility. To create the utility, cd into the makefont subdirectory and run “go build”. This will produce a standalone executable named makefont. Select the appropriate encoding file from the font subdirectory and run the command as in the following example. In your PDF generation code, call AddFont() to load the font and, as with the standard fonts, SetFont() to begin using it. Most examples, including the package example, demonstrate this method. Good sources of free, open-source fonts include Google Fonts and DejaVu Fonts. The draw2d package is a two dimensional vector graphics library that can generate output in different forms. It uses gofpdf for its document production mode. gofpdf is a global community effort and you are invited to make it even better. If you have implemented a new feature or corrected a problem, please consider contributing your change to the project. A contribution that does not directly pertain to the core functionality of gofpdf should be placed in its own directory directly beneath the contrib directory. Here are guidelines for making submissions. Your change should - be compatible with the MIT License - be properly documented - be formatted with go fmt - include an example in fpdf_test.go if appropriate - conform to the standards of golint and go vet, that is, golint . and go vet . should not generate any warnings - not diminish test coverage Pull requests are the preferred means of accepting your changes. gofpdf is released under the MIT License. It is copyrighted by Kurt Jung and the contributors acknowledged below. This package’s code and documentation are closely derived from the FPDF library created by Olivier Plathey, and a number of font and image resources are copied directly from it. Bruno Michel has provided valuable assistance with the code. Drawing support is adapted from the FPDF geometric figures script by David Hernández Sanz. Transparency support is adapted from the FPDF transparency script by Martin Hall-May. Support for gradients and clipping is adapted from FPDF scripts by Andreas Würmser. Support for outline bookmarks is adapted from Olivier Plathey by Manuel Cornes. Layer support is adapted from Olivier Plathey. Support for transformations is adapted from the FPDF transformation script by Moritz Wagner and Andreas Würmser. PDF protection is adapted from the work of Klemen Vodopivec for the FPDF product. Lawrence Kesteloot provided code to allow an image’s extent to be determined prior to placement. Support for vertical alignment within a cell was provided by Stefan Schroeder. Ivan Daniluk generalized the font and image loading code to use the Reader interface while maintaining backward compatibility. Anthony Starks provided code for the Polygon function. Robert Lillack provided the Beziergon function and corrected some naming issues with the internal curve function. Claudio Felber provided implementations for dashed line drawing and generalized font loading. Stani Michiels provided support for multi-segment path drawing with smooth line joins, line join styles, enhanced fill modes, and has helped greatly with package presentation and tests. Templating is adapted by Marcus Downing from the FPDF_Tpl library created by Jan Slabon and Setasign. Jelmer Snoeck contributed packages that generate a variety of barcodes and help with registering images on the web. Jelmer Snoek and Guillermo Pascual augmented the basic HTML functionality with aligned text. Kent Quirk implemented backwards-compatible support for reading DPI from images that support it, and for setting DPI manually and then having it properly taken into account when calculating image size. Paulo Coutinho provided support for static embedded fonts. Dan Meyers added support for embedded JavaScript. David Fish added a generic alias-replacement function to enable, among other things, table of contents functionality. Andy Bakun identified and corrected a problem in which the internal catalogs were not sorted stably. Paul Montag added encoding and decoding functionality for templates, including images that are embedded in templates; this allows templates to be stored independently of gofpdf. Paul also added support for page boxes used in printing PDF documents. Wojciech Matusiak added supported for word spacing. Artem Korotkiy added support of UTF-8 fonts. Dave Barnes added support for imported objects and templates. Brigham Thompson added support for rounded rectangles. Joe Westcott added underline functionality and optimized image storage. Benoit KUGLER contributed support for rectangles with corners of unequal radius, modification times, and for file attachments and annotations. - Remove all legacy code page font support; use UTF-8 exclusively - Improve test coverage as reported by the coverage tool. Example demonstrates the generation of a simple PDF document. Note that since only core fonts are used (in this case Arial, a synonym for Helvetica), an empty string can be specified for the font directory in the call to New(). Note also that the example.Filename() and example.Summary() functions belong to a separate, internal package and are not part of the gofpdf library. If an error occurs at some point during the construction of the document, subsequent method calls exit immediately and the error is finally retrieved with the output call where it can be handled by the application.

Package fault provides standard http middleware for fault injection in go. Use the fault package to inject faults into the http request path of your service. Faults work by modifying and/or delaying your service's http responses. Place the Fault middleware high enough in the chain that it can act quickly, but after any other middlewares that should complete before fault injection (auth, redirects, etc...). The type and severity of injected faults is controlled by options passed to NewFault(Injector, Options). NewFault must be passed an Injector, which is an interface that holds the actual fault injection code in Injector.Handler. The Fault wraps Injector.Handler in another Fault.Handler that applies generic Fault logic (such as what % of requests to run the Injector on) to the Injector. Make sure you use the NewFault() and NewTypeInjector() constructors to create valid Faults and Injectors. There are three main Injectors provided by the fault package: Use fault.RejectInjector to immediately return an empty response. For example, a curl for a rejected response will produce: Use fault.ErrorInjector to immediately return a valid http status code of your choosing along with the standard HTTP response body for that code. For example, you can return a 200, 301, 418, 500, or any other valid status code to test how your clients respond to different statuses. Pass the WithStatusText() option to customize the response text. Use fault.SlowInjector to wait a configured time.Duration before proceeding with the request. For example, you can use the SlowInjector to add a 10ms delay to your requests. Use fault.RandomInjector to randomly choose one of the above faults to inject. Pass a list of Injector to fault.NewRandomInjector and when RandomInjector is evaluated it will randomly run one of the injectors that you passed. It is easy to combine any of the Injectors into a chained action. There are two ways you might want to combine Injectors. First, you can create separate Faults for each Injector that are sequential but independent of each other. For example, you can chain Faults such that 1% of requests will return a 500 error and another 1% of requests will be rejected. Second, you might want to combine Faults such that 1% of requests will be slowed for 10ms and then rejected. You want these Faults to depend on each other. For this use the special ChainInjector, which consolidates any number of Injectors into a single Injector that runs each of the provided Injectors sequentially. When you add the ChainInjector to a Fault the entire chain will always execute together. The NewFault() constructor has WithPathBlocklist() and WithPathAllowlist() options. Any path you include in the PathBlocklist will never have faults run against it. With PathAllowlist, if you provide a non-empty list then faults will not be run against any paths except those specified in PathAllowlist. The PathBlocklist take priority over the PathAllowlist, a path in both lists will never have a fault run against it. The paths that you include must match exactly the path in req.URL.Path, including leading and trailing slashes. Simmilarly, you may also use WithHeaderBlocklist() and WithHeaderAllowlist() to block or allow faults based on a map of header keys to values. These lists behave in the same way as the path allowlists and blocklists except that they operate on headers. Header equality is determined using http.Header.Get(key) which automatically canonicalizes your keys and does not support multi-value headers. Keep these limitations in mind when working with header allowlists and blocklists. Specifying very large lists of paths or headers may cause memory or performance issues. If you're running into these problems you should instead consider using your http router to enable the middleware on only a subset of your routes. The fault package provides an Injector interface and you can satisfy that interface to provide your own Injector. Use custom injectors to add additional logic to the package-provided injectors or to create your own completely new Injector that can still be managed by a Fault. The package provides a Reporter interface that can be added to Faults and Injectors using the WithReporter option. A Reporter will receive events when the state of the Injector changes. For example, Reporter.Report(InjectorName, StateStarted) is run at the beginning of all Injectors. The Reporter is meant to be provided by the consumer of the package and integrate with services like stats and logging. The default Reporter throws away all events. By default all randomness is seeded with defaultRandSeed(1), the same default as math/rand. This helps you reproduce any errors you see when running an Injector. If you prefer, you can also customize the seed passing WithRandSeed() to NewFault and NewRandomInjector. Some Injectors support customizing the functions they use to run their injections. You can take advantage of these options to add your own logic to an existing Injector instead of creating your own. For example, modify the SlowInjector function to slow in a rancom distribution instead of for a fixed duration. Be careful when you use these options that your return values fall within the same range of values expected by the default functions to avoid panics or other undesirable begavior. Customize the function a Fault uses to determine participation (default: rand.Float32) by passing WithRandFloat32Func() to NewFault(). Customize the function a RandomInjector uses to choose which injector to run (default: rand.Intn) by passing WithRandIntFunc() to NewRandomInjector(). Customize the function a SlowInjector uses to wait (default: time.Sleep) by passing WithSlowFunc() to NewSlowInjector(). Configuration for the fault package is done through options passed to NewFault and NewInjector. Once a Fault is created its enabled state and participation percentage can be updated with SetEnabled() and SetParticipation(). There is no other way to manage configuration for the package. It is up to the user of the fault package to manage how the options are generated. Common options are feature flags, environment variables, or code changes in deploys. Example is a package-level documentation example.

Package poplar provides a set of tools for running and managing results for benchmarks in go. The Report type defines infrastructure that any project can use to create and upload test results to a cedar data service regardless of the execution model without needing to write client code that interacts with cedars RPC protocol. Additionally, poplar defines some-local only RPC interfaces on top of the github.com/mongodb/ftdc and github.com/mongodb/ftdc/events packages to support generating data payloads in real time. A Report structure structure, in YAML, would generally resemble the following: See the documentation of the Report, Test, TestInfo, TestArtifact and TestMetrics format for more information. Poplar contains a toolkit for running benchmark tests and collecting rich intra-run data from those tests. Consider the following example: You can also run a benchmark suite using go's standard library, as in: Each test in the suite is reported as a separate sub-benchmark. In addition to suites, workloads provide a way to define concurrent and parallel workloads that execute multiple instances of a single test at a time. Workloads function like an extended case of suites. You must specify either a single case or a list of sub-workloads. The executors for workloads run the groups of these tests in parallel, with the degree of parallelism controlled by the Instances value. When you specify a list of sub-workloads, poplar will execute a group of these workloads in parallel, but the workloads themselves are run sequentially, potentially with their own parallelism. Benchmark suites and workloads both report data in the same format. You can also execute workloads using the Standard method, as with suites, using default Go methods for running benchmarks.