swarm-bat

BAT - batter the bugs out of your apps

BAT is a simplistic blackbox testing tool. BAT is:

• stream-based (TCP, WebSocket or local pipes, no HTTP),
• line-based (like classic internet/Unix protocols, no JSON, no XML),
• implementation-agnostic (just i/o, no API calls).

BAT feeds the prescribed input into a stream, then listens to the output and compares it to the expected value. The test script format is extremely simple:

; comment
>input
<expected output

BAT's predecessors have been extremely useful in testing network servers in concurrent input/output scenarios. Then, the testing tool pretends to be several clients:

; client 1 says hi, server responds
client1> Hello
client1< Hi Joe
; client 2 joins
client2> Hello
client2< Hi Mike

Blackbox testing is particularly handy when you test protocol compliance of several implementations: their APIs differ, but tests are the same.

Usage

• bat -e command simple_script.batt
• bat -c host:port multistream_script.batt
• bat -l 127.0.0.1:12345 server_script.batt
• bat -e new_version -r new_script.batt old_script.batt

see test/ for examples of API and CLI usage, .batt scripts.

Options

• -e test an executable (stdin/stdout)
• -r record de-facto output to a new test script
• -i don't stop on errors, just log them
• -v comment on every step
• -L ignore empty lines
• -C ignore case
• -d max delay (let'em think); default 500ms
• -O output lines may go in any order
• --whitespace collapse|ignore|exact|count

Codes

• 0 OK
• 1 no match
• 2 script error
• 3 argument error
• 4 io error

TODO

C version for billion-line tests