Introduction
This program can compare the strength of two Go (围棋, 囲碁, 바둑) engines that are GTP-compatible by running a batch of games between them. It can also compare the strength under different options of the same engine.
Requirements
- Node.js 8 or higher installed.
Usage
Add a file config.json
, then run:
npx go-engine-tester config.json
config.json
example:
{
"first": {
"command": "./leelaz",
"args": ["-w", "network-173.gz", "--gtp", "--noponder"],
"initialGtpCommands": [
{"name": "komi", "args": ["7.5"]},
{"name": "time_settings", "args": ["0", "0", "1"]},
{"name": "boardsize", "args": ["19"]}
]
},
"second": {
"command": "./leelaz",
"args": ["-w", "network-157.gz", "--gtp", "--noponder"],
"initialGtpCommands": [
{"name": "komi", "args": ["7.5"]},
{"name": "time_settings", "args": ["0", "0", "1"]},
{"name": "boardsize", "args": ["19"]}
]
},
"times": 30
}
This will play 30 games between the two engines, alternating black and white for each game, and finally print the result. Note that the path in command
is relative to the working directory not the config file. Also note that on Windows, use \\
not \
for path separator, or just use /
.
For details of GTP commands such as time_settings
, click here.
There's an optional spawnOptions
property, so that it could pass environment variables to the engine processes. For example:
{
"first": {
"command": "./leelaz",
"args": ["-w", "network-173.gz", "--gtp", "--noponder"],
"initialGtpCommands": [
{"name": "komi", "args": ["7.5"]},
{"name": "time_settings", "args": ["0", "0", "1"]},
{"name": "boardsize", "args": ["19"]}
],
"spawnOptions": {
"env": {
"DYLD_LIBRARY_PATH": "/Users/your-name/boost_1_67_0/lib"
}
}
},
"second": {
"command": "./leelaz",
"args": ["-w", "network-157.gz", "--gtp", "--noponder"],
"initialGtpCommands": [
{"name": "komi", "args": ["7.5"]},
{"name": "time_settings", "args": ["0", "0", "1"]},
{"name": "boardsize", "args": ["19"]}
],
"spawnOptions": {
"env": {
"DYLD_LIBRARY_PATH": "/Users/your-name/boost_1_67_0/lib"
}
}
},
"times": 30
}
For details of spawnOptions
, click here.
FAQ
Q: Why does it restart the process for each game? Process initialization can be slow!
A: This ensures that each game is treated equally. A lot of programs are so smart that they can cache the previous calculation to speed up calculations for the next game. We must avoid that, otherwise the result may be incorrect.