orchestrator-core
Advanced tools
Comparing version 4.10.0-dev.20200811.9356f4b to 4.10.0-dev.20200813.b4c114e
@@ -5,3 +5,3 @@ { | ||
"description": "Orchestrator for Node.js", | ||
"version": "4.10.0-dev.20200811.9356f4b", | ||
"version": "4.10.0-dev.20200813.b4c114e", | ||
"license": "MIT", | ||
@@ -8,0 +8,0 @@ "keywords": [ |
158
README.md
# Orchestrator for node.js | ||
Currently, all the build is performed through node-gyp. Gyp generates Visual Studio 2017 project files on the fly, and there currently is nothing checked into the `oc.sln` file. | ||
>TODO: add a ceremonial/sacrificial C++ project to the `oc` solution to facilitate debugging. It won't be the actual project file used to generate the package! | ||
## Install | ||
```bash | ||
# Go to node directory. | ||
cd \src\TScience\Orchestrator\oc\oc_node_authoring | ||
# Build debug | ||
npm install . # Note period | ||
(or npm install -g .) # Global | ||
``` | ||
## Building | ||
- As a pre-requisite, build `oc.sln` as `Release`. | ||
```bash | ||
# Set up environment for 64-bit native | ||
"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\vcvars64.bat" | ||
# Build the solution as Release | x64 | ||
MsBuild.exe D:\src\TScience\Orchestrator\oc\oc.sln /t:Build /p:Configuration=Release /p:Platform=x64 | ||
``` | ||
- Build the node.js integration layer | ||
- See [node-gyp docs](https://github.com/nodejs/node-gyp#on-windows) for more info. | ||
```bash | ||
# Go to node directory. | ||
cd \src\TScience\Orchestrator\oc\oc_node_authoring | ||
# Set up visual studio for node-gyp | ||
npm config set msvs_version 2019 | ||
# Build debug | ||
npm run rebuild:dev | ||
``` | ||
- You can look at what the `rebuild:dev` command is doing by looking at `package.json` | ||
- You can see how the Visual Studio project file is being generated by inspecting the `binding.gyp` file. | ||
## Run tests | ||
```bash | ||
# Go to node directory. | ||
cd \src\TScience\Orchestrator\oc\oc_node_authoring | ||
# Run tests | ||
npm run test | ||
``` | ||
## Debugging | ||
There are tests and a sample. Most likely you will be debugging one of these. To debug, you can use Visual Studio or Vscode. | ||
### Output Directory | ||
To Debug C++ code in oc_node_authoring, I had to perform the following. | ||
Node-pre-gyp is responsible for the process of pulling down the "tarball" (.tgz) file and unarchiving and placing the binaries. This process is a direct stand-in for the "normal" path (node-gyp) of building the binaries on your machine when you perform "npm install" - it's just placing the binaries in the spot that the build process would normally copy the output. To that end, if you use something like "Visual Studio" or other platform tools, the output directory is something like "./build/Debug" or "./build/Release". | ||
Currently, in the build process, we truncate the build output directory so the user experience (ie, node.js code) can load the library without directory structure. For example: | ||
`const oc = require('orchestrator-core.node') <= "Good"` | ||
vs. | ||
`const oc = require('./build/Release/orchestrator-core.node') <= "Bad"` | ||
However, truncating the build output to the local directory (for some reason) causes havoc when attempting to debug on Windows with Visual Studio/Visual Studio Code. | ||
To debug C++ code, try the following: | ||
**package.json** | ||
```json | ||
"binary": { | ||
"module_name": "orchestrator-core", | ||
"module_path": "./build/Debug", | ||
"host": "https://bfsmodelsscratch.azureedge.net/native" | ||
}, | ||
``` | ||
**binding.gyp** | ||
```json | ||
# "target_defaults": { | ||
# "msvs_configuration_attributes": { | ||
# "OutputDirectory": "" | ||
# } | ||
# }, | ||
``` | ||
**oc_app.js** | ||
```js | ||
const oc = require('./build/Debug/orchestrator-core.node'); | ||
``` | ||
Rebuild (See below for more details): | ||
```bash | ||
npm run rebuild:dev | ||
``` | ||
This shifts the build to `./build/Debug`and enables you to debug C++ code. | ||
### Visual Studio | ||
Here's an example configuring Visual Studio using the sample. | ||
- These settings are using `oc_abi` but you can set breakpoints in code in the node.js interop layer. | ||
- Make sure this configuration is for **Release | x64** | ||
- Set the command to where your **node executable** is found. | ||
- Set command arguments to **oc_app.js** | ||
- Set Working directory to TScience\Orchestrator\oc\oc_node | ||
 | ||
Similarly, you can configure for **tests** - see the configuration for Visual Studio Code below. | ||
### Visual Studio Code | ||
In `launch.json` (Click Shift-Ctrl-D and click on the gear symbol), you can add something like the following (fixing the paths). | ||
> Note: This sets up the C++ debugger (cppvsdbg). | ||
```json | ||
{ | ||
"type": "cppvsdbg", | ||
"request": "launch", | ||
"name": "Launch nodejs app via NPM", | ||
"program": "C:\\Program Files\\nodejs\\node.exe", | ||
"args": [ | ||
"oc_app.js" | ||
], | ||
"cwd": "d:\\src\\TScience\\Orchestrator\\oc\\oc_node_authoring" | ||
}, | ||
{ | ||
"type": "cppvsdbg", | ||
"request": "launch", | ||
"name": "Launch MOCHA test via NPM", | ||
"program": "C:\\Program Files\\nodejs\\node.exe", | ||
"stopAtEntry": false, | ||
"args": [ | ||
".\\node_modules\\mocha\\bin\\mocha", | ||
"tests/", | ||
// "-g", | ||
// "Remove" | ||
], | ||
"cwd": "d:\\src\\TScience\\Orchestrator\\oc\\oc_node_authoring", | ||
"externalConsole": false | ||
}, | ||
``` | ||
**Note**: `-g` in the MOCHA test stands for grep. You can filter specific tests. For example: "Remove" | ||
```js | ||
it('Remove example.', async function () { | ||
// Arrange | ||
var orc = new oc.Orchestrator(); | ||
``` | ||
Bot Framework Orchestrator component. |
@@ -5,3 +5,4 @@ // Copyright (c) Microsoft Corporation. All rights reserved. | ||
const assert = require('assert'); | ||
const oc = require('../orchestrator-core.node'); | ||
const oc = require('../build/orchestrator-core.node'); | ||
const path = require('path'); | ||
@@ -22,2 +23,29 @@ // Finds onnx model | ||
var model_dir = null; | ||
function validateConfigFile(model_dir) { | ||
// console.log('ONNX: Found onnx file in ' + model_dir); | ||
try{ | ||
var config_file = path.join(model_dir, 'config.json'); | ||
// console.log(' => Seeing if ' + config_file + ' exists..') | ||
if (fs.existsSync(config_file)) { | ||
// console.log(' This file exists: ' + config_file); | ||
var jsonData = fs.readFileSync(config_file, 'utf8'); | ||
const config = JSON.parse(jsonData); | ||
if (config.hasOwnProperty('VocabFile') && config.hasOwnProperty('MergeFile') | ||
&& config.hasOwnProperty('ModelFile') && config.hasOwnProperty('Version')) { | ||
// console.log(' => VALID!'); | ||
return true; | ||
} | ||
// console.log(' => Invalid config!'); | ||
return false; | ||
} | ||
// console.log(' => Does not exist: ' + config_file ); | ||
return false; | ||
} catch(err) { | ||
// console.log(' => Does not exist EXCEPT!: ' + err ); | ||
return false; | ||
} | ||
return false; | ||
} | ||
function findModel(cwd) { | ||
@@ -36,8 +64,10 @@ | ||
if (stat.isFile() && dirInner.endsWith(".onnx")) { | ||
var dtemp = dirInner.split(sep); | ||
dtemp.pop(); | ||
dirInner = dtemp.join(sep); | ||
model_dir = dirInner; | ||
return true; | ||
if (validateConfigFile(dirInner)) { | ||
model_dir = dirInner; | ||
return true; | ||
} | ||
return false; | ||
} | ||
@@ -60,3 +90,5 @@ return false; | ||
var onnx_dir = findOnnxDir(3); | ||
const load_result = await this.orchestrator.loadAsync(onnx_dir); | ||
const load_result = this.orchestrator.load(onnx_dir); | ||
assert.strictEqual(load_result, true); | ||
@@ -88,2 +120,15 @@ }); | ||
}); | ||
it('should create a LabelResolver without a NLR nodel, but with a BLU snapshot.', async function () { | ||
this.timeout(10000); | ||
const orchestrator = new oc.Orchestrator(); | ||
assert.ok(orchestrator.load(), 'cannot load without a NLR'); | ||
const snapshotPath = path.join(__dirname, 'snapshot', 'Email.blu'); | ||
assert.ok(fs.existsSync(snapshotPath), `snapshot file ${snapshotPath} does not exist`); | ||
const snapshotContent = new TextEncoder().encode(fs.readFileSync(snapshotPath)); | ||
assert.ok(snapshotContent.length === 140732, `snapshot content length is ${snapshotContent.length}`); | ||
const labelResolver = await orchestrator.createLabelResolver(snapshotContent); | ||
assert.ok(labelResolver !== undefined, 'cannot create a labelResolver object'); | ||
const examples = labelResolver.getExamples(); | ||
assert.ok(examples.length === 601, `#examples is ${examples.length}`); | ||
}); | ||
it('should fail load bad path.', async function () { | ||
@@ -96,3 +141,19 @@ this.timeout(10000); | ||
}); | ||
it('should fail load - invalid config file.', async function () { | ||
this.timeout(10000); | ||
expected = Error; | ||
const nlr = new oc.Orchestrator(); | ||
const bad_config = path.join(__dirname, 'bad_config'); | ||
assert.throws(() => nlr.load(bad_config), /^TypeError: Invalid config JSON syntax in config file: .*/); | ||
}); | ||
it('should fail load - no config file.', async function () { | ||
this.timeout(10000); | ||
expected = Error; | ||
const nlr = new oc.Orchestrator(); | ||
const no_config = path.join(__dirname, 'no_config'); | ||
assert.throws(() => { nlr.load(no_config); }, /^TypeError: Invalid or non-existent JSON config file: .*/); | ||
}); | ||
it('Add example.', async function () { | ||
@@ -99,0 +160,0 @@ // Arrange |
Sorry, the diff of this file is not supported yet
462067
27
326
3