Huge News!Announcing our $40M Series B led by Abstract Ventures.Learn More →
Socket
Sign inDemoInstall
Socket
Sign inDemoInstall

mdast-util-to-nlcst

Package Overview
Dependencies
Maintainers
1
Versions
25
Alerts
File Explorer

Advanced tools

License
Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

mdast-util-to-nlcst - npm Package Compare versions

Comparing version 2.0.1 to 3.0.0

25

history.md

@@ -5,2 +5,7 @@ <!--remark setext-->

3.0.0 / 2016-06-22
==================
* Refactor module for changes in remark, retext ([`7d88f57`](https://github.com/wooorm/mdast-util-to-nlcst/commit/7d88f57))
2.0.1 / 2016-01-12

@@ -17,25 +22,5 @@ ==================

* Remove support for bower ([e8a41f6](https://github.com/wooorm/mdast-util-to-nlcst/commit/e8a41f6))
* Refactor npm test target ([379ee93](https://github.com/wooorm/mdast-util-to-nlcst/commit/379ee93))
* Refactor to replace mocha with tape ([1efd2ca](https://github.com/wooorm/mdast-util-to-nlcst/commit/1efd2ca))
* Update dependencies, dev-dependencies ([60235cd](https://github.com/wooorm/mdast-util-to-nlcst/commit/60235cd))
* Update `mdast` to `remark` ([ed96d00](https://github.com/wooorm/mdast-util-to-nlcst/commit/ed96d00))
1.0.0 / 2015-09-18
==================
* Refactor module ([0dd7eb9](https://github.com/wooorm/mdast-util-to-nlcst/commit/0dd7eb9))
0.2.1 / 2015-08-15
==================
* Fix mis-labeled dependencies ([2a81939](https://github.com/wooorm/mdast-util-to-nlcst/commit/2a81939))
* Update mdast ([d0939b8](https://github.com/wooorm/mdast-util-to-nlcst/commit/d0939b8))
0.2.0 / 2015-08-11
==================
* Refactor API for upcoming unified changes ([c9b2db7](https://github.com/wooorm/mdast-util-to-nlcst/commit/c9b2db7))
* Refactor for changes in mdast ([f5ded15](https://github.com/wooorm/mdast-util-to-nlcst/commit/f5ded15))
0.1.0 / 2015-07-25
==================

184

index.js
/**
* @author Titus Wormer
* @copyright 2015-2016 Titus Wormer
* @copyright 2015 Titus Wormer
* @license MIT
* @module mdast:util:to-nlcst
* @fileoverview Create a Natural Language Concrete Syntax Tree from
* a Markdown Abstract Syntax Tree.
* @module mdast:to-nlcst
* @fileoverview Transform MDAST to NLCST.
*/

@@ -14,60 +13,20 @@

/*
* Dependencies.
*/
var range = require('remark-range');
/* Dependencies. */
var repeat = require('repeat-string');
var vfileLocation = require('vfile-location');
var toString = require('nlcst-to-string');
var repeat = require('repeat-string');
/*
* Map of ignored mdast nodes: nodes which have no (simple)
* representation in NLCST.
*/
/* Map of ignored mdast nodes: nodes which have no (simple)
* representation in NLCST. */
var IGNORE = {
'horizontalRule': true,
'table': true,
'tableRow': true,
'tableCell': true
horizontalRule: true,
table: true,
tableRow: true,
tableCell: true
};
/*
* Constants.
*/
/* Constants. */
var C_NEWLINE = '\n';
/**
* Create an position object for `offset` in `file`.
*
* @param {number} offset - Offset in `file`.
* @param {File} file - Virtual file.
* @return {Object} - Positional information.
*/
function position(offset, file) {
var pos = file.offsetToPosition(offset);
pos.offset = offset;
return pos;
}
/**
* Create a location object for `start` and `end` in
* `file`.
*
* @param {number} start - Starting offset in `file`.
* @param {number} end - Ending offset in `file`.
* @param {File} file - Virtual file.
* @return {Object} - Location information.
*/
function location(start, end, file) {
return {
'start': position(start, file),
'end': position(end, file)
};
}
/**
* Patch a position on each node in `nodes`.

@@ -82,7 +41,7 @@ * `offset` is the offset in `file` this run of content

* @param {Array.<NLCSTNode>} nodes - NLCST nodes.
* @param {File} file - Virtual file.
* @param {Object} location - Bound location info.
* @param {number} offset - Starting offset for `nodes`.
* @return {Array.<NLCSTNode>} - `nodes`.
*/
function patch(nodes, file, offset) {
function patch(nodes, location, offset) {
var length = nodes.length;

@@ -100,3 +59,3 @@ var index = -1;

if (children) {
patch(children, file, start);
patch(children, location, start);
}

@@ -106,3 +65,6 @@

node.position = location(start, end, file);
node.position = {
start: location.toPosition(start),
end: location.toPosition(end)
}

@@ -115,9 +77,2 @@ start = end;

/*
* Transformers.
*/
var all;
var one;
/**

@@ -128,2 +83,3 @@ * Convert all nodes in `parent` (mdast) into NLCST.

* @param {File} file - Virtual file.
* @param {Object} location - Bound location info.
* @param {Parser} parser - NLCST parser.

@@ -133,3 +89,3 @@ * @return {Array.<NLCSTNode>} - Concatenation of calling

*/
all = function (parent, file, parser) {
function all(parent, file, location, parser) {
var children = parent.children;

@@ -156,3 +112,3 @@ var length = children && children.length;

patch([child], file, prevOffset);
patch([child], location, prevOffset);

@@ -166,3 +122,3 @@ if (child.value.length < 2) {

child = one(node, index, parent, file, parser);
child = one(node, index, parent, file, location, parser);

@@ -178,3 +134,3 @@ if (child) {

return result;
};
}

@@ -185,5 +141,6 @@ /**

* @param {MDASTNode} node - Node.
* @param {number} index - Position of `node` in `parent`.
* @param {MDASTNode} parent - Parent node of `node`.
* @param {number?} index - Position of `node` in `parent`.
* @param {MDASTNode?} parent - Parent node of `node`.
* @param {File} file - Virtual file.
* @param {Object} location - Bound location info.
* @param {Parser} parser - NLCST parser.

@@ -193,7 +150,7 @@ * @return {Array.<NLCSTNode>?} - A list of NLCST nodes, if

*/
one = function (node, index, parent, file, parser) {
function one(node, index, parent, file, location, parser) {
var type = node.type;
var pos = node.position;
var start = pos.start;
var end = pos.end;
var start = location.toOffset(pos.start);
var end = location.toOffset(pos.end);
var replacement;

@@ -206,3 +163,3 @@

if (node.children) {
replacement = all(node, file, parser);
replacement = all(node, file, location, parser);
} else if (

@@ -212,5 +169,5 @@ type === 'image' ||

) {
replacement = patch(parser.tokenize(
node.alt
), file, start.offset + 2);
replacement = patch(
parser.tokenize(node.alt), location, start + 2
);
} else if (

@@ -220,15 +177,17 @@ type === 'text' ||

) {
replacement = patch(parser.tokenize(node.value), file, start.offset);
replacement = patch(
parser.tokenize(node.value), location, start
);
} else if (node.type === 'break') {
replacement = patch([
parser.tokenizeWhiteSpace('\n')
], file, start.offset);
], location, start);
} else if (node.type === 'inlineCode') {
replacement = patch([parser.tokenizeSource(
file.toString().slice(start.offset, end.offset)
)], file, start.offset);
file.toString().slice(start, end)
)], location, start);
}
return replacement || null;
};
}

@@ -238,2 +197,3 @@ /**

*
* @param {Node} tree - MDAST node.
* @param {File} file - Virtual file.

@@ -244,10 +204,10 @@ * @param {Parser|Function} Parser - (Instance of) NLCST

*/
function toNLCST(file, Parser) {
var ast;
var space;
function toNLCST(tree, file, Parser) {
var parser;
var location;
/*
* Warn for invalid parameters.
*/
/* Warn for invalid parameters. */
if (!tree || !tree.type) {
throw new Error('mdast-util-to-nlcst expected node');
}

@@ -258,14 +218,14 @@ if (!file || !file.messages) {

space = file.namespace('mdast');
ast = space.tree || space.ast;
if (!ast || !ast.type) {
throw new Error('mdast-util-to-nlcst expected node');
/* Construct parser. */
if (!Parser) {
throw new Error('mdast-util-to-nlcst expected parser');
}
location = vfileLocation(file);
if (
!ast.position ||
!ast.position.start ||
!ast.position.start.column ||
!ast.position.start.line
!tree.position ||
!tree.position.start ||
!tree.position.start.column ||
!tree.position.start.line
) {

@@ -275,31 +235,11 @@ throw new Error('mdast-util-to-nlcst expected position on nodes');

/*
* Construct parser.
*/
if (!Parser) {
throw new Error('mdast-util-to-nlcst expected parser');
}
parser = 'parse' in Parser ? Parser : new Parser();
/*
* Patch ranges.
*/
range()(ast, file);
/*
* Transform mdast into NLCST tokens, and pass these
/* Transform mdast into NLCST tokens, and pass these
* into `parser.parse` to insert sentences, paragraphs
* where needed.
*/
return parser.parse(one(ast, null, null, file, parser));
* where needed. */
return parser.parse(one(tree, null, null, file, location, parser));
}
/*
* Expose.
*/
/* Expose. */
module.exports = toNLCST;

68

package.json
{
"name": "mdast-util-to-nlcst",
"version": "2.0.1",
"description": "Markdown to Natural Language",
"version": "3.0.0",
"description": "Transform MDAST to NLCST",
"license": "MIT",

@@ -10,2 +10,3 @@ "keywords": [

"nlcst",
"retext",
"markdown",

@@ -15,7 +16,5 @@ "natural",

],
"dependencies": {
"nlcst-to-string": "^1.0.0",
"remark-range": "^2.0.0",
"repeat-string": "^1.5.2"
},
"files": [
"index.js"
],
"repository": {

@@ -25,34 +24,37 @@ "type": "git",

},
"author": {
"name": "Titus Wormer",
"email": "tituswormer@gmail.com"
"bugs": "https://github.com/wooorm/mdast-util-to-nlcst/issues",
"author": "Titus Wormer <tituswormer@gmail.com> (http://wooorm.com)",
"contributors": [
"Titus Wormer <tituswormer@gmail.com> (http://wooorm.com)"
],
"dependencies": {
"nlcst-to-string": "^2.0.0",
"repeat-string": "^1.5.2",
"vfile-location": "^2.0.0"
},
"files": [
"index.js",
"LICENSE"
],
"devDependencies": {
"browserify": "^13.0.0",
"eslint": "^1.1.0",
"esmangle": "^1.0.0",
"browserify": "^13.0.1",
"eslint": "^2.0.0",
"esmangle": "^1.0.1",
"istanbul": "^0.4.0",
"jscs": "^2.0.0",
"jscs-jsdoc": "^1.0.0",
"parse-dutch": "^2.0.0",
"parse-english": "^2.0.0",
"parse-latin": "^2.0.0",
"remark": "^3.0.0",
"remark-comment-config": "^2.0.0",
"remark-github": "^2.0.0",
"remark-lint": "^2.0.0",
"remark-slug": "^3.0.0",
"remark-validate-links": "^2.0.0",
"retext": "^1.0.0",
"tape": "^4.4.0",
"vfile": "^1.0.0"
"jscs": "^3.0.0",
"jscs-jsdoc": "^2.0.0",
"parse-dutch": "^3.0.0",
"parse-english": "^3.0.0",
"parse-latin": "^3.1.0",
"remark": "^5.0.0",
"remark-cli": "^1.0.0",
"remark-comment-config": "^4.0.0",
"remark-github": "^5.0.0",
"remark-lint": "^4.0.0",
"remark-usage": "^4.0.0",
"remark-validate-links": "^4.0.0",
"tape": "^4.0.0",
"unist-util-inspect": "^4.0.0",
"vfile": "^1.4.0"
},
"scripts": {
"build-bundle": "browserify index.js --no-builtins -s mdastUtilToNLCST > mdast-util-to-nlcst.js",
"build-md": "remark . --quiet --frail",
"build-bundle": "browserify index.js --bare -s mdastUtilToNLCST > mdast-util-to-nlcst.js",
"build-mangle": "esmangle mdast-util-to-nlcst.js > mdast-util-to-nlcst.min.js",
"build-md": "remark . --quiet --frail",
"build": "npm run build-md && npm run build-bundle && npm run build-mangle",

@@ -59,0 +61,0 @@ "lint-api": "eslint .",

132

readme.md

@@ -1,12 +0,8 @@

# mdast-util-to-nlcst [![Build Status][travis-badge]][travis] [![Coverage Status][coverage-badge]][coverage]
# mdast-util-to-nlcst [![Build Status][travis-badge]][travis] [![Coverage Status][codecov-badge]][codecov]
[**mdast**][mdast] utility to transform markdown
into [**nlcst**][nlcst], while keeping location
information intact.
<!--lint disable heading-increment list-item-spacing-->
In plain English: this enables natural-language tooling to read markdown as
input.
Transform [MDAST][] to [NLCST][].
> **Note** You probably want to use
> [remark-retext][].
> **Note** You probably want to use [remark-retext][].

@@ -21,68 +17,66 @@ ## Installation

**mdast-util-to-nlcst** is also available for [duo][],
and as an AMD, CommonJS, and globals module,
[uncompressed and compressed][releases].
## Usage
```js
Dependencies:
```javascript
var toNLCST = require('mdast-util-to-nlcst');
var inspect = require('unist-util-inspect');
var English = require('parse-english');
var remark = require('remark');
var retext = require('retext');
var vfile = require('vfile');
```
/*
* Process.
*/
Process:
remark().process('Some *foo*s-_ball_.', function (err, file) {
var tree = toNLCST(file, retext().Parser);
```javascript
var file = vfile('Some *foo*sball.');
var tree = remark().parse(file);
```
console.log(inspect(tree));
/*
* Yields:
*
* RootNode[1] (1:1-1:20, 0-19)
* └─ ParagraphNode[1] (1:1-1:20, 0-19)
* └─ SentenceNode[4] (1:1-1:20, 0-19)
* ├─ WordNode[1] (1:1-1:5, 0-4)
* │ └─ TextNode: "Some" (1:1-1:5, 0-4)
* ├─ WhiteSpaceNode: " " (1:5-1:6, 4-5)
* ├─ WordNode[4] (1:7-1:18, 6-17)
* │ ├─ TextNode: "foo" (1:7-1:10, 6-9)
* │ ├─ TextNode: "s" (1:11-1:12, 10-11)
* │ ├─ PunctuationNode: "-" (1:12-1:13, 11-12)
* │ └─ TextNode: "ball" (1:14-1:18, 13-17)
* └─ PunctuationNode: "." (1:19-1:20, 18-19)
*/
});
Stringify:
```javascript
var nlcst = toNLCST(tree, file, English);
```
Which, when inspecting, yields:
```txt
RootNode[1] (1:1-1:17, 0-16)
└─ ParagraphNode[1] (1:1-1:17, 0-16)
└─ SentenceNode[4] (1:1-1:17, 0-16)
├─ WordNode[1] (1:1-1:5, 0-4)
│ └─ TextNode: "Some" (1:1-1:5, 0-4)
├─ WhiteSpaceNode: " " (1:5-1:6, 4-5)
├─ WordNode[2] (1:7-1:16, 6-15)
│ ├─ TextNode: "foo" (1:7-1:10, 6-9)
│ └─ TextNode: "sball" (1:11-1:16, 10-15)
└─ PunctuationNode: "." (1:16-1:17, 15-16)
```
## API
### `toNLCST(file, Parser | parser)`
### `toNLCST(node, file, Parser)`
Transform a by [**remark**][remark] processed
[**virtual file**][vfile] into an
[NLCST][nlcst] tree for
[**retext**][retext].
Transform an [MDAST][] syntax tree and corresponding [virtual file][vfile]
into an [NLCST][nlcst] tree.
Parameters:
###### Parameters
* `file` (`File`)
— [Virtual file][vfile], must be passed through
[`parse()`][remark-parse].
* `node` ([`MDASTNode`][mdast]) — Syntax tree (with positional
information);
* `file` ([`VFile`][vfile]);
* `parser` (`Function`)
— Constructor of an NLCST parser, such as
[**parse-english**][english], [**parse-dutch**][dutch],
or [**parse-latin**][latin].
* `parser` (`Function` or `Parser`, optional)
— You can pass the (constructor of) an NLCST parser, such as
[**parse-english**][parse-english], [**parse-dutch**][parse-dutch],
or [**parse-latin**][parse-latin].
###### Returns
Returns:
[`NLCSTNode`][nlcst].
[`NLCSTNode`][nlcst-node].
## License
[MIT][license] © [Titus Wormer][home]
[MIT][license] © [Titus Wormer][author]

@@ -95,11 +89,11 @@ <!-- Definitions -->

[coverage-badge]: https://img.shields.io/codecov/c/github/wooorm/mdast-util-to-nlcst.svg
[codecov-badge]: https://img.shields.io/codecov/c/github/wooorm/mdast-util-to-nlcst.svg
[coverage]: https://codecov.io/github/wooorm/mdast-util-to-nlcst
[codecov]: https://codecov.io/github/wooorm/mdast-util-to-nlcst
[vfile]: https://github.com/wooorm/vfile
[npm-install]: https://docs.npmjs.com/cli/install
[remark]: https://github.com/wooorm/remark
[license]: LICENSE
[retext]: https://github.com/wooorm/retext
[author]: http://wooorm.com

@@ -112,20 +106,8 @@ [mdast]: https://github.com/wooorm/mdast

[npm-install]: https://docs.npmjs.com/cli/install
[vfile]: https://github.com/wooorm/vfile
[duo]: http://duojs.org/#getting-started
[english]: https://github.com/wooorm/parse-english
[releases]: https://github.com/wooorm/mdast-util-to-nlcst/releases
[latin]: https://github.com/wooorm/parse-latin
[remark-parse]: https://github.com/wooorm/remark/blob/master/doc/remark.3.md#remarkparsefile-options
[nlcst-node]: https://github.com/wooorm/nlcst
[parse-english]: https://github.com/wooorm/parse-english
[parse-dutch]: https://github.com/wooorm/parse-dutch
[parse-latin]: https://github.com/wooorm/parse-latin
[license]: LICENSE
[home]: http://wooorm.com
[dutch]: https://github.com/wooorm/parse-dutch
LICENSE

Sorry, the diff of this file is not supported yet

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap
  • Changelog

About

Packages

npm

Go

Maven

PyPI

Rubygems

Stay in touch

Get open source security insights delivered straight into your inbox.

  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc