The smallest CommonMark compliant markdown parser with positional info and
concrete tokens.
Intro
micromark is a long awaited markdown parser.
It uses a state machine to parse the entirety of markdown into tokens.
It’s the smallest 100% CommonMark compliant markdown parser in JavaScript.
It’ll replace the internals of remark-parse
, the most
popular markdown parser (remarkjs/remark#536
).
Its interface is optimized to compile to HTML, but its parts can be used
to generate syntax trees (mdast-util-from-markdown
) or
compile to other output formats too.
It’s in open beta: up next are integration in remark, CMSM, and CSTs.
Contents
Install
npm:
npm install micromark
Use
Typical use (buffering):
var micromark = require('micromark')
console.log(micromark('## Hello, *world*!'))
Yields:
<h2>Hello, <em>world</em>!</h2>
Extensions (in this case micromark-extension-gfm
):
var micromark = require('micromark')
var gfmSyntax = require('micromark-extension-gfm')
var gfmHtml = require('micromark-extension-gfm/html')
var doc = '* [x] contact@example.com ~~strikethrough~~'
var result = micromark(doc, {
extensions: [gfmSyntax()],
htmlExtensions: [gfmHtml]
})
console.log(result)
Yields:
<ul>
<li><input checked="" disabled="" type="checkbox"> <a href="mailto:contact@example.com">contact@example.com</a> <del>strikethrough</del></li>
</ul>
Streaming interface:
var fs = require('fs')
var micromarkStream = require('micromark/stream')
fs.createReadStream('example.md')
.on('error', handleError)
.pipe(micromarkStream())
.pipe(process.stdout)
function handleError(err) {
throw err
}
API
Note that there are also as of yet undocumented APIs to use the individual parts
of micromark and to make extensions.
micromark(doc[, encoding][, options])
Compile markdown to HTML.
Parameters
doc
Markdown to parse (string
or Buffer
)
encoding
Character encoding to understand doc
as when it’s a
Buffer
(string
, default: 'utf8'
).
options.defaultLineEnding
Value to use for line endings not in doc
(string
, default: first line
ending or '\n'
).
Generally, micromark copies line endings ('\r'
, '\n'
, '\r\n'
) in the
markdown document over to the compiled HTML.
In some cases, such as > a
, CommonMark requires that extra line endings are
added: <blockquote>\n<p>a</p>\n</blockquote>
.
options.allowDangerousHtml
Whether to allow embedded HTML (boolean
, default: false
).
options.allowDangerousProtocol
Whether to allow potentially dangerous protocols in links and images (boolean
,
default: false
).
URLs relative to the current protocol are always allowed (such as, image.jpg
).
For links, the allowed protocols are http
, https
, irc
, ircs
, mailto
,
and xmpp
.
For images, the allowed protocols are http
and https
.
options.extensions
Array of syntax extensions (Array.<SyntaxExtension>
,
default: []
).
options.htmlExtensions
Array of HTML extensions (Array.<HtmlExtension>
, default:
[]
).
Returns
string
— Compiled HTML.
micromarkStream(options?)
Streaming interface of micromark.
Compiles markdown to HTML.
options
are the same as the buffering API above.
Available at require('micromark/stream')
.
Note that some of the work to parse markdown can be done streaming, but in the
end it requires buffering.
micromark does not handle errors for you, so you must handle errors on whatever
streams you pipe into it.
As markdown does not know errors, micromark
itself does not emit errors.
Extensions
There are two types of extensions for micromark:
SyntaxExtension
and HtmlExtension
.
They can be passed in extensions
or
htmlExtensions
, respectively.
SyntaxExtension
A syntax extension is an object whose fields are the names of tokenizers:
content
(a block of, well, content: definitions and paragraphs), document
(containers such as block quotes and lists), flow
(block constructs such as
ATX and setext headings, HTML, indented and fenced code, thematic breaks),
string
(things that work in a few places such as destinations, fenced code
info, etc: character escapes and -references), or text
(rich inline text:
autolinks, character escapes and -references, code, hard breaks, HTML, images,
links, emphasis, strong).
The values at such objects are character codes, mapping to constructs.
The built in constructs are an extension.
See it and the existing extensions for inspiration.
HtmlExtension
An HTML extension is an object whose fields are either enter
or exit
(reflecting whether a token is entered or exited).
The values at such objects are names of tokens mapping to handlers.
See the existing extensions for inspiration.
List of extensions
Syntax tree
A higher level project, mdast-util-from-markdown
, can give
you an AST.
var fromMarkdown = require('mdast-util-from-markdown')
var result = fromMarkdown('## Hello, *world*!')
console.log(result.children[0])
Yields:
{
type: 'heading',
depth: 2,
children: [
{type: 'text', value: 'Hello, ', position: [Object]},
{type: 'emphasis', children: [Array], position: [Object]},
{type: 'text', value: '!', position: [Object]}
],
position: {
start: {line: 1, column: 1, offset: 0},
end: {line: 1, column: 19, offset: 18}
}
}
Another level up is remark, which provides a nice interface and
hundreds of plugins.
CommonMark
The first definition of “Markdown” gave several examples of how it worked,
showing input Markdown and output HTML, and came with a reference implementation
(known as Markdown.pl
).
When new implementations followed, they mostly followed the first definition,
but deviated from the first implementation, and added extensions, thus making
the format a family of formats.
Some years later, an attempt was made to standardize the differences between
implementations, by specifying how several edge cases should be handled, through
more input and output examples.
This attempt is known as CommonMark, and many implementations
now work towards some degree of CommonMark compliancy.
Still, CommonMark describes what the output in HTML should be given some
input, which leaves many edge cases up for debate, and does not answer what
should happen for other output formats.
micromark passes all tests from CommonMark and has many more tests to match the
CommonMark reference parsers.
Finally, it comes with CMSM, which describes how to parse markup, instead
of documenting input and output examples.
Test
micromark is tested with the ~650 CommonMark tests and more than 1000 extra
tests confirmed with other markdown parsers.
These tests reach all branches in the code, thus this project has 100% coverage.
Finally, we use fuzz testing to ensure micromark is stable, reliable, and
secure.
Size & debug
micromark is really small.
A ton of time went into making sure it minifies well, by the way code is written
but also through custom build scripts to pre-evaluate certain expressions.
Furthermore, care went into making it compress well with GZip and Brotli.
Normally, you’ll use the pre-evaluated version of micromark, which is published
in the dist/
folder and has entries in the root.
While developing or debugging, you can switch to use the source, which is
published in the lib/
folder, and comes instrumented with assertions and debug
messages.
To see debug messages, run your script with a DEBUG
env variable, such as with
DEBUG="micromark" node script.js
.
Comparison
There are many other markdown parsers out there, and maybe they’re better suited
to your use case!
Here is a short comparison of a couple of ’em in JavaScript.
Note that this list is made by the folks who make micromark
and remark
, so
there is some bias.
micromark
micromark is the lowest you can go: it gives tremendous power, such as access to
all tokens with positional info, at the cost of being hard to get into.
It’s super small, pretty fast, and has 100% CommonMark compliance.
It has syntax extensions, such as supporting 100% GFM compliance (with
micromark-extension-gfm
), but they’re rather complex to write.
It’s the newest parser on the block.
If you’re looking for fine grained control, use micromark.
remark is the most popular markdown parser.
It’s built on top of micromark
and boasts syntax trees.
For an analogy, it’s like if Babel, ESLint, and more, were one project.
It supports the syntax extensions that micromark has (so it’s 100% CM compliant
and can be 100% GFM compliant), but most of the work is done in plugins that
transform or inspect the tree.
Transforming the tree is relatively easy: it’s a JSON object that can be
manipulated directly.
remark is stable, widely used, and extremely powerful for handling complex data.
If you’re looking to inspect or transform lots of content, use remark.
marked
marked is the oldest markdown parser on the block.
It’s been around for ages, is battle tested, small, popular, and has a bunch of
extensions, but doesn’t match CommonMark or GFM, and is unsafe by default.
If you have markdown you trust and want to turn it into HTML without a fuss, use
marked.
markdown-it
markdown-it is a good, stable, and essentially CommonMark compliant markdown
parser, with (optional) support for some GFM features as well.
It’s used a lot as a direct dependency in packages, but is rather big.
It shines at syntax extensions, where you want to support not just markdown, but
your (company’s) version of markdown.
If you’re in Node and have CommonMark-compliant (or funky) markdown and want to
turn it into HTML, use markdown-it.
Others
There are lots of other markdown parsers!
Some say they’re small, or fast, or that they’re CommonMark compliant — but
that’s not always true.
This list is not supposed to be exhaustive.
This list of markdown parsers is a snapshot in time of why (not) to use
(alternatives to) micromark
: they’re all good choices, depending on what your
goals are.
Version
The open beta of micromark starts at version 2.0.0
(there was a different
package published on npm as micromark
before).
micromark will adhere to semver at 3.0.0
.
Use tilde ranges for now: "micromark": "~2.9.2"
.
Security
It’s safe to compile markdown to HTML if it does not include embedded HTML nor
uses dangerous protocols in links (such as javascript:
or data:
).
micromark is safe by default if embedded HTML or dangerous protocols are used
too, as it encodes or drops them.
Turning on the allowDangerousHtml
or allowDangerousProtocol
options for
user-provided markdown opens you up to cross-site scripting (XSS)
attacks.
For more information on markdown sanitation, see
improper-markup-sanitization.md
by @chalker.
See security.md
in micromark/.github
for how to
submit a security report.
Contribute
See contributing.md
in micromark/.github
for ways
to get started.
See support.md
for ways to get help.
This project has a code of conduct.
By interacting with this repository, organisation, or community you agree to
abide by its terms.
Support this effort and give back by sponsoring on OpenCollective!
License
MIT © Titus Wormer