Huge News!Announcing our $40M Series B led by Abstract Ventures.Learn More
Socket
Sign inDemoInstall
Socket

tokenizer

Package Overview
Dependencies
Maintainers
1
Versions
11
Alerts
File Explorer

Advanced tools

Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

tokenizer - npm Package Compare versions

Comparing version 0.1.1 to 0.1.2

examples/underrun.js

7

lib/Tokenizer.js

@@ -65,3 +65,6 @@ var EventEmitter = require('events').EventEmitter;

if(nobuffer) {
this._gotToken(data.substr(last_tokenized), matching[0]);
// when no actual data was tokenized, matching is undefined
if(matching) {
this._gotToken(data.substr(last_tokenized), matching[0]);
}
}

@@ -75,3 +78,3 @@ // buffer data for the next write

Tokenizer.prototype.end = function end(data) {
this.write(data || '', true);
if(data) this.write(data, true);
this.write = noop;

@@ -78,0 +81,0 @@ this.emit('end');

{
"name": "tokenizer",
"description": "A wide purpose tokenizer for node.js which looks like a stream",
"version": "0.1.1",
"version": "0.1.2",
"homepage": "http://github.com/floby/node-tokenizer",

@@ -6,0 +6,0 @@ "repository": {

Sorry, the diff of this file is not supported yet

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap
  • Changelog

Packages

npm

Stay in touch

Get open source security insights delivered straight into your inbox.


  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc