transformers
Advanced tools
String/Data transformations for use in templating libraries, static site generators and web frameworks
The 'transformers' npm package provides a suite of tools for working with transformer models, particularly those developed by Hugging Face. It allows users to easily integrate state-of-the-art natural language processing (NLP) models into their applications for tasks such as text generation, translation, summarization, and more.
Text Generation
This feature allows you to generate text based on a given prompt using a pre-trained model like GPT-2.
const { pipeline } = require('transformers');
const generator = pipeline('text-generation', 'gpt2');
generator('Once upon a time,');Text Classification
This feature enables sentiment analysis, allowing you to classify the sentiment of a given text.
const { pipeline } = require('transformers');
const classifier = pipeline('sentiment-analysis');
classifier('I love using transformers!');Translation
This feature allows you to translate text from one language to another using pre-trained translation models.
const { pipeline } = require('transformers');
const translator = pipeline('translation_en_to_fr');
translator('Hello, how are you?');Summarization
This feature enables text summarization, allowing you to generate a concise summary of a longer text.
const { pipeline } = require('transformers');
const summarizer = pipeline('summarization');
summarizer('The quick brown fox jumps over the lazy dog.');spaCy is an open-source software library for advanced NLP in Python. It is designed specifically for production use and provides a wide range of functionalities including tokenization, part-of-speech tagging, named entity recognition, and more. Compared to transformers, spaCy is more focused on traditional NLP tasks and less on transformer-based models.
The Natural Language Toolkit (NLTK) is a suite of libraries and programs for symbolic and statistical natural language processing for English written in the Python programming language. NLTK provides easy-to-use interfaces to over 50 corpora and lexical resources. While NLTK is comprehensive for traditional NLP tasks, it does not focus on transformer models like the transformers package.
String/Data transformations for use in templating libraries, static site generators and web frameworks. This gathers the most useful transformations you can apply to text or data into one library with a consistent API. Transformations can be pretty much anything but most are either compilers or templating engines.
To use each of these transforms you will also need to install the associated npm module for that transformer.
<![CDATA[${INPUT_STRING]]> with the standard escape for ]]> (]]]]><![CDATA[>).cdata, but with surrounding comments suitable for inclusion into a HTML/JavaScript <script> block: //<![CDATA[\n${INPUT_STRING\n//]]>.cdata, but with surrounding comments suitable for inclusion into a HTML/CSS <style> block: /*<![CDATA[*/\n${INPUT_STRING\n/*]]>*/.${INPUT_STRING}npm install coffee-scriptmarked, supermarked, markdown-js or markdownnpm install component-builder options: {development: false}npm install component-builder options: {development: false}npm install html2jade - Converts HTML back into jadePull requests to add more transforms will always be accepted providing they are open-source, come with unit tests, and don't cause any of the tests to fail.
The exported object transformers is a collection of named transformers. To access an individual transformer just do:
var transformer = require('transformers')['transformer-name']
The following options are given special meaning by transformers:
filename is set by transformers automatically if using the renderFile APIs. It is used if cache is enabled.cache if true, the template function will be cached where possible (templates are still updated if you provide new options, so this can be used in most live applications).sudoSync used internally to put some asyncronous transformers into "sudo syncronous" mode. Don't touch this.minify if set to true on a transformer that isn't a minifier, it will cause the output to be minified. e.g. coffeeScript.renderSync(str, {minify: true}) will result in minified JavaScript.Returns an array of engines that can be used to power this transformer. The first of these that's installed will be used for the transformation.
To enable a transformation just take [engine] = Transformer.engines[0] and then do npm install [engine]. If [engine] is . there is no need to install an engine from npm to use the transformer.
Tranform the string str using the Transformer with the provided options and call the callback cb(err, res).
If no cb is provided, this method returns a promises/A+ promise.
Synchronous version of Transformer.render
Reads the file at filename into str and sets options.filename = filename then calls Transform.render(str, options, cb).
If no cb is provided, this method returns a promises/A+ promise.
Synchronous version of Tranformer.renderFile
A string, one of:
'xml''css''js''json''text'Adding to this list will not result in a major version change, so you should handle unexpected types gracefully (I'd suggest default to assuming 'text').
true if the transformer can be used syncronously, false otherwise.
The following transformations will always throw an exception if you attempt to run them synchronously:
The following transformations sometimes throw an exception if run syncronously, typically they only throw an exception if you are doing something like including another file. If you are not doing the things that cause them to fail then they are consistently safe to use syncronously.
then-jade instead of jade)@import is used with a url instead of a filename)The following libraries look like they might sometimes throw exceptions when used syncronously (if you read the source) but they never actually do so:
FAQs
String/Data transformations for use in templating libraries, static site generators and web frameworks
We found that transformers demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
ENISA’s 2024 report highlights the EU’s top cybersecurity threats, including rising DDoS attacks, ransomware, supply chain vulnerabilities, and weaponized AI.
Security News
NIST's new password guidelines remove periodic changes and special character requirements, focusing on longer, more secure passwords for better authentication practices.
Security News
A record 2,709 developers participated in the 2024 Ruby on Rails Community Survey, revealing key tools, practices, and trends shaping the Rails ecosystem.