@poppinss/utils
A toolkit of utilities used across all the AdonisJS, Edge, and Japa packages
Why this package exists?
Many of my open source projects (including AdonisJS) use many single-purpose utility packages from npm. Over the years, I have faced the following challenges when using these packages.
- It takes a lot of time to find a perfect package for the use case. The package should be well maintained, have good test coverage, and not accumulate debt by supporting some old versions of Node.js.
- Some packages are great, but they end up pulling a lot of unnecessary dependencies like (requiring TypeScript as a prod dependency)
- Sometimes I end up using different packages for the same utility (because, I cannot remember what I used last time in that other package). So I want to spend time once choosing the one I need and then bundle it inside
@poppinss/utils
. - Some authors introduce breaking changes too often (not a criticism). Therefore, I prefer wrapping their packages with my external API only to absorb breaking changes in one place.
- Rest are some handwritten utilities to fit my needs
Note: If you are creating an AdonisJS package, I highly recommend using this package since it is already part of the user's project dependencies.
Warning: This package is not for general use (outside the AdonisJS ecosystem). I will not add new helpers or remove any to cater to a broader audience.
Other packages to use
A note to self and others to consider the following packages.
Package | Description |
---|
he | For escaping HTML entities and encoding unicode symbols. Has zero dependencies |
@sindresorhus/is | For advanced type checking. Has zero dependencies |
Package size
Even though I do not care much about the package size (most of work is consumed on server side), I am mindful around the utilities and ensure not end up using really big packages for smaller use-cases.
| |
---|
Source code size | 272K (approx) |
Dependencies size | 432K (approx) |
Installation
Install the package from the npm registry as follows:
npm i @poppinss/utils
yarn add @poppinss/utils
Exported modules
Following are the exported modules. Only the generic helpers are shipped from the main path. The rest of the helpers are grouped inside sub-modules.
import string from '@poppinss/utils/string'
import json from '@poppinss/utils/json'
import lodash from '@poppinss/utils/lodash'
import { base64, Exception, fsReadAll } from '@poppinss/utils'
String helpers
A collection of helpers to perform operations on/related to a string value.
import string from '@poppinss/utils/string'
excerpt
Generate an excerpt from a string value. If the input value contains HTML tags, we will remove them from the excerpt.
const html = `<p>AdonisJS is a Node.js framework, and hence it requires Node.js to be installed on your computer. To be precise, we need at least the latest release of <code>Node.js v14</code>.</p>`
console.log(string.excerpt(html, 70))
Argument | Type | Description |
---|
sentence | string | The value for which to generate excerpt |
charactersLimit | string | The number of characters to keep |
options.completeWords | boolean | When set to true , the truncation will happen only after complete words. This option might go over the defined characters limit |
options.suffix | string | The value to append after the truncated string. Defaults to three dots ... |
truncate
Truncate a string value to a certain length. The method is the same as the excerpt
method but does not remove any HTML tags. It is a great fit when you are truncating a non-HTML string.
const text = `AdonisJS is a Node.js framework, and hence it requires Node.js to be installed on your computer. To be precise, we need at least the latest release of Node.js 14.`
console.log(string.truncate(text, 70))
Argument | Type | Description |
---|
sentence | string | The value to truncate |
charactersLimit | string | The number of characters to keep |
options.completeWords | boolean | When set to true , the truncation will happen only after complete words. This option might go over the defined characters limit |
options.suffix | string | The value to append after the truncated string. Defaults to three dots ... |
slug
Generate slug for a string value. The method is exported directly from the slugify package.
Please check the package documentation for available options.
console.log(string.slug('hello ♥ world'))
You can add custom replacements for Unicode values as follows.
string.slug.extend({ '☢': 'radioactive' })
console.log(string.slug('unicode ♥ is ☢'))
interpolate
Interpolate variables inside a string. The variables must be inside double curly braces.
string.interpolate('hello {{ user.username }}', { user: { username: 'virk' } })
You can also replace array values by mentioning the array index.
string.interpolate('hello {{ users.0 }}', { users: ['virk'] })
You can escape the curly braces by prefixing them with \\
.
string.interpolate('hello \\{{ users.0 }}', {})
plural
Convert a word to its plural form. The method is exported directly from the pluralize package.
string.plural('test')
singular
Convert a word to its singular form. The method is exported directly from the pluralize package.
string.singular('tests')
pluralize
This method combines the singular
and plural
methods and uses one or the other based on the count. For example:
string.pluralize('box', 1)
string.pluralize('box', 2)
string.pluralize('box', 0)
string.pluralize('boxes', 1)
string.pluralize('boxes', 2)
string.pluralize('boxes', 0)
The addPluralRule
, addSingularRule
, addIrregularRule
, and addUncountableRule
methods exposed by the pluralize package can be called as follows.
string.pluralize.addUncountableRule('paper')
string.pluralize.addSingularRule(/singles$/i, 'singular')
isPlural
Find if a word is already in plural form. The method is exported directly from the pluralize package.
string.isPlural('tests')
isSingular
Find if a word is already in a singular form. The method is exported directly from the pluralize package.
string.isSingular('test')
camelCase
Convert a string value to camelcase.
string.camelCase('user_name')
Following are some of the conversion examples.
Input | Output |
---|
'test' | 'test' |
'test string' | 'testString' |
'Test String' | 'testString' |
'TestV2' | 'testV2' |
'foo_bar' | 'fooBar' |
'version 1.2.10' | 'version1210' |
'version 1.21.0' | 'version1210' |
capitalCase
Convert a string value to a capital case.
string.capitalCase('helloWorld')
Following are some of the conversion examples.
Input | Output |
---|
'test' | 'Test' |
'test string' | 'Test String' |
'Test String' | 'Test String' |
'TestV2' | 'Test V 2' |
'version 1.2.10' | 'Version 1.2.10' |
'version 1.21.0' | 'Version 1.21.0' |
dashCase
Convert a string value to a dash case.
string.dashCase('helloWorld')
Optionally, you can capitalize the first letter of each word.
string.dashCase('helloWorld', { capitalize: true })
Following are some of the conversion examples.
Input | Output |
---|
'test' | 'test' |
'test string' | 'test-string' |
'Test String' | 'test-string' |
'Test V2' | 'test-v2' |
'TestV2' | 'test-v-2' |
'version 1.2.10' | 'version-1210' |
'version 1.21.0' | 'version-1210' |
dotCase
Convert a string value to a dot case.
string.dotCase('helloWorld')
Optionally, you can also convert the first letter of all the words to lowercase.
string.dotCase('helloWorld', { lowerCase: true })
Following are some of the conversion examples.
Input | Output |
---|
'test' | 'test' |
'test string' | 'test.string' |
'Test String' | 'Test.String' |
'dot.case' | 'dot.case' |
'path/case' | 'path.case' |
'TestV2' | 'Test.V.2' |
'version 1.2.10' | 'version.1210' |
'version 1.21.0' | 'version.1210' |
noCase
Remove all sorts of casing from a string value.
string.noCase('helloWorld')
Following are some of the conversion examples.
Input | Output |
---|
'test' | 'test' |
'TEST' | 'test' |
'testString' | 'test string' |
'testString123' | 'test string123' |
'testString_1_2_3' | 'test string 1 2 3' |
'ID123String' | 'id123 string' |
'foo bar123' | 'foo bar123' |
'a1bStar' | 'a1b star' |
'CONSTANT_CASE ' | 'constant case' |
'CONST123_FOO' | 'const123 foo' |
'FOO_bar' | 'foo bar' |
'XMLHttpRequest' | 'xml http request' |
'IQueryAArgs' | 'i query a args' |
'dot.case' | 'dot case' |
'path/case' | 'path case' |
'snake_case' | 'snake case' |
'snake_case123' | 'snake case123' |
'snake_case_123' | 'snake case 123' |
'"quotes"' | 'quotes' |
'version 0.45.0' | 'version 0 45 0' |
'version 0..78..9' | 'version 0 78 9' |
'version 4_99/4' | 'version 4 99 4' |
' test ' | 'test' |
'something_2014_other' | 'something 2014 other' |
'amazon s3 data' | 'amazon s3 data' |
'foo_13_bar' | 'foo 13 bar' |
pascalCase
Convert a string value to pascal case. Great for generating JavaScript class names.
string.pascalCase('user team')
Following are some of the conversion examples.
Input | Output |
---|
'test' | 'Test' |
'test string' | 'TestString' |
'Test String' | 'TestString' |
'TestV2' | 'TestV2' |
'version 1.2.10' | 'Version1210' |
'version 1.21.0' | 'Version1210' |
sentenceCase
Convert a value to a sentence.
string.sentenceCase('getting-started-with-adonisjs')
Following are some of the conversion examples.
Input | Output |
---|
'test' | 'Test' |
'test string' | 'Test string' |
'Test String' | 'Test string' |
'TestV2' | 'Test v2' |
'version 1.2.10' | 'Version 1 2 10' |
'version 1.21.0' | 'Version 1 21 0' |
snakeCase
Convert value to snake case.
string.snakeCase('user team')
Following are some of the conversion examples.
Input | Output |
---|
'_id' | 'id' |
'test' | 'test' |
'test string' | 'test_string' |
'Test String' | 'test_string' |
'Test V2' | 'test_v2' |
'TestV2' | 'test_v_2' |
'version 1.2.10' | 'version_1210' |
'version 1.21.0' | 'version_1210' |
titleCase
Convert a string value to title case.
string.titleCase('small word ends on')
Following are some of the conversion examples.
Input | Output |
---|
'one. two.' | 'One. Two.' |
'a small word starts' | 'A Small Word Starts' |
'small word ends on' | 'Small Word Ends On' |
'we keep NASA capitalized' | 'We Keep NASA Capitalized' |
'pass camelCase through' | 'Pass camelCase Through' |
'follow step-by-step instructions' | 'Follow Step-by-Step Instructions' |
'this vs. that' | 'This vs. That' |
'this vs that' | 'This vs That' |
'newcastle upon tyne' | 'Newcastle upon Tyne' |
'newcastle *upon* tyne' | 'Newcastle *upon* Tyne' |
random
Generate a cryptographically secure random string of a given length. The output value is URL safe base64 encoded string.
string.random(32)
toSentence
Convert an array of words to a comma-separated sentence.
string.toSentence(['routes', 'controllers', 'middleware'])
You can replace the and
with an or
by specifying the options.lastSeparator
property.
string.toSentence(['routes', 'controllers', 'middleware'], {
lastSeparator: ', or ',
})
In the following example, the two words are combined using the and
separator, not the comma (usually advocated in English). However, you can use a custom separator for a pair of words.
string.toSentence(['routes', 'controllers'])
string.toSentence(['routes', 'controllers'], {
pairSeparator: ', and ',
})
condenseWhitespace
Remove multiple whitespaces from a string to a single whitespace.
string.condenseWhitespace('hello world')
string.condenseWhitespace(' hello world ')
ordinal
Get the ordinal letter for a given number.
string.ordinal(1)
string.ordinal(2)
string.ordinal(3)
string.ordinal(4)
string.ordinal(23)
string.ordinal(24)
seconds.(parse/format)
Parse a string-based time expression to seconds.
string.seconds.parse('10h')
string.seconds.parse('1 day')
Passing a numeric value to the parse
method is returned as it is, assuming the value is already in seconds.
string.seconds.parse(180)
You can format seconds to a pretty string using the format
method.
string.seconds.format(36000)
string.seconds.format(36000, true)
milliseconds.(parse/format)
Parse a string-based time expression to milliseconds.
string.milliseconds.parse('1 h')
string.milliseconds.parse('1 day')
Passing a numeric value to the parse
method is returned as it is, assuming the value is already in milliseconds.
string.milliseconds.parse(180)
Using the format
method, you can format milliseconds to a pretty string.
string.seconds.format(3.6e6)
string.seconds.format(3.6e6, true)
bytes.(parse/format)
Parse a string-based unit expression to bytes.
string.bytes.parse('1KB')
string.bytes.parse('1MB')
Passing a numeric value to the parse
method is returned as it is, assuming the value is already in bytes.
string.bytes.parse(1024)
Using the format
method, you can format bytes to a pretty string. The method is exported directly from the bytes package. Please reference the package README for available options.
string.bytes.format(1048576)
string.bytes.format(1024 * 1024 * 1000)
string.bytes.format(1024 * 1024 * 1000, { thousandsSeparator: ',' })
JSON helpers
Following are the helpers we use to stringify
and parse
JSON.
safeParse
The native implementation of JSON.parse
opens up the possibility for prototype poisoning. The safeParse
method removes the __proto__
and the constructor.prototype
properties from the JSON string at the time of parsing it.
The method is a wrapper over secure-json-parse package.
safeStringify
The native implementation of JSON.stringify
cannot handle circular references or language-specific data types like BigInt
.
Therefore, we use the safe-stable-stringify package under the hood to overcome the limitations of native implementation.
import { safeStringify } from '@poppinss/utils/json'
const value = {
b: 2,
c: BigInt(10),
}
value.a = value
safeStringify(value)
- The circular references are removed from the final JSON string.
- The BigInt values are converted to a string.
The safeStringify
API is the same as the JSON.stringify
method.
- You can pass a replacer function as the second parameter.
- And number of spaces as the third parameter.
Lodash helpers
Lodash is quite a big library, and we do not use all its helper methods. Therefore we create a custom build using the lodash CLI and bundle only the once we need.
Why not use something else: All other helpers I have used are not as accurate or well implemented as lodash.
- pick
- omit
- has
- get
- set
- unset
- mergeWith
- merge
- size
- clone
- cloneDeep
- toPath
You can use the methods as follows.
import lodash from '@poppinss/utils/lodash'
lodash.pick(collection, keys)
All other helpers
The following helpers are exported from the package main module.
import { base64, compose } from '@poppinss/utils'
base64
Utility methods to base64 encode and decode values.
import { base64 } from '@poppinss/utils'
base64.encode('hello world')
Similar to the encode
method, you can use the urlEncode
to generate a base64 string safe to pass in a URL.
The urlEncode
method performs the following replacements.
- Replace
+
with -
. - Replace
/
with _
. - And remove the
=
sign from the end of the string.
base64.urlEncode('hello world')
You can use the decode
and the urlDecode
methods to decode a previously encoded base64 string.
base64.decode(base64.encode('hello world'))
base64.urlDecode(base64.urlEncode('hello world'))
The decode
and the urlDecode
methods return null
when the input value is an invalid base64 string. You can turn on the strict
mode to raise an exception instead.
base64.decode('hello world')
base64.decode('hello world', 'utf-8', true)
compose
The compose
helper allows you to use TypeScript class mixins with a cleaner API. Following is an example of mixins usage without the compose helper.
class User extends UserWithAttributes(UserWithAge(UserWithPassword(UserWithEmail(BaseModel)))) {}
Following is an example with the compose
helper.
- There is no nesting.
- The order of mixins is from left to right. Whereas earlier, it was inside out.
import { compose } from '@poppinss/utils'
class User extends compose(
BaseModel,
UserWithEmail,
UserWithPassword,
UserWithAge,
UserWithAttributes
) {}
defineStaticProperty
The defineStaticProperty
method allows you to define static properties on a class with different reference strategies.
If you use class inheritance alongside static properties, then either, you will share properties by reference, or you will define them directly on the parent class.
In the following example, we are not inherting columns
from the AppModel
. Instead, we define a new set of columns on the UserModel
.
class AppModel {
static columns = ['id']
}
class UserModel extends AppModel {
static columns = ['username']
}
In the following example, we are inherting columns
from the AppModel
. However, the mutations (array.push) from the UserModel
will reflect on the AppModel
as well.
class AppModel {
static columns = ['id']
}
class UserModel extends AppModel {}
UserModel.columns.push('username')
The ideal behavior is to deep clone the columns
array and then push new values to it.
import lodash from '@poppinss/utils/lodash'
class AppModel {
static columns = ['id']
}
const inheritedColumns = lodash.cloneDeep(AppModel.columns)
class UserModel extends AppModel {
static columns = inheritedColumns.push('username')
}
The defineStaticProperty
method abstracts the logic to clone and also performs some interal checks to see if the value is already defined as an ownProperty
or not.
class UserModel extends AppModel {}
defineStaticProperty(UserModel, 'columns', {
strategy: 'inherit',
initialValue: [],
})
- The
inherit
strategy clones the value from the parent class. - The
define
strategy always re-defines the property, discarding any values on the parent class. - The
strategy
value can be function to perform a custom clone operations.
Exception
A custom exception class with support for defining the error status, error code, and help description. This class aims to standardize exceptions within your projects.
import { Exception } from '@poppinss/utils'
class ResourceNotFound extends Exception {
static code = 'E_RESOURCE_NOT_FOUND'
static status = 404
static message = 'Unable to find resource'
}
throw new ResourceNotFound()
Anonymous error classes
You can also create an anonymous exception class using the createError
method. The return value is a class
constructor that accepts an array of values to use for interpolation.
The interpolation of error message is performed using the util.format
message.
import { createError } from '@poppinss/utils'
const E_RESOURCE_NOT_FOUND = createError('Unable to find resource with id %d', 'E_RESOURCE_NOT_FOUND')
const id = 1
throw new E_RESOURCE_NOT_FOUND([id])
flatten
Create a flat object from a nested object/array. The nested keys are combined with a dot-notation (.
). The method is exported from the flattie package.
import { flatten } from '@poppinss/utils'
flatten({
a: 'hi',
b: {
a: null,
b: ['foo', '', null, 'bar'],
d: 'hello',
e: {
a: 'yo',
b: undefined,
c: 'sup',
d: 0,
f: [
{ foo: 123, bar: 123 },
{ foo: 465, bar: 456 },
],
},
},
c: 'world',
})
fsReadAll
Get a list of all the files from a directory. The method recursively fetches files from the main and the sub-folders. The dotfiles are ignored implicitly.
import { fsReadAll } from '@poppinss/utils'
const files = await fsReadAll(new URL('./config', import.meta.url), { pathType: 'url' })
await Promise.all(files.map((file) => import(file)))
You can also pass the options along with the directory path as the second argument.
type Options = {
ignoreMissingRoot?: boolean
filter?: (filePath: string, index: number) => boolean
sort?: (current: string, next: string) => number
pathType?: 'relative' | 'unixRelative' | 'absolute' | 'unixAbsolute' | 'url'
}
const options: Partial<Options> = {}
await fsReadAll(location, options)
Argument | Type | Description |
---|
ignoreMissingRoot | boolean | By default, an exception is raised when the root directory is missing. Setting ignoreMissingRoot to true will not result in an error and an empty array is returned back. |
filter | method | Define a filter to ignore certain paths. The method is called on the final list of files. |
sort | method | Define a custom method to sort file paths. By default, the files are sorted using natural sort. |
pathType | enum | Define how to return the collected paths. By default, OS-specific relative paths are returned. If you want to import the collected files, you must set the pathType = 'url' |
fsImportAll
The fsImportAll
method imports all the files recursively from a given directory and set the exported value from each module on an object.
import { fsImportAll } from '@poppinss/utils'
const collection = await fsImportAll(new URL('./config', import.meta.url))
console.log(collection)
- Collection is an object with a tree of key-value pair.
- The key is the nested object created from the file path.
- Value is the exported values from the module. If a module exports both the
default
and named
values, then only the default values are used.
The second param is the options to customize the import behavior.
Argument | Type | Description |
---|
ignoreMissingRoot | boolean | By default, an exception is raised when the root directory is missing. Setting ignoreMissingRoot to true will not result in an error and an empty object is returned back. |
filter | method | Define a filter to ignore certain paths. By default only files ending with .js , .ts , .json , .cjs , and .mjs are imported. |
sort | method | Define a custom method to sort file paths. By default, the files are sorted using natural sort. |
transformKeys | method | Define a callback method to transform the keys for the final object. The method receives an array of nested keys and must return an array back. |
isScriptFile
A filter to know if the file path ends with .js
, .json
, .cjs
, .mjs
or .ts
. In the case of .ts
files, the .d.ts
returns false.
import { isScriptFile } from '@poppinss/utils'
isScriptFile('foo.js')
isScriptFile('foo/bar.cjs')
isScriptFile('foo/bar.mjs')
isScriptFile('foo.json')
isScriptFile('foo/bar.ts')
isScriptFile('foo/bar.d.ts')
The goal of this method is to use it as a filter with the fsReadAll
method.
import { fsReadAll, isScriptFile } from '@poppinss/utils'
const dir = new URL('./config', import.meta.url)
const options = { pathType: 'url', filter: isScriptFile }
const files = await fsReadAll(dir, options)
await Promise.all(
files.map((file) => {
if (file.endsWith('.json')) {
return import(file, { assert: { type: 'json' } })
}
return import(file)
})
)
naturalSort
A sorting function to use natural sort for ordering an array.
import { naturalSort } from '@poppinss/utils'
const values = ['1_foo_bar', '12_foo_bar'].sort()
const values = ['1_foo_bar', '12_foo_bar'].sort(naturalSort)
safeEqual
Check if two buffer or string values are the same. This method does not leak any timing information and prevents timing attack.
Under the hood, this method uses Node.js crypto.timeSafeEqual method, with support for comparing string values. (crypto.timeSafeEqual does not support string comparison)
import { safeEqual } from '@poppinss/utils'
const trustedValue = 'hello world'
const userInput = 'hello'
if (safeEqual(trustedValue, userInput)) {
} else {
}
slash
Convert OS-specific file paths to Unix file paths. The method is exported directly from the slash package.
import { slash } from '@poppinss/utils'
slash('foo\\bar')
MessageBuilder
Message builder is a convenience layer to stringify JavaScript values with an expiry date and a purpose. For example:
import { MessageBuilder } from '@poppinss/utils'
const builder = new MessageBuilder()
const encoded = builder.build(
{
token: string.random(32),
},
'1 hour',
'email_verification'
)
Once you have the JSON string with the expiry and the purpose, you can encrypt it (to prevent tampering) and share it with the client.
During the email verification, you can decrypt the key and then ask the MessageBuilder
to verify the payload.
const decoded = builder.verify(value, 'email_verification')
if (!decoded) {
return 'Invalid token'
}
console.log(decoded.token)
Now let's imagine someone presents the same token to reset their account password. In the following example, the validation will fail since the purpose of the original token is not the same as the purpose set during the verify
method call.
const decoded = builder.verify(value, 'reset_password')
ObjectBuilder
The ObjectBuilder
is a convenience class to create an object with dynamic properties. Consider the following example, where we wrap our code inside conditionals before adding the property b
to the startingObject
.
const startingObject = {
a: 1
...(b ? { b } : {})
}
if (b) {
startingObject.b = b
}
Instead of writing conditionals, you can consider using the Object builder fluent API.
const builder = new ObjectBuilder({ a: 1 })
const plainObject = builder
.add('b', b)
.toObject()
By default, only the undefined
values are ignored. However, you can also ignore null
values.
const ignoreNullValues = true
const builder = new ObjectBuilder({ a: 1 }, ignoreNullValues)
Following are the available methods on the ObjectBuilder
class.
builder.remove(key)
builder.has(key)
builder.get(key)
builder.add(key)
builder.toObject()
dirname/filename
ES modules does not have magic variables __filename
and __dirname
. You can use these helpers to get the current directory and filenames as follows.
import { getDirname, getFilename } from '@poppinss/utils'
const dirname = getDirname(import.meta.url)
const filename = getFilename(import.meta.url)