inote-util
A collection of utility functions and classes for Node.js.
Contents
- Features - method-by-method description of the utilities.
- Installing - how to install from npm or source.
- Licensing -
inote-util
is made available under an MIT License. - How to Contribute - tips on reporting issues, creating pull-requests for or just generally hacking
inote-util
.
Using
note-util
exports a number of objects, each containing several utility functions.
Hence, require(‘note-util’)
will return a map with several sub-objects (ArrayUtil
, AsyncUtil
, etc.)
There are two common idioms for handling this.
Idiom 1: Keep a handle to this “parent” object:
var IU = require(“note-util”);
IU.LogUtil.tlog(“Here is a random string:”, IU.RandomUtil.random_alphanumeric());
Idiom 2: Keep a handle to the individual utilities:
var LogUtil = require(“note-util”).LogUtil;
var RandomUtil = require(“note-util”).RandomUtil;
LogUtil.tlog(“Here is a random string:”, RandomUtil.random_alphanumeric());
Personally, the author prefers the second approach, but the difference is almost entirely cosmetic. In particular, note that in Node.js require
-ing the same module multiple times DOES NOT reload or reparse the files in the module. The file is only load once.
The following section lists methods under the associated object. E.g., under NetUtil
you’ll find get_unused_port()
. This means that the get_unused_port
method is exported within the top-level object named “NetUtil” and could be accessed via:
var gup = require(“note-util”).NetUtil.get_unused_port;
Features
Feature Index
ArrayUtil |
AsyncUtil |
ColorUtil |
Config |
DateUtil |
FileUtil |
FileUtil MIME |
IOUtil |
L10nUtil |
LogUtil |
NetUtil |
NumberUtil |
ObjectUtil |
RandomUtil |
SimpleCache |
Stopwatch |
StringUtil |
Util |
WebUtil |
WorkQueue |
ZipUtil |
Errors
ArrayUtil
- lpad(value,width,pad) - adds
pad
elements to the beginning of value
until value
is width
elements long. (Also accepts strings, see StringUtil.lpad
, which is identical.) - rpad(value,width,pad) - adds
pad
elements to the end of value
until value
is width
elements long. (Also accepts strings, see StringUtil.rpad
, which is identical.) - smart_join(array,delim,last_delim) - identical to
Array.join
, except the specified last_delim
is used between the last two elements (if any). E.g., smart_join(["Tom","Dick","Harry"],", "," and ")
yields Tom, Dick and Harry
. - trim_trailing_null(array) - returns a copy of
array
with trailing null
elements removed - right_shift_args(...) - returns an array the same length as the given arguments, but any trailing
null
values are converted to leading null
values. (Most useful in the CoffeeScript idiom [a,b,c,d] = right_shift_args(a,b,c,d)
.) - paginate_list(list[,offset=0[,limit=20]]) - returns the specified section of the given array.
- subset_of(a,b) / ArrayUtil.is_subset_of(a,b) - returns
true
if every element of array a is also an element of b. - strict_subset_of(a,b) / ArrayUtil.is_strict_subset_of(a,b) - returns
true
if every element of array a appears exacty the same number of times in array b as it does in array a. (E.g., ['a','a','b']
is subset of but not a strict subset of ['a','b','c']
, according to this definition). - sets_are_equal(a,b) - compares arrays as if they were sets.
- arrays_are_equal(a,b) -
true
if and only if array a and array b contain the exact same elements in the exact same order. DEPRECATED: please use ObjectUtil.deep_equal
- uniquify(array[,key]) - returns a clone of
array
with duplicate values removed. When the array contains objects (maps) and a key
is provided, two elements will be considered duplicates if they have the same value for the attribute key
.
Back to Index
AsyncUtil
- wait_until(predicate[,interval],callback) / wait_for(predicate[,interval],callback) - polls
predicate()
every interval
milliseconds until it returns true
(or throws an exception). Callback signature is (err,complete)
where complete
is true
if and only if predicate()
returned true
. interval
defaults to 100
. - wait(delay,callback) / set_timeout(delay,callback) / setTimeout(delay,callback) - just like
setTimeout(callback,delay)
but with a more CoffeeScript-friendly parameter order. - cancel_wait(id) / clear_timeout(id) / clearTimeout(id) - alias for
window.clearTimeout(id)
. - interval(delay,callback) / set_interval(delay,callback) / setInterval(delay,callback) - just like
setInterval(callback,delay)
but with a more CoffeeScript-friendly parameter order. - cancel_interval(id) / cancelInterval(id) / clear_interval(id) / clearlInterval(id) - alias for
window.clearInterval(id)
. - for_async(initialize,condition,action,increment[,options],whendone) - executes an asynchronous
for
loop. Accepts 5 function-valued parameters:
initialize
- an initialization function (no arguments passed, no return value is expected);condition
- a predicate that indicates whether we should continue looping (no arguments passed, a boolean value is expected to be returned);action
- the action to take (a single callback function is passed and should be invoked at the end of the action, no return value is expected);increment
- called at the end of every action
, prior to condition
(no arguments passed, no return value is expected);options
- optional options map
delay
- time added before each successive invocation, in millisecondstimeout
- max time to wait for an action to complete, in millisecondscatch_exceptions
- boolean
whendone
- called at the end of the loop (when condition
returns false
), (no arguments passed, no return value is expected).
- for_each_async(list,action[,options],whendone) - executes an asynchronous
forEach
loop. Accepts 3 (or 4) parameters:
list
- the array to iterate over;action
- a function with the signature (value,index,list,next)
indicating the action to take for each element (must call next
for processing to continue);options
- optional options map
delay
- time added before each successive invocation, in millisecondstimeout
- max time to wait for an action to complete, in millisecondscatch_exceptions
- boolean
whendone
- called at the end of the loop.
- fork(methods,args_for_methods[,options],callback) - runs the given array of methods "simultaneously" (asynchronously), invoking
callback
when they are all complete:
options
- optional options map
delay
- time added before each successive invocation, in millisecondstimeout
- max time to wait for an action to complete, in millisecondscatch_exceptions
- boolean
- throttled_fork(max_parallel,methods,args_for_methods[,options],callback) - just like
fork
, but never running more than max_parallel
functions at the same time. - fork_for_each_async(list,action[,options],whendone) - like
for_each_async
but running action
in parallel for each element of the list
. The whendone
callback is provided with a list of "responses", in the same order as the original list. - throttled_fork_for_each_async(max_parallel,list,action[,options],whendone) - like
fork_for_each_async
but running at most max_parallel
methods at any one time. - invoke_with_timeout(method,args[,options],callback)
- maybe_invoke_with_timeout(method,args[,options],callback)
- procedure() - generates a
Sequencer
object, as described below
The Sequencer
The methods Util.procedure()
, AsyncUtil.procedure()
, and new Sequencer()
will create a new "sequencer" object that provides a simple way to "flatten" long chains of nested asynchronous methods.
For example, rather than writing:
method_one( function(e,a,b) {
method_two(a, b, function() {
method_three( function(e,c,d) {
and_so_on();
});
});
});
We can flatten the calls out like this:
var procedure = AsyncUtil.procedure();
procedure.first( method_one );
procedure.then( method_two );
procedure.then( method_three );
procedure.then( and_so_on );
procedure.run();
Each call to then
appends the given callback method to the chain.
Each callback method is passed a next_step
function that must be called to trigger the next step in the processing.
Note that any arguments passed to next_step()
will be passed to the next method in the sequence (with the next
function appended). For example, given a method in the sequence such as:
procedure.next(function(next_step) {
next_step(1,"A",[]);
});
The following method in the procedure will be invoked with the following signature:
the_next_method_in_the_sequence(1,"A",[],next_step);
Hence the typical use of the class looks something like this:
var s = new Sequencer()
s.first( function(done) {
done();
});
s.next( function(done) {
done();
});
s.next( function(done) {
done();
});
s.run();
When run
is invoked, each asynchronous method is executed in sequence.
The first
method is optional (you can just use next
instead), but when invoked first
will remove any methods previously added to the chain.
You last
methods is an optional short-hand for adding one final method to the chain and then running it. E.g., the last two lines of our example:
procedure.then( and_so_on )
procedure.run()
Could be re-written:
procedure.last( and_so_on )
Note that the sequence is not cleared when run
is invoked, so one may invoke run
more than once to execute the sequence again.
Back to Index
ColorUtil
- hex_to_rgb_triplet(hex) - converts a hex-based
#rrggbb
string to decimal [r,g,b]
values. - hex_to_rgb_string(hex) - converts a hex-based
#rrggbb
string to a string of the form rgb(r,g,b)
. - rgb_string_to_triplet(rgb) - converts a string of the form
rgb(r,g,b)
to decimal [r,g,b]
values. - rgb_triplet_to_string(r,g,b) - convert an array or sequence of r, g, b values to a string of the form
rgb(r,g,b)
.
Back to Index
Config
A thin wrapper around nconf
providing a consistent way to load configuration data from files or the environment.
Note that config now allows //
and /* */
style comments in JSON files.
Example of use
var config = require('inote-util').config.init();
var prop = config.get('my:property');
Config
will load the configuration from several sources.
In order of precedence:
-
"Override" values passed to the init
function.
-
Command line parameters (--name value
).
-
Environment variables. Note that __
can be used in place of :
as a delimiter.
-
A JSON-format configuration file (from a location deterimined by
NODE_ENV
, config_dir
or config_file
).
-
A "default" JSON-format configuration file at ${config_dir}/config.json
.
-
"Default" values passed to the init
function.
To discover a configuration file (as used in step 3 above), Config
will:
-
Look for NODE_ENV
, config_dir
or config_file
environment variables.
-
If config_dir
is set, the value will be used as the "parent" directory
of the configuration files. (If config_dir
is not set, it defaults to
the directory config
within the working directory from which the current
Node.js process was launched.)
-
If NODE_ENV
is set and a file named ${NODE_ENV}.json
exists within
the config_dir
, that file will be used.
-
If config_file
is set, that file will be used.
Back to Index
DateUtil
- start_time - timestamp at which
inote-util
was loaded (hence approximately the time the application was started in most circumstances). - duration(end_time,begin_time) - returns an object that breaks-down the time between
begin_time
and end_time
in several ways (as described below). When missing, end_time
defaults to Date.now()
and begin_time
defaults to start_time
. - iso_8601_regexp() - returns a regular expression that can be used to validate an ISO 8601 formatted date.
Here is an example of the object returned by the DateUtil.duration
, with brief annotations.
{
"begin":1462806430444,
"end":1462851730757,
"delta":45300313,
"in_millis":{
"millis":313,
"seconds":313,
"minutes":2100313,
"hours":45300313,
"days":45300313,
"weeks":45300313,
"years":45300313
},
"raw":{
"millis":313,
"seconds":0.313,
"minutes":35.00521666666667,
"hours":12.583420277777778,
"days":0.5243091782407407,
"weeks":0.07490131117724867,
"years":0.00020520907171848953
},
"whole":{
"millis":313,
"seconds":0,
"minutes":35,
"hours":12,
"days":0,
"weeks":0,
"years":0
},
"array":{
"full":{
"values":[0,0,0,12,35,0,313],
"short":["0y","0w","0d","12h","35m","0s","313m"],
"long":["0 years","0 weeks","0 days","12 hours","35 minutes","0 seconds","313 milliseconds"],
"no_millis":{
"values":[0,0,0,12,35,0],
"short":["0y","0w","0d","12h","35m","0s"],
"long":["0 years","0 weeks","0 days","12 hours","35 minutes","0 seconds"]
}
},
"brief":{
"values":[12,35,0,313],
"short":["12h","35m","0s","313m"],
"long":["12 hours","35 minutes","0 seconds","313 millis"],
"no_millis":{
"values":[12,35,0],
"short":["12h","35m","0s"],
"long":["12 hours","35 minutes","0 seconds"]
}
},
"min":{
"units":["hour","minute","millisecond"],
"short":["12h","35m","313m"],
"long":["12 hours","35 minutes","313 milliseconds"],
"no_millis":{
"units":["hour","minute"],
"short":["12h","35m"],
"long":["12 hours","35 minutes"]
}
}
},
"string":{
"full":{
"micro":"0y0w0d12h35m0s313m",
"short":"0y 0w 0d 12h 35m 0s 313m",
"long":"0 years 0 weeks 0 days 12 hours 35 minutes 0 seconds 313 milliseconds",
"verbose":"0 years, 0 weeks, 0 days, 12 hours, 35 minutes, 0 seconds and 313 milliseconds",
"no_millis":{
"micro":"0y0w0d12h35m0s",
"short":"0y 0w 0d 12h 35m 0s",
"long":"0 years 0 weeks 0 days 12 hours 35 minutes 0 seconds",
"verbose":"0 years, 0 weeks, 0 days, 12 hours, 35 minutes and 0 seconds"
}
},
"brief":{
"micro":"12h35m0s313m",
"short":"12h 35m 0s 313m",
"long":"12 hours 35 minutes 0 seconds 313 millis",
"verbose":"12 hours, 35 minutes, 0 seconds and 313 millis",
"no_millis":{
"micro":"12h35m0s",
"short":"12h 35m 0s",
"long":"12 hours 35 minutes 0 seconds",
"verbose":"12 hours, 35 minutes and 0 seconds"
}
},
"min":{
"micro":"12h35m313m",
"short":"12h 35m 313m",
"long":"12 hours 35 minutes 313 milliseconds",
"verbose":"12 hours, 35 minutes and 313 milliseconds",
"no_millis":{
"micro":"12h35m",
"short":"12h 35m",
"long":"12 hours 35 minutes",
"verbose":"12 hours, 35 minutes and 48 seconds"
}
}
}
}
Back to Index
FileUtil
- file_age(file,callback) / file_age_sync(file) - obtain the age of a file (time since last modfied) in milliseconds
- file_mtime(file,callback) / file_mtime_sync(file)- obtain the Unix epoch timestamp at which a file was last modified
- ls(dir[,options],callback) - list the files in a directory; options:
recurse
- when true
, perform the operation recursivelypattern
- when a non-null
RegExp, only list files matching the specified patterntypes
- an array or string containing file
or directory
- is_dir(filename,callback) - test if the specified filename is a directory
- is_file(filename,callback) - test if the specified filename is a plain file (not a directory).
- sanitize_filename(filename) - removes invalid characters from and truncates extremely long filenames; only operates on the last segement of the given path. That is, if
filename
is /foo/bar/xyz
, only the xyz
part of the string will be modified. - uniquify_filename(dir,basename[,ext=''[,minpadwidth=3[,maxpadwidth=5]]) - attempts to generate a unique filename in
dir
based on basename
. - mkdir(dir) -
mkdir -p dir
- touch(file) -
touch file
- rm(files...) - remove one or more files, ignoring errors. (Returns
true
if any errors are encountered, false
otherwise). - rmdir(dirs...) - recursively remove one or more directories or files, ignoring errors. (Returns
true
if any errors are encountered, false
otherwise). - read_stdin_sync([end_byte="\x04"[,buffer_size=512]]) - synchronously read all of stdin (up to
end_byte
), returning the resulting buffer - load_json_file(file[, options], callback) (also just
load_json
)- asynchronously read and parse a JSON file. When options.ignore_errors
is true, calls-back with null, null
rather than err, null
. When options.allow_comments
is true
(the default) JS-style comments are allowed. When options.strip_comments
is true
(the default) comments do NOT appear in the returned JSON object. - load_json_file_sync(file[,options]) - synchronously read and parse a JSON file. When
options.ignore_errors
is true, returns null
rather than throwing an exception when the file is not found or does not contain value JSON. When options.allow_comments
is true
(the default) JS-style comments are allowed. When options.strip_comments
is true
(the default) comments do NOT appear in the returned JSON object. - load_json_stdin_sync([end_byte="\x04"[,buffer_size=512[,options]]]) - synchronously read and parse JSON object from stdin. When
options.ignore_errors
is true, returns null
rather than throwing an exception. When options.allow_comments
is true
(the default) JS-style comments are allowed. When options.strip_comments
is true
(the default) comments do NOT appear in the returned JSON object. - copy_file(src,dest,callback) - copy a file from
src
to dest
; works across filesystems. - move_file(src,dest,callback) - move (rename) a file from
src
to dest
; works across filesystems.
Back to Index
FileUtil - MIME and File-Extension related
- get_ext(fname) / get_extension(fname) - equivalent to
path.extname
, save that fname
can be the extension itself. (E.g., both path.extname('.foo')
and path.extname('foo')
return ''
while FileUtil.get_ext('.foo')
and FileUtil.get_ext('foo')
return 'foo'
). Specifically, returns the part of fname
following the final dot (.
), unless fname
contains no dot, in which case the entire fname
value is returned. - strip_ext(fname) / strip_extension(fname) - returns a version of
fname
with the file extension removed. - replace_ext(fname,ext) / replace_extension(fname,ext) - returns a version of
fname
with the file extension changed to ext
. - get_mime_for_ext(ext) / get_mime_for_extension(ext) - returns the "standard" MIME type based on the extension found in
ext
. (ext
may be a full filename or just the extension). - get_ext_for_mime(mime) / get_extension_for_mime(mime) - returns the "standard" file extension for the given MIME type.
- get_mime_via_magic(file,callback) / get_mime_type_via_magic(file,callback) / get_file_mime_type_via_magic(file,callback) - determines MIME type of
file
by magic number (ignoring the file extension if any). - get_mime(file,callback) / get_mime_type(file,callback) / get_file_mime_type(file,callback) - determines MIME type of
file
by magic number or extension. Given a choice, the specific type is selected (e.g., application/json
vs. text/plain
). The magic-number-based MIME type is used if the two MIME types seem equally specific. - file_is_mime(file,pattern,callback) - calls-back with
null, true
when the MIME type of file
matches pattern.
- file_is_pdf(file,callback) - calls-back with
null, true
when the MIME type of file
is application/pdf
. - get_mime_to_ext_map() / get_mime_to_extension_map() - returns the MIME-type to "standard" file-extension mapping used for the other methods.
- get_ext_to_mime_map() / get_extension_to_mime_map() - returns the file-extension to "standard" MIME-type mapping used for the other methods.
- set_mime_to_ext_map(map) / set_mime_to_extension_map(map) - sets the MIME-type to "standard" file-extension mapping used for the other methods. Extensions should NOT contain a leading dot (
.
). Set to null
to restore the default mapping. - set_ext_to_mime_map(map) / set_extension_to_mime_map(map) - sets the file-extension to "standard" MIME-type mapping used for the other methods. Extensions should NOT contain a leading dot (
.
). Set to null
to restore the default mapping. - add_to_mime_to_ext_map(map | mime, ext) / add_to_mime_to_ext_map(map | mime, ext) - adds a collection or a single instance of a mime-to-file-extension mapping to the currently active set.
- add_to_ext_to_mime_map(map | ext, mime) / add_to_extension_to_mime_map(map | ext, mime) - adds a collection or a single instance of a file-extension-to-mime-type mapping to the currently active set.
Note that it is not necessarily the case that get_ext_for_mime(get_mime_for_ext( EXT )) == EXT
and vice-versa. See the data
directory for the default mappings.
Back to Index
IOUtil
- pipe_to_file(readable_stream,dest,options,callback) - write a stream to a file. (
callback(err,file)
). - pipe_to_buffer(readable_stream,callback) - write a stream to a buffer. (
callback(err,buffer)
). - download_to_file(url,dest,options,callback) - write the contents of a URL to a file. (
callback(err,file)
). - download_to_buffer(url,callback) - write the contents of a URL to a buffer. (
callback(err,buffer)
). - download_to_data_uri(url,callback) - convert the contents of a URL to a data-uri. (
callback(err,uri)
).
Back to Index
L10nUtil
- identify_locales(req) - identify a list of
[ lang, REGION ]
pairs based on the given request object. - expand_locales(locales) - given a list of
[ lang, REGION ]
pairs, adds [ lang, null ]
to this list if otherwise missing. - identify_and_expand_locales(req) - `expand_locales(identify_locales(req))``
- match_locale(accepted, available, default_value) - given a list of accepted
[ lang, REGION ]
pairs and a map of lang-region: non-null
, returns an acceptable key (or the default_value
) when no matching locale is found. - load_l10n_files(dir, [options], callback) - loads a set of localization files (JSON + comments) from the specified directory, returning a map of
'name-without-extension': loaded-content
. When present options
are options passed to FileUtil.ls
, notably recurse
and pattern
(defaulting to false
and /^.+.json$/
, respectively). Note that a specific lang-REGION
's attributes will inherit default values from those in lang
(when set), such that if en.json
has the key foo
but en-US.json
does not, the returned en-us
object will contain the key foo
. Callback signature is (err, map_of_localiation_data)
- localize(localization_data, key, [args...]) - returns the localized value for the
sprintf
template localization_data[key]
, passing in the optional args
arguments when provided. Returns null
if no matching template is found.
- note that a localization entry may be a map from which one specific template will be selected according to the value of the first argument (see unit tests for details).
- make_localizer(localization_data) - returns the function of the form
fn(key, [args...])
which is equivalent to localize(localization_data, key, [args...])
.
LogUtil
- Some notes on configuration:
require('inote-util').LogUtil
yields a "singleton" object with a default configuration.require('inote-util').LogUtil.init
is a function that invokes the LogUtil
constructorrequire('inote-util').LogUtil.LogUtil
and require('inote-util').LogUtil.constructor
provide direct access to the LogUtil
constructor.- Both LogUtil.init and the LogUtil.LogUtil constructor accept a single "configuration" object, a map with the following (optional) attributes:
debug
- when true
LogUtil.[t[p]]debug(...)
while write to LogUtil.[t[p]]log(...)
; otherwise LogUtil.debug
produces no output.prefix
- when a non-null
string is provided it will be prefixed to every log line (after the timestamp and pid identifiers, where applicable).logger
- an object that defines the various underlying logging methods; by default this is console
. The logger
map can be used to override the default behavior by providing any of the following methods:
log(...)
info(...)
warn(...)
error(...)
- log(...), info(...), warn(...), err(...), error(...) - writes to stdout/stderr or to the equivalent method in the
logger
object provided to LogUtil.init
- tlog(...), tinfo(...), twarn(...), terr(...), terror(...) - prepends a timestamp to each log message (E.g.,
[2015-05-17T22:03:58.569Z] Hello World!
) - tplog(...), tpinfo(...), tpwarn(...), tperr(...), tperror(...) - prepends a timestamp and process ID to each log message (E.g.,
[2015-05-17T22:03:58.569Z] [p:123] Hello World!
) - debug(...), tdebug(...), tpdebug(...) - like
log
/tlog
,tplog
but only writes to the log if the debug
attribute passed to the constructor is true
.
Back to Index
NetUtil
- is_port_in_use(port,callback) - attempt to determine whether or not a port is currently in use
- get_unused_port(callback) - attempt to obtain an unused port
- normalize_url
Back to Index
NumberUtil
- round_decimal(value[,digits=0]) - round a number to the specified number of digits to the right of the decimal point.
- is_int(val) - returns
true
if and only if val
is a simple integer (matching /^-?((0)|([1-9][0-9]*))$/
). - to_int(val) - returns
parseInt(val)
when val
is a simple integer (matching is_int
), null
otherwise. (Compare with the default behavior of parseInt
, which returns 17
for parseInt("17.93 meters")
). - is_float(val) - returns
true
if and only if val
is a simple decimal value (matching /^-?((((0)|([1-9][0-9]*))(\.[0-9]+)?)|(\.[0-9]+))$/
). - to_float(val) - returns
parseFloat(val)
when val
is a simple decimal value (matching is_float
), null
otherwise.
Back to Index
ObjectUtil
- get_json_path(json,path...) - fetches that attribute stored at the specified path
- get_funky_json_path(json,path...) - fetches that attribute stored at the specified path, handlig the
@name
and name.$
attributes that some XML-to-JSON parsers produce. - deep_equal(a,b) - performs a deep comparison of the two specified objects; handles arrays, maps, strings, booleans, numbers,
null
and undefined
, results are undefined for other object types; considers null
, undefined
and missing elements to be equal. - is_true_object(a) -
true
iff a
is a non-null
, non-array object for which typeof a == 'object'
. - diff_json(a,b) - recursively compares elements of
a
and b
, returning a map describing where the objects differ; handles arrays, maps, strings, booleans, numbers, null
and undefined
, results are undefined for other object types; considers null
, undefined
and missing elements to be equal. Some examples:
{foo:{bar:1}}
, {foo:{bar:1}}
yields undefined
(no difference)3
, 3}
yields undefined
(no difference){foo:{bar:1}}
, {foo:{bar:2}}
yields {foo:{bar:'c'}}
(value of foo.bar
has c
hanged.{foo:{bar:null}}
, {foo:{bar:2}}
yields {foo:{bar:'a'}}
(value of foo.bar
was a
dded.{foo:{bar:1}}
, {foo:{x:true}}
yields {foo:{bar:'d',x:'a'}}
(value of foo.bar
was d
eleted and value of foo.x
was a
dded).
- remove_null(map,recurse=false) - generates a (shallow) clone of the map or array, with
null
entries removed; when recurse
is true
the action is repeated for each map or array-value element or attribute - remove_falsey(map,recurse=false) - generates a (shallow) clone of the map, with "falsey" entries removed (see
falsey_string
); when recurse
is true
the action is repeated for each map or array-value element or attribute - merge(maps...) - given two or more maps
a
and b
, creates new new map containing the union of elements from each. If a
and b
share a key, the value in b
will overwrite the value in a
. - deep_merge(maps...) - just like
merge
except that sub-maps are recursively merged (e.g., merging {a:{a1:1}}
and {a:{a2:2}}
yields {a:{a1:1,a2:2}}
rather than {a:{a2:2}}
). - shallow_clone(obj) - create a shallow clone of the given map or array.
- deep_clone(obj) - recursively copy the elements
obj
into a new map (or array) - object_array_to_map(array,key_field[,options={})]) - Given a list of objects, creates a map from
elt[key_field]
to elt
for each elt
in the array.
options.transform
- an optional function used to transform the value of elt[key_field]
before using it as the map key.options.duplicates
- a string indicating how to handle duplicate keys:
"overwrite"
- replace the old value with the new value (the default)"stack"
- create an array containing both values (in sequence)"merge"
- merge the objects using Util.merge(old,new)
"skip"
- keep the old value and ignore the new one
Back to Index
RandomUtil
- random_bytes([count=32[,encoding='hex']]) - returns
count
random bytes in the specified encoding
. - seed_rng(seed) - returns a new
random()
function with the specified seed
value. - set_rn([rng = Math.random]) - sets the
random()
function used by the RandomUtil
methods. - random_hex([count=32[,rng]]) - returns
count
random hexadecimal digits ([a-f0-9]
) (using the given random number generator if provided). - random_alphanumeric([count=32[,rng]]) - returns
count
random digits from the set [a-z0-9]
(using the given random number generator if provided). - random_alpha([count=32[,rng]]) - returns
count
random digits from the set [a-z]
(using the given random number generator if provided). - random_numeric([count=32[,rng]]) - returns
count
random digits from the set [0-9]
(using the given random number generator if provided). - random_Alphanumeric([count=32[,rng]]) - returns
count
random digits from the set [0-9a-zA-Z]
(using the given random number generator if provided). - random_Alpha([count=32[,rng]]) - returns
count
random digits from the set [a-zA-Z]
(using the given random number generator if provided). - random_ALPHA([count=32[,rng]]) - returns
count
random digits from the set [A-Z]
(using the given random number generator if provided). - random_element(collection[,rng]) - returns a random element from an array, or
[key,value]
pair given a map (using the given random number generator if provided). - random_value([min],[max],[rng]) - returns a random float between
min
and max
, inclusive; min
defaults to 0
, max
defaults to 1
- randomly_assign(id[,categories]) - assign the given (non-null) identifier to a randomly selected category such that (A) fgor a given collection of identifiers results will be randomly distributed among the categories and (B) for a given identifier the result will be the same every time the method is called. The
categories
parameter is optional: when missing, identifiers will be randomly assigned to either true
or false
; when an value between 0 and 1, identifiers will be assigned to either true
or false
, with true
appearing 100*value
% of the time; when an array, identifiers will be assigned to an element of the array with equal probability - shuffle(list) - performs an in-place shuffle of the given list
Back to Index
SimpleCache
- new SimpleCache([{default_ttl:, default_value:, purge_interval:}])
- cache.put(key, value, [{ttl:}])
- cache.get(key, [{ignore_ttl:, default:}]) / cache.set(key, [{ignore_ttl:, default:}])
- cache.get_put(key, value, [{ttl:, ignore_ttl:, default:}]) / cache.get_set(key, value, [{ttl:, ignore_ttl:, default:}])
- cache.clear(key)
- cache.get_clear(key, [{ignore_ttl:, default:}])
- cache.clear_all()
- cache.clear_matching(/regexp/)
- cache.keys([{ignore_ttl:}])
- cache.values([{ignore_ttl:}])
- cache.pairs([{ignore_ttl:}])
Back to Index
Stopwatch
A simple utility that can be used to track and report the time it takes to do some thing in your JavaScript code.
Basic Use
var SW = require('inote-util').Stopwatch;
var timer = SW.start();
timer.stop();
console.log("Start Time: ",timer.start_time);
console.log("Finish Time: ",timer.finish_time);
console.log("Elapsed Time:",timer.elapsed_time);
Wrapped (Synchronous)
timer = SW.time( some_method );
console.log("some_method took",timer.elapsed_time,"millis to complete.");
"Cookies"
The start
and time
methods accept an optional map of attributes that will be bundled with the returned timer. For example:
timer = SW.start({label:"foo"});
timer.stop();
console.log(timer.label," Start Time: ",timer.start_time);
console.log(timer.label,"Finish Time: ",timer.finish_time);
console.log(timer.label,"Elapsed Time:",timer.elapsed_time);
Back to Index
StringUtil
- trim(str) - equivalent to
String.trim()
save that str
can be null
. - is_blank(str) -
true
whenever str
is empty, composed entirely of whitespace, null
or undefined
. - isnt_blank(str) - opposite of
is_blank()
- blank_to_null(str) - given a "blank" string, returns
null
. Given an object (map), removes any top-level "blank" attributes. - truncate(str,width[,marker='…']) - a minimally "smart" truncation that attempts to truncate a string at a word boundaries. The specified
marker
will be added if and only if the string was actually truncated. - escape_for_json(str) - escapes a (possibly
null
) string for use as literal characters in a JSON string. - escape_for_regexp(str) - escapes a (possibly
null
) string for use as literal characters in a regular expression. - escape_for_bash(param,esc_specials=false) - escapes a (possibly
null
) string for use as command-line parameter. When esc_specials
is false
(the default) parameters such as <
, >
, &&
, etc. are not escaped. - sanitize_for_html
- json_or_null
- StringUtil.sanitize_for_sql_like
- truthy_string(str) -
true
if the given string is t
, true
, y
, yes
, on
, 1
, etc. - falsey_string(str) -
true
if the given string is f
, false
, no
, off
, 0
, etc. - lpad(value,width,pad) - adds
pad
characters to the beginning of value
until value
is width
characters long. (Also accepts arrays, see ArrayUtil.lpad
, which is an identical method.) - rpad(value,width,pad) - adds
pad
characters to the end of value
until value
is width
characters long. (Also accepts arrays, see ArrayUtil.rpad
, which is an identical method.) - shuffle(list) - performs an in-place shuffle of the given list
Back to Index
Util
- get_funky_json(json,keys...) -t he call
get_funky_json(json, "foo", "bar")
will return Xyzzy
for { foo: { bar: "Xyzzy" } }
and { foo: { bar: {$:"Xyzzy" } } }
and { foo: { @bar: "Xyzzy" } }
, etc. This is used to mask the differences between certain XML-to-JSON translators. This method is also aliased as gfj. - version_satisfies([version_string,]range_string) - returns
true
if the given version_string
satisfies the semver criteria specified in range_string
. When version_string
is omitted, the Node version (the value of process.version
) is used. E.g., if(!Util.version_satisifes(">=0.12.3")) { console.error("Expected version 0.12.3 or later."); }
. - slow_equals(a,b) - constant-time comparison of two buffers for equality.
- compare(a,b) - a minimally-smart comparision function (allows
null
, uses localeCompare
when available, folds case so that both A
and a
appear before B
, etc.). - field_comparator(field[,use_locale_compare=false]) - returns a comparator (
function(a,b)
) that compares two maps on the field
attribute. - path_comparator(path[,use_locale_compare=false]) - like
field_comparator
, but path
may be an array of fields that will be interpreted as nested-attributes. (E.g., ["foo","bar"]
compares a.foo.bar
with b.foo.bar
.) - descending_comparator(comparator) / desc_comparator(comparator) - reverses the order of the given
comparator
. - composite_comparator(list) - generates a comparator that first compares elements by
list[0]
then (if equal) list[1]
and so on, until a non-equal comparison is found or we run out of comparators. - handle_error(err[,callback[,throw_when_no_callback=true]]) - if
err
is not null
, invokes callback(err)
or throw err
as appropriate. Returns true
if an error was encountered, false
otherwise. (function my_callback(err,other,stuff) { if(!handle_error(err,callback)) { /* keep going */ } }
) - uuid(val[,generate=false]) - normalize
val
to all a lower-case, no-dash version of a UUID. If generate
is true
, generate an new UUID when given a null val
, otherwise returns null
in that scenario. (DEPRECATED)
- As of v1.8.3, when called with no arguments
uuid()
generates a UUID value, while uuid(null)
retains the original behavior.
- normalize_uuid(val[,generate=false]) - non-deprecated replacement for
uuid(val,generate)
. - make_uuid([val]), make_uuid_v1([val]), make_uuid_v4([val]) - return a normalized version of the specified UUID, creating a new one if none is proivded.
- pad_uuid(val[,generate=false]) - normalize
val
to all a lower-case, with-dashes version of a UUID. If generate
is true
, generate an new UUID when given a null val
, otherwise returns null
in that scenario. - b64e(buf[,encoding='utf8']) / Base64.encode(buf[,encoding='utf8']) - Base64 encode the given buffer.
- b64d(buf[,encoding='utf8']) / Base64.decode(buf[,encoding='utf8']) - Base64 decode the given buffer.
Back to Index
WebUtil
- remote_ip(req,name,default_value) - attempts to discover the proper "client IP" for the given request using various approaches.
- param(req, name, default_value) - replaces the now deprecated
req.param(name,default_value)
found in Express.js. name
can also be an array of names to check. - map_to_qs(map) (also map_to_query_string(map)) - convert the given map to a string of the form
name1=value1&name2=value2&etc.
for which names and values are properly URL-encoded. When a key maps to an array the key is appended multiple times (key=value1&key=value2&etc.
). - append_qs(url,map) / append_qs(url,qs_fragment) / append_qs(url,name,value) (also append_query_string())
Back to Index
WorkQueue
WorkQueue
implements a basic priority queue for asynchronous tasks.
Users may add "tasks" to the work queue, which will be asynchronously executed according to priority order.
Each task consists of:
- a function
- optionally, an array of arguments to the function
- optionally, a priority for the task
- optionally, a callback function to invoke when the task is complete
The final parameter to the task-function will always be the WorkQueue's done
function. Users must call this function to indicate that the task is complete (or, optionally throw an exception).
Any arguments passed to the done
function will in turn be passed to the callback function registered with the task (if any).
Example
var WorkQueue = require("inote-util").WorkQueue;
var wq = new WorkQueue();
function taskOne(a,b,cb) {
console.log("taskOne is executing. a=",a,"b=",b);
cb(1,false,a);
}
var argsOne = [ "A", "B" ];
function taskTwo(cb) {
console.log("taskTwo is executing.");
cb(2,true,null);
}
function afterTask(num, exit, a) {
console.log("Task number",num,"is completed. a=",a);
if (exit) {
wq.stop_working();
}
}
wq.enqueue_task(taskOne, argsOne, afterTask);
wq.enqueue_task(taskTwo, afterTask);
wq.start_working();
When run, that script generates the following output:
taskOne is executing. a= A b= B
Task number 1 is completed. a= A
taskTwo is executing.
Task number 2 is completed. a= null
Events
The WorkQueue is also an EventEmitter
, with the following events:
-
work-beginning - called when the queue starts processing. Arguments: the WorkQueue instance.
-
task-enqueued - called when a task is added to the queue. Arguments: the WorkQueue instance and the task ({priority,method,args,callback}
).
-
task-dequeued - called when a task is selected for processing. Arguments: the WorkQueue instance and the task ({priority,method,args,callback}
).
-
task-completed - called when a task is done (and didn't throw an uncaught exception.) Arguments: the WorkQueue instance, the task and an array of arguments passed to the done
function..
-
error - called when an uncaught exception is encountered while processing the task. Arguments: the WorkQueue instance, the task and the error.
-
busy - called when a task is deferred because the maximum number of tasks are currently active. Arguments: the WorkQueue instance.
-
work-ending - called when the queue stops processing. Arguments: the WorkQueue instance.
Methods
-
new WorkQueue([options]) - create a new WorkQueue instance. Options:
priority
- the default priority for tasks in the queue (when none is specified when the task is added). Defaults to 5. Larger numbers are executed before smaller numbers.interval
- the time (in milliseconds) between "polling" the queue for new tasks to execute. Defaults to 200
.fuzz
- a floating point value that will be used to "fuzz" the interval between polling runs. When fuzz
is a non-zero float, a random value between -1 × fuzz × interval
and fuzz × interval
will be added to the interval. This is used to avoid synchronization when several WorkQueues are launched with the same configuration. (E.g., when a node app that uses the WorkQueue is launched in a cluster.) Defaults to 0.1
.workers
- the maximum number of tasks that can be active at one time. Defaults to 1
.
-
pending_task_count() - returns the number of tasks in the queue waiting to be executed.
-
active_task_count() - returns the number of tasks from the queue currently being executed.
-
enqueue_task:(method[,args][,priority][,callback]) - add a task the the queue.
-
start_working([options]) - start queue processing. The options
parameter may contain interval
or fuzz
attributes, which will override those provided in the construction for the duration of this processing.
-
stop_working() - stop queue processing. Note that if a queue is started but not stopped, a function will be called roughly every interval
milliseconds via JavaScript's setInterval
method.
Back to Index
ZipUtil
- zip([wd],zipfile,inputs,callback) - creates (or appends to)
zipfile
, adding the file or array of files in inputs
. When wd
is specified the action will take place relative to that directory. - unzip([wd],zipfile,dest,callback) - unzips
zipfile
into the specified dest
. - contents(zipfile,callback) - obtains a list of files within
zipfile
.
Back to Index
Errors
- TimeoutError
- ExceptionThrownError
Back to Index
Up to ToC
Installing
From Source
The source code and documentation for inote-util is available on GitHub at intellinote/inote-util. You can clone the repository via:
git clone git@github.com:intellinote/inote-util
Assuming you have node installed, you can then install the remaining dependencies by running npm install
from the root repository directory.
A Makefile
is provided. Run make install
to install. Run make test
to run the unit test suite. Run make help
for a list of other useful Make targets.
From npm
inote-util is deployed as an npm module under the name inote-util
. Hence you can install a pre-packaged version with the command:
npm install inote-util
and you can add it to your project as a dependency by adding a line like:
"inote-util": "latest"
to the dependencies
or devDependencies
part of your package.json
file.
Up to ToC
Licensing
The inote-util library and related documentation are made available
under an MIT License. For details, please see the file LICENSE.txt in the root directory of the repository.
Up to ToC
How to contribute
Your contributions, bug reports and pull-requests are greatly appreciated.
We're happy to accept any help you can offer, but the following
guidelines can help streamline the process for everyone.
-
You can report any bugs at
github.com/intellinote/inote-util/issues.
- We'll be able to address the issue more easily if you can
provide an demonstration of the problem you are
encountering. The best format for this demonstration is a
failing unit test (like those found in
./test/), but
your report is welcome with or without that.
-
Our preferred channel for contributions or changes to the
source code and documentation is as a Git "patch" or "pull-request".
-
If you've never submitted a pull-request, here's one way to go
about it:
- Fork or clone the repository.
- Create a local branch to contain your changes (
git checkout -b my-new-branch
). - Make your changes and commit them to your local repository.
- Create a pull request as described here.
-
If you'd rather use a private (or just non-GitHub) repository,
you might find
these generic instructions on creating a "patch" with Git
helpful.
-
If you are making changes to the code please ensure that the
unit test suite still passes.
-
If you are making changes to the code to address a bug or introduce
new features, we'd greatly appreciate it if you can provide one
or more unit tests that demonstrate the bug or
exercise the new feature.
-
This repository follows the "git flow" branching convention. The master
branch contains snapshots of each stable release. New development occurs within the develop
branch. We'd prefer most pull-requests to be based off the develop
branch.
Please Note: We'd rather have a contribution that doesn't follow
these guidelines than no contribution at all. If you are confused
or put-off by any of the above, your contribution is still welcome.
Feel free to contribute or comment in whatever channel works for you.
Up to ToC
Up to ToC