mcode-cache

MicroCODE's 'mcode-cache' package

A public NPM Package of our internal data caching tools for Frontend and Backend JavaScript NodeJS projects.

This is an extremely 'light weight' package with dependencies on our internal 'data' and 'log' packages and on node-cache and Redis for an optional caches.

Description

This is our own internal data caching code for common operations like caching files contents, data structures, lists (for out mcode-list processing), and automatic key creation and standardization.

• To demo, from the CLI in the package folder...

> node examples

• Example of package use...

• Corresponding results (logged to console by our mcode-log functions)...

Dependencies

• Production

• mcode-log - our standard logging package (just for displaying errors or test results)
• node-cache - an in memory local/app private cache from Node.js.
• Redis - Remote Dictionary Server, the standard community addition.

• Development

• Node.js - standard runtime environment
• JSDocs - our preferred JavaScript documentation system
• Jest.js - our preferred JavaScript testing framework

Usage

When using data caching it's best to start with a good definition of the objects your App will cache and why.

USE CASE #1: Files, for speed. When a Web App frequently goes to disk, HDD or SSD, to serve clients there is inherent latency. Automatically caching these frequenlty used files in RAM and serving them from there is typically 10X faster.

Normal, uncached code retrieving a file...

        const iconContent = await fs.readFile(iconPath, 'utf8');
                                  -----------

To cache for repeated usage with our package, just change "fs.readFile()" to "mcode.fileRead()"...

        const iconContent = await mcode.fileRead(iconPath, 'utf8');
                                  --------------

...this does four (4) things within one line change:

• Automatically generates a unique Cache Key under your App's namespace representing this file.
• Reads and returns the file contents.
• Caches the file for subsequent use.
• Retrieves the file from cache in the future with the exact same line of code in your app.
  
  Note what you did not have to do...

• No creation of a unique key required in your code.
• No explicit code to conditionally retrieve from cache vs. read the file.
• No storage of the key in your code to benefit from the caching, just read the same file path again with "mcode.fileRead()"
  
  

USE CASE #2: Context, for speed. When building a rich App a large part of the UX is context, the feeling that the App knows (and remembers) what you are doing. This is held in two forms of memory analogous tohuman 'short term' and 'long term' memory...

• CONTEXT - 'short term' memory - breadcrumbs, App module, forward/back navigation, etc.
• CONFIGURATION - 'long term' memory - user preferences, app settings, etc.

Both of these shoudl be cached, with different Time-To-Live (TTL), and different invalidation schemes. This is all handling in a standard way in our package, for our App designs.

USE CASE #3: Database, for speed. Because anything you can retrieve from RAM instead of the disk based DB will be at 10X to 50X faster.

Installing

• Get to a terminal session in the local repo folder of your project.
• Use 'npm install' to load the package. It can be used 'stand-alone'...

npm install mcode-cache

Testing

This package includes a simple demo module: examples.js. Running it directly will show you a set of examples for using fileRead() and fileChange().

• From your project directory after installation...

node .\node_modules\mcode-cache\examples

...this will demonstrate thru console logging various uses of the mcode-cache functions.

• To test with JEST:
• From the mcode-cache package directory...

npm install --save-dev jest
npm test

• A view of the JEST tests in the console...

• A view of the JSON now returned for cacheNamespaces() and cacheListAll()...

Included Functions

These are the functions we want at the ready in any module for development and debug.

Function	Description	Usage
General
addNamespace	Creates a new Namespace--in Node or Redis--for caching or accessing data.	mcode.addNamespace(({name: 'MicroCODE', type: 'node'})
cacheGet	Gets the value of a Key from the Cache, from App's namespace.	value = mcode.cacheGet(key, defaultCallback)
cacheSet	Sets the value of a Key from the Cache, in App's namespace.	mcode.cacheSet(key, value)
cacheDrop	Drops a key from the Cache.	count = mcode.cacheDrop(key)
cacheDropAll	Drops all keys from a namespace in the Cache, defaults to current.	count = mcode.cacheDropAll({cache: 'redis', namespace: 'GM-GPS-eMITS-DB', pattern: '*'})
cacheListAll	Lists all keys from a namespace in the Cache, defaults to current.	array = mcode.cacheListAll({cache: 'node', namespace: '', pattern: ''})
cacheMakeKey	Generates a well formatted Cache Key form a resource key.	key = mcode.cacheMakeKey(key)
cacheOn	Turns the caching of Node data ON. (The default state).	void mcode.cacheOn(cacheName)
cacheOff	Turns the caching of Node data OFF. (For active development).	void mcode.cacheOff(cacheName)
redisOn	Turns the caching of Redis data ON. (The default state).	DEPRECATED in v0.8.0
redisOff	Turns the caching of Redis data OFF. (For active development).	DEPRECATED in v0.8.0
cacheClose	Closes the Node and Redis caches, and the connection to the Redis Server.	void mcode.cacheClose(path)
File Specific	These directly replace "fs" file operations
fileRead	Reads a file from storage with a standard 'path' and caches it.	contents = mcode.fileRead(path, encoding)
fileWrite	Writes a file to storage with a standard 'path' and caches it.	state = mcode.writeRead(path, contents, encoding)
fileDrop	Invalidates a standard 'path', forcing a fresh read/cache on next access.	count = mcode.fileDrop(path)
fileMakeKey	Generates a well formatted Cache Key from a standard file path.	key = mcode.fileMakeKey(path)
fileGetRoot	Gets the root directory for Cache Keys based on app's execution path.	path = mcode.fileGetRoot(path)

Included Properties

These are the properties for interacting with the mcode-cache instance.

Property	Description	Usage
cacheNamespaces	The active namespaces and their types (Node or Redis).	const namespaces = await mcode.cacheNamespaces;
cacheReady	The Cache is ready for use, Redis online, Namespace is established.	if (mcode.cacheReady)
cacheTTL	The current Time-To-Live, the expiration in milliseconds of current tags.	mcode.cacheTTL = 30000
redisURL	The network address of the Redis Server, 'redis://:.	mcode.redisURL = 'redis://127.0.0.1:6379'
cacheNamespace	The namespace for all tags until changed, defaults to 'MicroCODE'.	mcode.cacheNamespace = 'MyAppName'
cacheEnabled	The current state of Node namespace caching, True = Caching is ON.	if (mcode.cacheEnabled(cacheName))
redisEnabled	The current state of Redis namespace caching, True = Caching is ON.	DEPRECATED in v0.8.0

 

Documentation

We believe is explicit code documentation, for other users, and for our 'future selves'.
JSDocs is a standardized system for documenting functions and data structures that produces three (3) primary outputs:

• Inline documentation for the coder.
• Intellisense popup documentation for the coder for every function.
• External 'reference manual' documentation for your entire code base, if used consistently.

• This entire project--like all our projects--is documented with JSDocs.

• To install JSDocs use, get to a terminal session in the project folder...

npm install --save-dev jsdoc

• Configure JSDoc processing in...

jsdoc.json

• To regenerate the JSDocs from all source code, use the following command (from the project root directory)...

jsdoc -c .jsdoc.json

...then open ./docs/index.html

Help

Contact Timothy McGuire, [e-mail].

Terminology

Word or Acronym	Description/Definition
NPM	Node Package Manager, actually “Node PM”, “Node pkgmakeinst” a system to deploy, install, and maintain NodeJS Apps. (PM was a BASH utility).
NVM	Node Version Manager, a tool that supports changing NodeJS versions.
MERN	MongoDB, Express, React, Node JS.
MongoDB	A ‘NoSQL’ database designed for Cloud applications, also referred to as a ‘Document Store’.
Express	Express is not a database but rather an ‘extensible routing language’ for communication between a Client and a Server.
React	A Web UI development system, a JavaScript library developed by Facebook and made public—and Open Source—since 2013.
Redis	A Remote Dictionary Server, a standard through the web development industry.
node-cache	The Node.js Cache for 'in app' caching on the App's own server.
Node.js	A development stack that executes from a local file store—on a local Server—instead of from a network of servers.
JSDocs	A toolset to automatically generate API-style documentation from source code tagging.

Authors

Contributor's names and contact info...

• Timothy McGuire @TimothyMcGuire - Founder, President-CEO of MicroCODE, Inc. a software and controls engineering company in Detroit, Michigan USA.

Version History

• 0.8.1

  • Updated dependencies to mcode-data v0.6.4 and mcode-log v0.8.0.

• 0.8.0

  • NOTE: This is a BREAKING CHANGE, several functions now require parameters for distinguishing which cache is being referenced.
  • Updated 'cacheNamespaces' and 'cacheListAll' to include more detailed type and stats information and return proper JSON structure.
  • This record structure is different from previous versions.
  • REDIS is just treated as a common CACHE type now without specific API Endpoints.
  • Add 20+ Test cases for the new and updated functions.

• v0.7.1

  • Moved to _ naming for all internal use of our packages from within another package.
  • Corrected 'get' functions to return true 'undefined' if the key is not found in the cache, instead of a string ''.

• v0.7.0

  • Added more standardized JSDoc module headers for inclusion in end-product documentation.
  • upgraded to mcode-data v0.6.0, mcode-log v0.6.0
  • Corrected internal usage of mcode-log to use log.() instead of mcode.log.() -- all are in mcode.
  • Corrected intenral usage of mcode-data to use data.() instead of mcode.data.() -- all are in mcode.
  • Removed dependency on mcode-package, this belongs to that, not the other way around.

• v0.6.10

  • Updated to mcode-data v0.5.10 and mcode-log v0.5.10, corrected internal use of mcode-log ==> log.(), after adding mcode-data (for data.default()) I saw I was using it as mcode.log() instead of log.().

• v0.6.9

  • Updated to mcode-data v0.5.9 and mcode-log v0.5.9.

• v0.6.8

  • Updated to mcode-data v0.5.8 and mcode-log v0.5.8.

• v0.6.7

  • Minor bug fixes and performance improvements.

• v0.6.6

  • Enhanced error handling for cache operations.

• v0.6.5

  • Added file access verification to Read and Write with condition handlers.
  • Corrected cache enable/disable switch on REDIS Cache
  • Corrected REDIS defaults for Port, Username and Password

• v0.6.4

  • Corrected cross-spawn 7.0.0 - 7.0.4; Severity: high; Regular Expression Denial of Service
  • Update mcode-log, mcode-data to v0.5.7 similar corrections.

• v0.6.3

  • Added USER and PASSWORD to optional REDIS Params for connection.

• v0.6.2

  • Updated mcode-data to v0.5.5 and mcode-log to v0.5.5

• v0.6.1

  • Corrected 'cacheGet()' and documented the optional callback for a default methd.

• v0.6.0

  • Added support for multiple caches: node-cache (new default), and kept Redis as an option.
  • One of the original purposes of this package was easy/fast data caching within an App, having to go to a network connected Redis defeats that purpose and so we added node-cache as the default.
  • This is BREAKING change, several functions now require parameters for distinguishing which cache is being referenced.

• v0.5.5

  • Remove debug logging.

• v0.5.4

  • Added Cache control for testing and debug: redisCaching, redisCacheOn(), redisCacheOff().

• v0.5.3

  • Corrected a namespace prefixing error.

• v0.5.0 - 0.5.2

  • All 'mcode-*' packages updated with 'ready()' only implemented in 'mcode-log'.

• v0.4.0

  • made public and synchronized into mcode-package along with mcode-data, mcode-log, mcode-list.

• v0.0.5

  • made public to test integration - NOT READY FOR USE.

• v0.0.4

  • Updated README, uninstalled JSDocs and Jest for publishing.

• v0.0.3

  • Corrected JSDocs and Jest to DEV ONLY dependencies.

• v0.0.2

  • Added JSDocs, Jest and updated README.

• v0.0.1

  • Initial movement of our internal code into an NPM package for ease of use in other projects.

Future Development

• v0.1.*
  • Any additional core code we will develop for general list processing work.
  • Complex function execution with passed arguments or passed functions.

License

This project is licensed under the MIT License - see the LICENSE.md file for details

MicroCODE Mantra

MicroCODE, Inc. was founded in 1987 as a controls engineering and software development company.
We specialize in manufacturing and quality control applications that must run 24x7x365 for years at a time.

Our slogan, distilled from over three decades of developing, testing, installing, and supporting 24x7x365 manufacturing applications, is..