Research
Security News
Quasar RAT Disguised as an npm Package for Detecting Vulnerabilities in Ethereum Smart Contracts
Socket researchers uncover a malicious npm package posing as a tool for detecting vulnerabilities in Etherium smart contracts.
rest-easy-loki
Advanced tools
A REST interface for lokijs supporting CRUD operations and automatic creation of new collections.
A simple REST interface for the in-memory database, lokijs
, featuring:
LOKI_
via RESTThis version has moved from the default LokiFsAdapter
to the more performing LokiFsStructuredAdapter
. Besides a performance gain, it also means that we don't end up with a single database file anymore, but one overall database file and one per collection.
npm install # Or pnpm i
npm start # Will transpile the TypeScript project to JavaScript and run node on every change.
To simply run the lokijs
server and expose the CRUD services.
npm run serve
To embed it in your own project, do something like the following:
import * as Koa from 'koa'; // You only need to include @types/koa in your devDependencies, not Koa itself.
import { createApi, db } from 'rest-easy-loki';
export const collectionName = 'documents';
const port = process.env.LOKI_PORT || '3000';
const dbName = process.env.LOKI_DB;
const cors = (process.env.LOKI_CORS || 'true') === 'true';
const sizeLimit = process.env.LOKI_SIZE_LIMIT || '25mb';
export const startService = () => {
db.startDatabase(dbName, () => {
const { api } = createApi({
cors,
sizeLimit,
compression: true, // Compress data using gzip
upload: 'upload', // Allow uploading data to this folder
public: 'public' // Serve all files in this folder, e.g. SPA
}) as Koa;
api.listen(port);
console.log(`Server running on port ${port}.`);
});
};
startService();
Reads .env
file for specifying the database name, port, CORS and message size limits. E.g.
LOKI_PORT=3030
LOKI_DB="simple.db"
LOKI_CORS=true
LOKI_COMPRESSION=true
LOKI_CONFIG="config.json"
LOKI_POLICIES="policies.json"
LOKI_SIZE_LIMIT="250mb"
LOKI_PRETTY=true
LOKI_DEBUG=false
LOKI_AUTHZ_JWT_SHARED=""
LOKI_AUTHZ_JWT_JWKS=""
LOKI_AUTHZ_READ=""
LOKI_AUTHZ_CREATE="key1"
LOKI_AUTHZ_UPDATE="key1"
LOKI_AUTHZ_DELETE="key1"
LOKI_AUTHZ_WHITELIST="localhost"
When creating the database for the first time, you optionally can also configure the database collections using LokiJS options, e.g. by specifying unique property names, or properties that must be indexed. In addition, you can import any existing JSON file in one go. For example, see config.json
below: with it, you create two collections, users
and projects
, and each collection has a unique property id
and several indices. In addition, it imports the file specified by jsonImport
.
{
"collections": {
"users": {
"jsonImport": "./employees.json",
"unique": ["id"],
"indices": ["first", "last", "keywords", "summary"]
},
"projects": {
"jsonImport": "./projects.json",
"unique": ["id"],
"indices": ["name", "keywords", "summary"]
}
}
}
The configuration file needs to adhere to the ILokiConfiguration
interface, as specified below:
/** From LokiJS typings, but not exported */
export interface CollectionOptions<E> {
disableMeta: boolean;
disableChangesApi: boolean;
disableDeltaChangesApi: boolean;
adaptiveBinaryIndices: boolean;
asyncListeners: boolean;
autoupdate: boolean;
clone: boolean;
cloneMethod: 'parse-stringify' | 'jquery-extend-deep' | 'shallow' | 'shallow-assign' | 'shallow-recurse-objects';
serializableIndices: boolean;
transactional: boolean;
ttl: number;
ttlInterval: number;
exact: (keyof E)[];
unique: (keyof E)[];
indices: keyof E | (keyof E)[];
}
export interface ExtendedCollectionOptions<E> extends CollectionOptions<E> {
/** JSON file to import: expects a JSON array which will be inserted into the collection */
jsonImport?: string;
}
export interface ILokiConfiguration<T = {}> {
/** Create collections on startup if there are no collections yet */
collections?: {
/** Name of the collection */
[collectionName: string]: ExtendedCollectionOptions<T>;
};
}
If you do specify one or more unique names, you can query the REST interface via https://localhost:3000/api/COLLECTION_NAME/USERS/THOR.
application/json
body to https://localhost:3000/api/COLLECTION_NAME.$loki
ID: https://localhost:3000/api/COLLECTION_NAME/ID.UNIQUE_NAME
: https://localhost:3000/api/COLLECTION_NAME/UNIQUE_PROP_NAME/PROP_VALUE.$loki
ID: Make a DELETE request to https://localhost:3000/api/COLLECTION_NAME/ID.application/json
body to https://localhost:3000/api/COLLECTION_NAME/ID. Alternatively, change the original item (from the GET, so including $loki
ID) and PUT it back to https://localhost:3000/api/COLLECTION_NAMEapplication/json
body to https://localhost:3000/api/COLLECTION_NAME/ID. The send patch object is defined as specified below. In case saveChanges
is specified, the patch is also saved to the appropriate collection (after removing the saveChanges
property).export interface IMutation extends ILokiObj {
/**
* Save changes to collection: if set, save this object,
* except the `saveChanges` property, to the `saveChanges` collection
*/
saveChanges?: string;
/** RFC6902 JSON patch */
patch?: Operation[];
}
q={"name": "a name"}
: https://localhost:3000/api/COLLECTION_NAME?from=0&to=10&q=%7B%20%22name%22:%20%22My%20third%20lesson%22%20%7Dq={"name": {"$ne": "a name"}}
: https://localhost:3000/api/COLLECTION_NAME?from=0&to=10&q=%7B%20%22name%22:%20%7B%20%22$ne%22:%20%22My%20third%20lesson%22%20%7D%7D.You can use the public
folder for sharing static files or your own web application. Enabled by default.
You can use the upload
folder for uploading files to a (automatically created) CONTEXT folder, if enabled on start-up using the -u
instruction. Test it via curl -F "file=@filename.jpg" http://localhost:3030/upload/:CONTEXT
. Files will be served from http://localhost:3030/:CONTEXT/ORG_FILENAME
. When uploading the same filename in the same context, the previous version will be overwritten. No index file is created, so the contents of the locally created folders are not visible externally. Also note that the CONTEXT supports sub-folders too.
If enabled using the io
flag (or -i) so clients can subscribe to receive updates when a value has changed. Clients can either subscribe to a collection socket.subscribe('COLLECTION_NAME')
, or to a collection item socket.subscribe('COLLECTION_NAME/$LOKI')
. The latter is, for example, useful when you have multiple editors. Subscribers receive the updated item.
The http://localhost:3000/api/env serves all environment variables that start with LOKI_
(except the LOKI_AUTHZ_
, so you don't accidentally share secrets). Since all key-value pairs are strings, a type conversion to boolean, number and arrays (using the , as separator) is performed.
LOKI_AUTHZ_CREATE
, LOKI_AUTHZ_READ
, LOKI_AUTHZ_UPDATE
, LOKI_AUTHZ_DELETE
, where the value is a comma-separated list of API keys. If no authz
is set, all CRUD actions are allowed. Otherwise, your query needs to set the x-api-key
header.LOKI_AUTHZ_JWT_SHARED
. This disables the other LOKI_AUTHZ_
methods. The JWT token has to be generated by the same shared key as used here. For testing purposes, you can create JWT tokens online. JWT tokens should be given via the Authorization
header as a Bearer
token, i.e. Authorization: Bearer <YOUR_JWT>
. Set LOKI_AUTHZ_JWT_ANONYMOUS_READ
to true
to allow anonymous reads.LOKI_AUTHZ_JWT_JWKS
. This disables the other LOKI_AUTHZ_
methods. The JWT token is generated as part of an OIDC flow. JWT tokens should be given via the Authorization
header as a Bearer
token. Set LOKI_AUTHZ_JWT_ANONYMOUS_READ
to true
to allow anonymous reads.For both JWT methods, the JWT should contain realm_access.roles
key with a list of roles.
If the editor
role is in there, access is granted to write to the database.
When using JWT or JWKS authorization, you can specify route based access control. In that case, the LOKI_POLICIES
should be set to a JSON file with the following structure (as specified in the rule-policy-schema.json
file):
{
"$schema": "./rule-policy-schema.json",
"rules": [
{
"method": "GET",
"path": "/api/users/:sub"
},
{
"method": "GET",
"path": "/api/users/*",
"abac": {
"roles": "admin"
}
},
{
"method": "GET",
"path": "/api/cases",
"abac": {
"roles": "admin"
}
},
{
"method": "POST",
"path": "/api/users",
"abac": {
"roles": "admin"
}
},
{
"method": "GET",
"path": "/api/cases",
"query": {
"q": "{ 'members': { '$contains': ':sub' } }"
}
}
]
}
Currently, there are two main open source libraries for access control, both used by Amazon Web Services and others: Open Policy Agent (OPA) and Cedar. See the comparison here. Although the former is more general, and also used to control access to all kinds of micro-services, it has a steeper learning curve. Cedar is more to the point, and focusses on Role-based Access Control (RBAC) or Attribute-based Access Control (ABAC) with a subject, action and resource. However, both suffer from one major drawback, namely using attributes from the resource to determine access.
Consider the following example: you allow a user to create case files. In each case file, the user specifies other users that have access too. Let's assume the database holds a collection of case files (rows or objects in my database), and each case contains a members
array with the user IDs of the users that have access. When users queries the server for their case files, the authorization function (the Policy Decision Point or PEP) would need to know the attributes of each case file to determine whether that user has access, i.e. is the user's ID part of the case file's members
property. However, this would mean that the server first needs to query the database to get access to all case files, potentially thousands, and next the authorization service would need to verify each of them. Clearly, this does not scale, so the common solution is called partial evaluation, where the authorization service only verifies the attributes of the subject and the action, but skips the verification of the resource attributes. In case access is partially granted, the resource attributes that are not are converted to a SQL or other database query, which is subsequently executed. As the latter step is not trivial, and open to security issues, this repository has chosen a different approach which is coined Route-based Access Control (RouBAC).
How does RouBAC work: you create a policy file, a JSON object that contains a list of rules (see the example below). Each rule is evaluated until a rule allows access, or all rules are processed, in which case access is denied.
How does the rule evaluation work:
:sub
, are present in the user's JWT payload, e.g. the first rule only allows access to /users/123
if the payload contains a property sub
whose value is 123 (using strict checking).abac
object, if present, is checked against the user's JWT payload, so the second rule only allows access to GET /users
if the payload contains a property roles
whose value is admin
. In case roles
is an array, access is permitted only if the rule's role is a subset of the user's roles. Note that the role only allows a single string, so if the editor
role has access too, a new rule is needed.realm_access.roles
would also work.You can use Bruno to test a few policies that are found in the test-rest-easy-loki
folder.
A general 401 error will be thrown if the Authorization
header is missing or malformed, and when a token is expired or incorrectly signed.
In other words, rest-easy-loki
will not tell you what is wrong exactly.
In a similar vein, no feedback is given about the specific reason why a request does not match any policy rule.
This means that you should critically review your requests and policy file for formatting, especially regarding the use of whitespace, quotation marks, and regex.
Also, make sure that any used JWT properties are present and correct if you use placeholders.
For example, the following GET request:
https://example.org/api/reports?q={%22case%22:%2242%22,%22editor%22:%22someuser@somedomain.org%22,%20%22allowed%22:%20{%20%22$contains%22:%20%22someotheruser@somedomain.org%22}}
would satisfy the below policy rule (if the signed JWT contains a property email
with value someotheruser@somedomain.org
):
{
"method": "GET",
"path": "/api/reports",
"query": {
"q": "{ 'case': '\\w+', 'editor': '.+', 'allowed': { '$contains': ':email' } }"
}
},
The LOKI_DEBUG
environment variable can help you debug your requests and policies.
FAQs
A REST interface for lokijs supporting CRUD operations and automatic creation of new collections.
The npm package rest-easy-loki receives a total of 1 weekly downloads. As such, rest-easy-loki popularity was classified as not popular.
We found that rest-easy-loki demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 0 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Socket researchers uncover a malicious npm package posing as a tool for detecting vulnerabilities in Etherium smart contracts.
Security News
Research
A supply chain attack on Rspack's npm packages injected cryptomining malware, potentially impacting thousands of developers.
Research
Security News
Socket researchers discovered a malware campaign on npm delivering the Skuld infostealer via typosquatted packages, exposing sensitive data.