Security News
tea.xyz Spam Plagues npm and RubyGems Package Registries
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
jsonreststores
Advanced tools
Readme
Rundown of features:
NOTE. When creating a store, you can take the following shortcuts:
paramIds
. If not specified, it will be worked out from publicURLidProperty
. If idProperty not specified, it will be assumed last element of paramIdsImagine that you have a web application with bookings, and users connected to each booking, and that you want to make this information available via a JSON Rest API. You would have to define the following routes in your application:
GET /bookings/
GET /bookings/:bookingId
PUT /bookings/:bookingId
POST /bookings/
DELETE /bookings:/bookingId
And then to access users for that booking:
GET /bookings/:bookingId/users/
GET /bookings/:bookingId/users/:userId
PUT /bookings/:bookingId/users/:userId
POST /bookings/:bookingId/users/
DELETE /bookings/:bookingId/users/:userId
It sounds simple enough (although it's only two tables and it already looks rather boring). It gets tricky when you consider that:
GET /bookings/
, you need to parse the URL in order to enable data filtering (for example, GET /bookings?dateFrom=1976-01-10&name=Tony
will need to filter, on the database, all bookings made after the 10th of January 1976 by Tony).GET /bookings/
, you need to return the right Content-Range
HTTP headers in your results so that the clients know what range they are getting.GET /bookings/
, you also need to make sure you take into account any Range
header set by the client, which might only want to receive a subset of the dataPUT
, you need to consider the HTTP headers If-match
and If-none-match
to see if you can//should//must overwrite existing records501 Unimplemented Method
server responseThis is only a short list of obvious things: there are many more to consider. The point is, when you make a store you should be focusing on the important parts (the data you gather and manipulate, and permission checking) rather than repetitive, boilerplate code.
With JsonRestStores, you can create JSON REST stores without ever worrying about any one of those things. You can concentrate on what really matters: your application's data, permissions and logic.
If you are new to REST and web stores, you will probably benefit by reading a couple of important articles. Understanding the concepts behind REST stores will make your life easier. I suggest you read John Calcote's article about REST, PUT, POST, etc.. It's a fantastic read, and I realised that it was written by John, who is a long term colleague and fellow writer at Free Software Magazine, only after posting this link here!
JsonRestStores handles everything except:
Data-oriented calls are the bridge between your stores and your actual data. Data can be stored in a database, text files, or even remotely.
Here is the list of calls that must be implemented -- and which methods need them:
implementFetch(request)
. Required for methods put
, get
, delete
.implementInsert(request)
. Required for methods post
and put
(new records)implementUpdate(request)
. Required for method put
(existing records)implementDelete(request
. Required for method delete
.implementQuery(request)
. Required for methods getQuery
.Looking it from a different perspective, here are the implement***
methods you will need to implement for each method to work properly:
get
: implementFetch()
.getQuery
: implementQuery()
put
: implementFetch()
, implementInsert()
, implementUpdate()
post
: implementInsert()
delete
: implementFetch()
, implementDelete()
When developing these methods, it's important to make sure that they function exactly as expected.
implementFetch(request)
This method is used to fetch a single record from the data source. The attributes taken into consideration in request
are:
request.remote
.request.params
. Sets the filter to search for the record to be fetched. For local request, it's acceptable to just match the key in request.params
matching self.idProperty
. For remote requests, it's best to filter data so that every key in request.params
matches the equivalent key in the record. This means that fetching /manager/3/document/44
ought to create a query where managerId
is 3
and documentId
is 44
Returning a falsy value implies that the item wasn't found.
implementInsert(request)
This method is used to add a single record to the data source. The attributes taken into consideration in request
are:
request.remote
.request.body
. The record that will effectively be written onto the databaserequest.options.placement
. If set, it can be first
(place the record first), last
(place the record last) or after
(place the record after another one)request.options.placementAfter
. If request.options.placement
is after
, the new record will be placed after the one with ID matching placementAfter
.The inserted record should be returned by this method.
implementUpdate(request)
This method is used to update a single record in the data source. The attributes taken into consideration in request
are:
request.remote
.request.params
. Sets the filter to search for the record to be updated. See implementFetch()
request.body
. The fields to be updated in the data source. Even though it depends on design decisions, generally speaking each field in the schema should be updatedrequest.field
. If set, this is a field update. This means that the query is only meant to update one specific field.request.options.placement
. If set, it can be first
(place the record first), last
(place the record last) or after
(place the record after another one)request.options.placementAfter
. If request.options.placement
is after
, the new record will be placed after the one with ID matching placementAfter
.The updated record should be returned by this method.
implementDelete(request)
This method is used to delete a single record from the data source. The attributes taken into consideration in request
are:
request.remote
.request.params
. Sets the filter to search for the record to be deleted. See implementFetch()
The deleted record should be returned by this method.
implementQuery(request)
This method is used to fetch a set of records from the data source. The attributes taken into consideration in request
are:
request.remote
.request.params
. For remote requests, adds filtering restrictions to the query so that only matching records will be fetched. For local request, such extra filtering should be avoided.request.options
. The options object that will define what data is to be fetched. Specifically:
request.options.conditions
: an object specifying key/value criteria to be applied for filtering.request.options.sort
: an object specifying how sorting should happen. For example { surname: -1, age: 1 }
.request.options.range
: an object with up to attributes:
limit
must make sure that only a limited number of records are fetched;skip
must make sure that a number of records are skipped.The list of returned records (as array) should be returned by this method.
Creating a store is a matter of implementing the calls described above. You can be lazy (or smart), and create stores based on MySql and using schema validation by using jsonreststores-mysql which comes with all of those functions implemented automatically, as well as the ability to customise queries, data validation and permissions.
jsonreststores-mysql is the ideal solution for DB-oriented stores; however, there is often a strong case for non-DB stores (see: fetching data remotely on-the-fly, or providing endpoints which will perform complex maintenance operations, etc.)
Creating a store with JsonRestStores is very simple. Here is code that can be plugged directly into an Express application:
var Store = require('jsonreststores2') // The main JsonRestStores module
var HTTPMixin = require('jsonreststores2/HTTPMixin') // HTTP Mixin
var Schema = require('simpleschema2') // The main schema module
// This is for MySql
var promisify = require('util').promisify
var mysql = require('mysql')
// ======= BEGIN: Normally in separate file in production =============
// Create MySql connection. NOTE: in production this will likely be placed
// in a separate file, so that `connection` can be required and shared...
var connection = mysql.createPool({
host: 'localhost',
user: 'user',
password: 'YOUR PASSOWORD',
database: 'testing',
port: 3306
})
// Promisified versions of the query function for MySql
connection.queryP = promisify(connection.query)
// ======= END: Normally in separate file in production =============
// Make up an object that will contain all of the stores
var stores = {}
// Basic definition of the managers store
class Managers extends HTTPMixin(Store) {
static get schema () {
return new Schema({
name: { type: 'string', trim: 60 },
surname: { type: 'string', searchable: true, trim: 60 }
})
}
static get storeName () { return 'managers' }
static get publicURL () { return '/managers/:id' }
static get handlePut () { return true }
static get handlePost () { return true }
static get handleGet () { return true }
static get handleGetQuery () { return true }
static get handleDelete () { return true }
async implementFetch (request) {
return (await connection.queryP('SELECT * FROM managers WHERE id = ?', request.params.id))[0]
}
async implementInsert (request) {
let insertResults = await connection.queryP('INSERT INTO managers SET ?', request.body)
let selectResults = await connection.queryP('SELECT * FROM managers WHERE id = ?', insertResults.insertId)
return selectResults[0] || null
}
async implementUpdate (request) {
await connection.queryP('UPDATE managers SET ? WHERE id = ?', [request.body, request.params.id])
return (await connection.queryP('SELECT * FROM managers WHERE id = ?', request.params.id))[0]
}
async implementDelete (request) {
let record = (await connection.queryP('SELECT * FROM managers WHERE id = ?', request.params.id))[0]
await connection.queryP('DELETE FROM managers WHERE id = ?', [request.params.id])
return record
}
async implementQuery (request) {
var result = await connection.queryP(`SELECT * FROM managers LIMIT ?,?`, [ request.options.skip || 0, request.options.limit || 0 ])
var grandTotal = (await connection.queryP(`SELECT COUNT (*) as grandTotal FROM managers`))[0].grandTotal
return { data: result, grandTotal: grandTotal }
}
}
stores.managers = new Managers()
exports = module.exports = stores
In app.js, after app.use(express.static)
, you would have:
var stores = require('./stores.js');
stores.managers.protocolListenHTTP({app: app});
You will also need to create your MySql table in the 'testing' database:
CREATE TABLE managers (
id INT(10) PRIMARY KEY NOT NULL AUTO_INCREMENT,
name VARCHAR(60),
surname VARCHAR(60)
);
That's it: this is enough to add, to your Express application, a full store which will handle properly all of the HTTP calls. Note that the database side of things has no sanity check: that's because sanity checking has already happened at this stage.
Also note that this setup will query a MySql database; however, anything can be the source of data.
Note that:
Managers
is a new constructor function that inherits from Store
(the main constructor for JSON REST stores) mixed in with HTTPMixin
(which implements protocolListenHTTP()
schema
is an object of type Schema that will define what's acceptable in a REST call.publicURL
is the URL the store is reachable at. The last one ID is the most important one: the last ID in publicURL
(in this case it's also the only one: id
) defines which field, within your schema, will be used as the record ID when performing a PUT and a GET (both of which require a specific ID to function).storeName
(mandatory) needs to be a unique name for your store.handleXXX
are attributes which will define how your store will behave. If you have handlePut: false
and a client tries to PUT, they will receive an NotImplemented
HTTP error.protocolListen( 'HTTP', { app: app } )
creates the right Express routes to receive HTTP connections for the GET
, PUT
, POST
and DELETE
methods.A bit of testing with curl
:
$ curl -i -XGET http://localhost:3000/managers/
HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: application/json; charset=utf-8
Content-Length: 2
ETag: "223132457"
Date: Mon, 02 Dec 2013 02:20:21 GMT
Connection: keep-alive
[]
curl -i -X POST -d "name=Tony&surname=Mobily" http://localhost:3000/managers/
HTTP/1.1 201 Created
X-Powered-By: Express
Location: /managers/2
Content-Type: application/json; charset=utf-8
Content-Length: 54
Date: Mon, 02 Dec 2013 02:21:17 GMT
Connection: keep-alive
{
"id": 2,
"name": "Tony",
"surname": "Mobily"
}
curl -i -X POST -d "name=Chiara&surname=Mobily" http://localhost:3000/managers/
HTTP/1.1 201 Created
X-Powered-By: Express
Location: /managers/4
Content-Type: application/json; charset=utf-8
Content-Length: 54
Date: Mon, 02 Dec 2013 02:21:17 GMT
Connection: keep-alive
{
"id": 4,
"name": "Chiara",
"surname": "Mobily"
}
$ curl -i -GET http://localhost:3000/managers/
HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: application/json; charset=utf-8
Content-Length: 136
ETag: "1058662527"
Date: Mon, 02 Dec 2013 02:22:29 GMT
Connection: keep-alive
[
{
"id": 2,
"name": "Tony",
"surname": "Mobily"
},
{
"id": 4,
"name": "Chiara",
"surname": "Mobily"
}
]
$ curl -i -X PUT -d "name=Merc&surname=Mobily" http://localhost:3000/managers/2
HTTP/1.1 200 OK
X-Powered-By: Express
Location: /managers/2
Content-Type: application/json; charset=utf-8
Content-Length: 54
Date: Mon, 02 Dec 2013 02:23:50 GMT
Connection: keep-alive
{
"id": 2,
"name": "Merc",
"surname": "Mobily"
}
$ curl -i -XGET http://localhost:3000/managers/2
HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: application/json; charset=utf-8
Content-Length: 54
ETag: "-264833935"
Date: Mon, 02 Dec 2013 02:24:58 GMT
Connection: keep-alive
{
"id": 2,
"name": "Merc",
"surname": "Mobily"
}
It all works!
It's important to be consistent in naming conventions while creating stores. In this case, code is clearer than a thousand bullet points:
var Managers = declare( Store, {
schema: new Schema({
// ...
});
publicURL: '/managers/:id',
storeName: `managers`
// ...
}
var managers = new Managers();
var People = declare( Store, {
schema: new Schema({
// ...
});
publicURL: '/people/:id',
storeName: `people`
// ...
}
var people = new People();
storeName
) should be lowercase and are plural (they are collections representing multiple entries)Store
) are in capital letters/Capital/Urls/Are/Silly
)This is the comprehensive list of errors the class can create:
BadRequestError
Like this: new BadRequestError( { errors: errors } )
UnauthorizedError
ForbiddenError
NotFoundError
PreconditionFailedError
UnprocessableEntityError
Like this: UnprocessableEntityError( { errors: errors } )
NotImplementedError
ServiceUnavailableError
. Like this: ServiceUnavailableError( { originalErr: error } )
These error constructors are borrowed from the Allhttperrors module -- you should its short and concise documentation. The short version is that errorObject.status
and errorObject.httpError
will be set as the error number, and the constructor can have either a string or an object as parameters.
The error objects are all pretty standard. However:
ServiceUnavailableError
errors will be created with an originalErr
parameter containing the original error object. For example if the database server goes down, the module will return a ServiceUnavailableError
error object which will include the original MongoDB error in its originalErr
parameter.
UnprocessableEntityError
and BadRequestError
are both created when field validation fails. They error objects will always have an errors
attribute, which will represent an array of errors as they were returned by SimpleSchema. For example:
[ { field: 'nameOfFieldsWithProblems', message: 'Message to the user for this field' }, { field: 'nameOfAnotherField', message: 'Message to the user for this other field' }, ]
JsonRestStores only ever throws (generic) Javascript errors if the class constructor was called incorrectly, or if an element in paramIds
is not found within the schema. So, it will only ever happen if you use the module incorrectly. Any other case is chained through.
At some point in your program, one of your callbacks might have the dreaded err
first parameter set to an error rather than null. This might happen with your database driver (for example your MongoDB process dies), or within your own module (validation after a PUT
fails).
JsonRestStores allows you to decide what to do when this happens.
chainErrors
You can control what happens when an error occurs with the chainErrors
attribute. There are three options:
all
If you have chainErrors: all
in your class definition: JsonRestStores will simply call next( error )
where error
is the error object. This means that it will be up to another Express middleware to deal with the problem.
none
If you have chainErrors: none
in your class definition: if there is a problem, JsonRestStores will not call the next()
callback at all: it will respond to the client directly, after formatting it with the object's self.formatErrorResponse()
method (see below).
Do this if you basically want to make absolute sure that every single request will end right there, whether it went well or not. If you do this, you might want to define your own self.formatErrorResponse()
method for your store classes, so that the output is what you want it to be.
nonhttp
If you have chainErrors: nonhttp
in your class definition, JsonRestStores will only call next( err )
for non-HTTP errors -- any other problem (like your database server going down) will be handled by the next Express error management middleware. Use this if you want the server to respond directly in case of an HTTP problem (again using self.formatErrorResponse()
to send a response to the client), but then you want to manage other problems (for example a database problem) with Express.
self.formatErrorResponse()
In those cases where you decide to send a response to the client directly (with chainErrors
being none
or nonhttp
), the server will send a response to the client directly. The body of the response will depend on what you want to send back.
The stock self.formatErrorResponse()
method will simply return a Json representation of the error message and, if present, the errors
array within the error.
self.logError()
Whenever an error happens, JsonRestStores will run self.logError()
. This happens regardless of what self.chainErrors
contains (that is, whether the error is chained up to Express or it's managed internally). Note that there is no callback
parameter to this method: since it's only a logging method, if it fails, it fails.
The reponse is delivered by JsonRestStores using the function protocolSendHTTP
for HTTP transport, and protocolSendXXX
for XXX transport. At the moment, only HTTP is implemented via the HTTPMixin
mixin. protocolSendXXX
has the following signature:
protocolSendXXX( request, method, data, status, cb )
In case there was no error, the parameters are as follows:
request
: the request objectmethod
: the method being handled; it can be get
, getQuery
, put
, post
, delete
,data
: the data being delivered.status
: the status set by JsonRestStores. For non-errors, the status is set as:
get
: 200.getQuery
: 200put
: 201 for newly created records, and 200 for overwriting of existing recordspost
: 201delete
: 204In case there was an error, the store will only send data if chainError
is set to none
or nonhttp
(and the error isn't an HTTP error). Otherwise, it will simply call the callback with the error as its first parameter, as per normal in node (it will be up to your application to manage the error, which is simply pussed up the callback chain).
If chaining is off, and a response is to be sent to the client, then protocolSendXXX
will be used. In this case, the data
argument of protocolSendXXX
will be set as the error itself, with two properties added to it:
formattedErrorResponse
(the formatted error response, formatted thanks to the store's formatErrorResponse()
method)originalMethod
(the method that triggered the error).Also, status
will be set to the HTTP status code.
protocolSendXXX()
is always passed an HTTP error. In case the error was caused by something else (for example, the connection to the database dropped), the error will be set as 503, and the error object will also have a originalErr
attribute which represents the original error that triggered the 503 HTTP error.
HTTPMixin
TODO: Document HTTPMixin, explaining how each header is transformed into an option
FAQs
A module to create full Json REST stores in minutes
The npm package jsonreststores receives a total of 100 weekly downloads. As such, jsonreststores popularity was classified as not popular.
We found that jsonreststores demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
Security News
As cyber threats become more autonomous, AI-powered defenses are crucial for businesses to stay ahead of attackers who can exploit software vulnerabilities at scale.
Security News
UnitedHealth Group disclosed that the ransomware attack on Change Healthcare compromised protected health information for millions in the U.S., with estimated costs to the company expected to reach $1 billion.