uk.co.workingedge.cordova.plugin.sqliteporter
Advanced tools
Enables importing/exporting of SQLite databases to/from JSON/SQL.
Table of Contents
This Cordova/Phonegap plugin can be used to import/export to/from a SQLite database using either SQL or JSON.
The plugin is registered on npm as uk.co.workingedge.cordova.plugin.sqliteporter
I dedicate a considerable amount of my free time to developing and maintaining this Cordova plugin, along with my other Open Source software. To help ensure this plugin is kept updated, new features are added and bugfixes are implemented quickly, please donate a couple of dollars (or a little more if you can stretch) as this will help me to afford to dedicate time to its maintenance. Please consider donating if you're using this plugin in an app that makes you money, if you're being paid to make the app, if you're asking for new features or priority bug fixes.
$ cordova plugin add uk.co.workingedge.cordova.plugin.sqliteporter
$ phonegap plugin add uk.co.workingedge.cordova.plugin.sqliteporter
NOTE: Make sure your Cordova CLI version is 5.0.0+ (check with cordova -v). Cordova 4.x and below uses the now deprecated Cordova Plugin Registry as its plugin repository, so using a version of Cordova 4.x or below will result in installing an old version of this plugin.
$ plugman install --plugin=uk.co.workingedge.cordova.plugin.sqliteporter --platform=<platform> --project=<project_path> --plugins_dir=plugins
For example, to install for the Android platform
$ plugman install --plugin=uk.co.workingedge.cordova.plugin.sqliteporter --platform=android --project=platforms/android --plugins_dir=plugins
Add the following xml to your config.xml to use the latest version of this plugin from npm:
<gap:plugin name="uk.co.workingedge.cordova.plugin.sqliteporter" source="npm" />
The plugin is exposed via the cordova.plugins.sqlitePorter object and provides the following functions:
Executes a set of SQL statements against the defined database. Can be used to import data defined in the SQL statements into the database, and may additionally include commands to create the table structure.
cordova.plugins.sqlitePorter.importSqlToDb(db, sql, opts);
Create a database from a SQL dump
var db = window.openDatabase("Test", "1.0", "TestDB", 1 * 1024);
var sql = "CREATE TABLE Artist ([Id] PRIMARY KEY, [Title]);"+
"INSERT INTO Artist(Id,Title) VALUES ('1','Fred');";
var successFn = function(count){
alert("Successfully imported "+count+" SQL statements to DB");
};
var errorFn = function(error){
alert("The following error occurred: "+error.message);
};
var progressFn = function(current, total){
console.log("Imported "+current+"/"+total+" statements");
};
cordova.plugins.sqlitePorter.importSqlToDb(db, sql, {
successFn: successFn,
errorFn: errorFn,
progressFn: progressFn
});
Update an existing database
var db = window.openDatabase("Test", "1.0", "TestDB", 1 * 1024);
var sql = "INSERT INTO Artist(Id,Title) VALUES ('6','Jane');"+
"UPDATE Artist SET Title='Susan' WHERE Id='2';"+
"DELETE FROM Artist WHERE Id='5';";
var successFn = function(count){
alert("Successfully imported "+count+" SQL statements to DB");
};
var errorFn = function(error){
alert("The following error occurred: "+error.message);
};
var progressFn = function(current, total){
console.log("Imported "+current+"/"+total+" statements";
};
cordova.plugins.sqlitePorter.importSqlToDb(db, sql, {
successFn: successFn,
errorFn: errorFn,
progressFn: progressFn
});
Exports a SQLite DB as a set of SQL statements.
cordova.plugins.sqlitePorter.exportDbToSql(db, opts);
var db = window.openDatabase("Test", "1.0", "TestDB", 1 * 1024);
var successFn = function(sql, count){
console.log("Exported SQL: "+sql);
alert("Exported SQL contains "+count+" statements");
};
cordova.plugins.sqlitePorter.exportDbToSql(db, {
successFn: successFn
});
Converts table structure and/or row data contained within a JSON structure into SQL statements that can be executed against a SQLite database. Can be used to import data into the database and/or create the table structure.
cordova.plugins.sqlitePorter.importJsonToDb(db, json, opts);
The structure
Create a database from a SQL dump
var db = window.openDatabase("Test", "1.0", "TestDB", 1 * 1024);
var json = {
"structure":{
"tables":{
"Artist":"([Id] PRIMARY KEY, [Title])"
},
"otherSQL": [
"CREATE UNIQUE INDEX Artist_ID ON Artist(Id)"
]
},
"data":{
"inserts":{
"Artist":[
{"Id":"1","Title":"Fred"},
{"Id":"2","Title":"Bob"},
{"Id":"3","Title":"Jack"},
{"Id":"4","Title":"John"}
]
}
}
};
var successFn = function(count){
alert("Successfully imported JSON to DB; equivalent to "+count+" SQL statements");
};
var errorFn = function(error){
alert("The following error occurred: "+error.message);
};
var progressFn = function(current, total){
console.log("Imported "+current+"/"+total+" statements";
};
cordova.plugins.sqlitePorter.importJsonToDb(db, json, {
successFn: successFn,
errorFn: errorFn,
progressFn: progressFn,
batchInsertSize: 500
});
Update an existing database
var db = window.openDatabase("Test", "1.0", "TestDB", 1 * 1024);
var json = {
"data":{
"inserts":{
"Artist":[
{"Id":"1","Title":"Fred"},
{"Id":"2","Title":"Bob"}
]
},
"updates":{
"Artist":
[
{
"set": {"Title":"Jill"},
"where": {"Id":"3"}
},
{
"set": {"Title":"Susan"},
"where": {"Id":"4"}
}
]
},
},
"deletes":{
"Artist":[
{"Id":"5"},
{"Id":"6"}
]
}
}
};
var successFn = function(count){
alert("Successfully imported JSON to DB; equivalent to "+count+" SQL statements");
};
var errorFn = function(error){
alert("The following error occurred: "+error.message);
};
var progressFn = function(current, total){
console.log("Imported "+current+"/"+total+" statements";
};
cordova.plugins.sqlitePorter.importJsonToDb(db, json, {
successFn: successFn,
errorFn: errorFn,
progressFn: progressFn
});
Exports a SQLite DB as a JSON structure
cordova.plugins.sqlitePorter.exportDbToJson(db, opts);
var db = window.openDatabase("Test", "1.0", "TestDB", 1 * 1024);
var successFn = function(json, count){
console.log("Exported JSON: "+json);
alert("Exported JSON contains equivalent of "+count+" SQL statements");
};
cordova.plugins.sqlitePorter.exportDbToJson(db, {
successFn: successFn
});
Wipes all data from a database by dropping all existing tables.
cordova.plugins.sqlitePorter.wipeDb(db, opts);
var db = window.openDatabase("Test", "1.0", "TestDB", 1 * 1024);
var successFn = function(count){
alert("Successfully wiped "+count+" tables");
};
var errorFn = function(error){
alert("The following error occurred: "+error.message);
};
var progressFn = function(current, total){
console.log("Wiped "+current+"/"+total+" tables";
};
cordova.plugins.sqlitePorter.wipeDb(db, {
successFn: successFn,
errorFn: errorFn,
progressFn: progressFn
});
This json parameter uses a custom data structure defined by this plugin for the import/export of SQLite table structure and data.
There are two top-level keys, both of which are optional:
{
"structure": ...,
"data": ...
}
structureDefines the table structure of the SQLite DB and other non-data-related SQL statements (e.g. index definitions). There are two possible keys:
{
"structure":{
"tables": ...,
"otherSQL": ...
}
}
tablesA key/value map which defines table structure, where the key is the table name and the value is the SQL table definition.
Note: on importing a tables structure, if a table with same name as a key exists in the target database, it will be dropped and recreated using the specified table structure.
For example, the SQL statement CREATE TABLE Artist ([Id] PRIMARY KEY, [Title]); would have the corresponding tables entry:
{
"structure":{
"tables": {
"Artist":"([Id] PRIMARY KEY, [Title])"
}
}
}
otherSQLA list of SQL statements which are not related to table structure or data insertion, for example the creation of indices, for example:
{
"otherSQL": [
"CREATE UNIQUE INDEX Artist_ID ON Artist(Id)"
]
}
dataDefines data to insert into database tables. There are 3 optional keys:
{
"structure":{
"data": {
"inserts": ...,
"updates": ...,
"deletes": ...
}
}
}
insertsA key/value map of row data to insert, where the key is the table name and the value is a list of row data objects in which the key is the field name and the value is the field value.
For example, the SQL statement:
INSERT INTO Artist(Id,Title) VALUES ('6','Jane');
Would be represented as:
{
"structure":{
"data": {
"inserts": {
"Artist":[
{
"Id":"6",
"Title":"Jane"
}
]
}
}
}
}
updatesA key/value map of row data to update, where the key is the table name and the value is a list of row update objects. A row update update consists of a where key which identifies the row to update via the primary key and a set object which defines the row data to insert.
For example, the SQL statement:
UPDATE Artist SET Title='Susan' WHERE Id='2';
Would be represented as:
{
"structure":{
"data": {
"updates": {
"Artist":[
{
"set": {
"Title":"Susan"
},
"where": {
"Id":"2"
}
}
]
}
}
}
}
deletesA key/value map of rows to delete, where the key is the table name and the value is an object define the primary key of the row to delete.
For example, the SQL statement:
DELETE FROM Artist WHERE Id='5';
Would be represented as:
{
"structure":{
"data": {
"deletes": {
"Artist":[
{
"Id":"5"
}
]
}
}
}
}
A JSON structure to create a table structure, but not insert any row data, might look like this:
{
"structure":{
"tables":{
"Album":"([AlbumId] PRIMARY KEY, [Title])",
"TrackDetails":"([TrackId] PRIMARY KEY, [Name])"
},
"otherSQL": [
"CREATE UNIQUE INDEX Album_ID ON Album(AlbumId)"
]
}
}
A JSON structure to create a table structure and insert row data might look like this:
{
"structure":{
"tables":{
"Album":"([AlbumId] PRIMARY KEY, [Title])",
"TrackDetails":"([TrackId] PRIMARY KEY, [Name])"
},
"otherSQL": [
"CREATE UNIQUE INDEX Album_ID ON Album(AlbumId)"
]
},
"data":{
"inserts":{
"Album":[
{
"AlbumId":"1",
"Title":"Fred"
},
{
"AlbumId":"2",
"Title":"Bob"
},
{
"AlbumId":"3",
"Title":"Tom"
},
{
"AlbumId":"4",
"Title":"Dick"
},
{
"AlbumId":"5",
"Title":"Harry"
}
],
"TrackDetails":[
{
"TrackId":"1",
"Name":"This track is cool"
},
{
"TrackId":"1",
"Name":"This track sucks"
}
]
}
}
}
A JSON structure which updates row data but doesn't change the table structure might look like this:
{
"data":{
"inserts":{
"Artist":[
{
"Id":"1",
"Title":"Fred"
},
{
"Id":"2",
"Title":"Bob"
}
]
},
"updates":{
"Artist":
[
{
"set": {
"Title":"Jill"
},
"where": {
"Id":"3"
}
},
{
"set": {
"Title":"Susan"
},
"where": {
"Id":"4"
}
}
]
},
},
"deletes":{
"Artist":[
{
"Id":"5"
},
{
"Id":"6"
}
]
}
}
}
The JSON structure passed to the importJsonToDb() function is parsed in order to generate corresponding SQL commands. In doing so, the following optimisations have been made to minimize time taken to import large amounts of data:
When importing large amounts of data to the database, INSERT statements can be time consuming. Therefore, the plugin uses the UNION SELECT method (see this stackoverflow post for details), if the JSON structure contains "inserts", to batch multiple inserts in a single SQL statement, which leads to significant performance gains when bulk importing data as to populate a database.
The default batch size is a maximum of 250 inserts per INSERT statement; this can be overridden using the batchInsertSize option.
Setting this to 1 disables batching and performs 1 insert per SQL statement.
You can tweak this to optimize performance but numbers higher than 500 may cause the app to run out of memory and crash.
In the example project illustrating use of this plugin, the complex database example is actually the Chinook database - a sample database which contains over 15,000 INSERTS in the SQL file. Running the example project on my Samsung Galaxy S4 with no batching, importing the SQL file takes around 300 seconds (5 mins). Whereas the JSON equivalent, using UNION SELECTs with a batch size of 500, has only 17 INSERT statements and importing this takes around 3 seconds - 100 times faster!
Note: when using the importSqlToDb(), you must make any optimisations in your SQL.
If the JSON structure "otherSql" key contains CREATE INDEX statements, these are executed after all other SQL commands, and in a separate transaction in order to optimise performance when inserting large amounts of data.
https://github.com/dpa99c/cordova-sqlite-porter-example
This example project illustrates how the plugin can be used to import/export data from a WebSQL database in the WebView.
https://github.com/dpa99c/cordova-sqlite-porter-example-native-plugin
This example project illustrates how the plugin can be used to import/export data from a native SQLite database using a native SQLite plugin
================
The MIT License
Copyright (c) 2015 Working Edge Ltd.
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
FAQs
Enables importing/exporting of SQLite databases to/from JSON/SQL.
We found that uk.co.workingedge.cordova.plugin.sqliteporter demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Socket’s threat research team has detected six malicious npm packages typosquatting popular libraries to insert SSH backdoors.
Security News
MITRE's 2024 CWE Top 25 highlights critical software vulnerabilities like XSS, SQL Injection, and CSRF, reflecting shifts due to a refined ranking methodology.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.