csvgeocode
Advanced tools
Comparing version
{ | ||
"name": "csvgeocode", | ||
"version": "0.0.5", | ||
"version": "0.0.6", | ||
"description": "Bulk geocode addresses in a CSV.", | ||
@@ -10,3 +10,3 @@ "main": "index.js", | ||
"scripts": { | ||
"test": "echo \"Error: no test specified\" && exit 1" | ||
"test": "node test/test.js" | ||
}, | ||
@@ -13,0 +13,0 @@ "repository": { |
302
README.md
csvgeocode | ||
========== | ||
Bulk geocode addresses in a CSV with one line of code (OK, two lines of code). | ||
For when you have a CSV with addresses and you want a lat/lng for every row. Bulk geocode addresses a CSV with one or two lines of code. | ||
The defaults are configured to use Google's geocoder but can be configured to work with any other similar geocoding service. | ||
The defaults are configured to use [Google's geocoder](https://developers.google.com/maps/documentation/geocoding/) but it can be configured to work with any other similar geocoding service. | ||
## Installation | ||
## Basic command line usage | ||
Install via `npm`: | ||
Install globally via npm: | ||
``` | ||
npm install csvgeocode | ||
```` | ||
To use the command line version, install it globally: | ||
``` | ||
npm install -g csvgeocode | ||
``` | ||
## Basic Usage | ||
Use it: | ||
In a script, use it like this: | ||
```js | ||
var csvgeocode = require("csvgeocode"); | ||
//Write a new CSV with lat/lng columns added | ||
csvgeocode("path/to/input.csv","path/to/output.csv"); | ||
``` | ||
From the command line, use it like this: | ||
``` | ||
$ csvgeocode path/to/input.csv path/to/output.csv | ||
``` | ||
## A Little More | ||
If you don't specify an output file, the output will stream to stdout instead, so you can do something like: | ||
### csvgeocode(input[,output][,options]) | ||
You must specify an `input` filename as the first argument. | ||
You can optionally specify an `output` filename to write the results to a new CSV. If you don't specify one, the results will be streamed to `stdout`. | ||
You can specify `options` to override the defaults (see below). | ||
```js | ||
//Write to a file with default options | ||
csvgeocode("input.csv","output.csv"); | ||
//Write to a file with some custom options | ||
csvgeocode("input.csv","output.csv",{ | ||
address: "MY_ADDRESS_COLUMN_NAME" | ||
}); | ||
//Stream to stdout with default options | ||
csvgeocode("input.csv"); | ||
//Stream to stdout with some custom options | ||
csvgeocode("input.csv",{ | ||
delay: 500 //wait 500ms between geocoding calls | ||
}); | ||
``` | ||
You can supply all the same options to the command line version: | ||
$ csvgeocode path/to/input.csv | grep "greppin for somethin" | ||
``` | ||
$ csvgeocode path/to/input.csv path/to/output.csv --delay 500 --address MY_ADDRESS_COLUMN_NAME | ||
``` | ||
## Options | ||
The following options are available: | ||
You can add extra options when running `csvgeocode`. For example: | ||
#### `url` | ||
``` | ||
$ csvgeocode input.csv output.csv --address MY_ADDRESS_COLUMN_HAS_THIS_WEIRD_NAME --delay 1000 --verbose | ||
``` | ||
The URL template to use for geocoding. The placeholder `{{a}}` will be replaced by each individual address. | ||
None of the options are required. | ||
**Default:** `https://maps.googleapis.com/maps/api/geocode/json?address={{a}}` | ||
#### `--address [address column name]` | ||
In a script: | ||
The name of the column that contains the address to geocode. | ||
```js | ||
csvgeocode("input.csv","output.csv",{ | ||
url: "http://myspecialgeocoder.com/?address={{a}}" | ||
}) | ||
``` | ||
**Default:** Tries to automatically detect if there's a relevant column name in the input CSV, like `address` or `street_address`. | ||
From the command line: | ||
#### `--lat [latitude column name]` | ||
``` | ||
$ csvgeocode input.csv output.csv --url "http://myspecialgeocoder.com/?address={{a}}" | ||
``` | ||
The name of the column that should contain the resulting latitude. If this column doesn't exist in the input CSV, it will be created in the output. | ||
#### `address` | ||
**Default:** Tries to automatically detect if there is a relevant column name in the input CSV, like `lat` or `latitude`. If none is found, it will add the new column `lat` to the output. | ||
The name of the column that contains the address to geocode. This must exist in the CSV. | ||
#### `--lng [longitude column name]` | ||
**Default:** Automatically detects if there is a relevant column name like `address` or `street_address`. | ||
The name of the column that should contain the resulting longitude. If this column doesn't exist in the input CSV, it will be created in the output. | ||
In a script: | ||
**Default:** Tries to automatically detect if there is a relevant column name in the input CSV, like `lng` or `longitude`. If none is found, it will add the new column `lng` to the output. | ||
```js | ||
csvgeocode("input.csv","output.csv",{ | ||
address: "MY_ADDRESS_COLUMN_HAS_THIS_DUMB_NAME" | ||
}) | ||
``` | ||
#### `--delay [milliseconds]` | ||
From the command line: | ||
The number of milliseconds to wait between geocoding calls. Setting this to 0 is probably a bad idea because most geocoders limit how fast you can make requests. | ||
``` | ||
$ csvgeocode input.csv output.csv --address MY_ADDRESS_COLUMN_HAS_THIS_DUMB_NAME | ||
``` | ||
**Default:** 250 | ||
#### `lat` | ||
#### `--force` | ||
The name of the column that should contain the resulting latitude. If this column doesn't exist in the input CSV, it will be created in the output. | ||
By default, if a lat/lng is already found in an input row, that will be kept. If you want to re-geocode every row no matter what and replace any lat/lngs that already exist, add `--force`. This means you'll hit API limits faster and the process will take longer. | ||
**Default:** Automatically detects if there is a relevant column name like `lat` or `latitude`. If none exists, uses `lat`. | ||
#### `--verbose` | ||
In a script: | ||
See extra output while csvgeocode is running. | ||
```js | ||
csvgeocode("input.csv","output.csv",{ | ||
lat: "MY_LATITUDE_COLUMN" | ||
}) | ||
``` | ||
$ csvgeocode input.csv output.csv --verbose | ||
160 Varick St, New York NY: SUCCESS | ||
1600 Pennsylvania Ave, Washington, DC: SUCCESS | ||
123 Fictional St: NO MATCH | ||
From the command line: | ||
Rows geocoded: 2 | ||
Rows failed: 1 | ||
Time elapsed: 1.8 seconds | ||
``` | ||
$ csvgeocode input.csv output.csv --lat MY_LATITUDE_COLUMN | ||
``` | ||
#### `lng` | ||
#### `url` | ||
The name of the column that should contain the resulting longitude. If this column doesn't exist in the input CSV, it will be created in the output. | ||
The URL template to use for geocoding. The placeholder `{{a}}` will be replaced by each individual address. You might want to use this to add extra arguments to a Google geocoding request, like bounds. If you want to use a different geocoder entirely, you can do that by using `csvgeocode` as a Node module (see below). | ||
**Default:** Automatically detects if there is a relevant column name like `lng` or `longitude`. If none exists, uses `lng`. | ||
**Default:** `https://maps.googleapis.com/maps/api/geocode/json?address={{a}}` | ||
In a script: | ||
## Using as a Node module | ||
```js | ||
csvgeocode("input.csv","output.csv",{ | ||
lng: "MY_LONGITUDE_COLUMN" | ||
}) | ||
``` | ||
Install via `npm`: | ||
From the command line: | ||
``` | ||
$ csvgeocode input.csv output.csv --lng MY_LONGITUDE_COLUMN | ||
npm install csvgeocode | ||
``` | ||
#### `delay` | ||
Use it: | ||
The number of milliseconds to wait between geocoding calls. Setting this to 0 is probably a bad idea because most geocoders limit how fast you can make requests. | ||
```js | ||
var csvgeocode = require("csvgeocode"); | ||
**Default:** `250` | ||
csvgeocode("path/to/input.csv","path/to/output.csv"); | ||
``` | ||
In a script: | ||
You can add all the same options in a script, except for `verbose`. | ||
```js | ||
csvgeocode("input.csv","output.csv",{ | ||
delay: 1000 | ||
}) | ||
``` | ||
var options = { | ||
"address": "MY_SPECIAL_ADDRESS_COLUMN_NAME", | ||
"lat": "MY_SPECIAL_LATITUDE_COLUMN_NAME", | ||
"lng": "MY_SPECIAL_LONGITUDE_COLUMN_NAME", | ||
"delay": 1000, | ||
"force": true, | ||
"url": "https://maps.googleapis.com/maps/api/geocode/json?bounds=40,-74|41,-72&address={{a}}" | ||
}; | ||
From the command line: | ||
//write to a file | ||
csvgeocode("input.csv","output.csv",options); | ||
//stream to stdout | ||
csvgeocode("input.csv",options); | ||
``` | ||
$ csvgeocode input.csv output.csv --delay 1000 | ||
``` | ||
`csvgeocode` runs asynchronously, but you can listen for two events: `row` and `complete`. | ||
#### `force` | ||
`row` is triggered when each row is processed. It passes a string error message if geocoding the row failed, and the row itself. | ||
Set to `true` if you want to re-geocode every row even if an existing lat/lng is detected. Setting this to true means you'll hit API limits faster. | ||
**Default:** `false` | ||
In a script: | ||
```js | ||
csvgeocode("input.csv","output.csv",{ | ||
force: true | ||
}) | ||
csvgeocode("input.csv","output.csv") | ||
.on("row",function(err,row){ | ||
if (err) { | ||
console.warn(err); | ||
} | ||
/* | ||
`row` is an object like: | ||
{ | ||
first: "John", | ||
last: "Keefe", | ||
address: "160 Varick St, New York NY", | ||
employer: "WNYC", | ||
lat: 40.7267926, | ||
lng: -74.00537369999999 | ||
} | ||
*/ | ||
}); | ||
``` | ||
From the command line: | ||
`complete` is triggered when all geocoding is done. It passes a `summary` object with three properties: `failures`, `successes`, and `time`. | ||
```js | ||
csvgeocoder("input.csv","output.csv") | ||
.on("complete",function(summary){ | ||
/* | ||
`summary` is an object like: | ||
{ | ||
failures: 1, //1 row failed | ||
successes: 49, //49 rows succeeded | ||
time: 8700 //it took 8.7 seconds | ||
} | ||
*/ | ||
}); | ||
``` | ||
$ csvgeocode input.csv output.csv --force | ||
``` | ||
#### `handler` | ||
## Using a non-Google geocoder | ||
Acceptable values are `"google"`, `"mapbox"`, or a custom handler function for a geocoding API response. A custom handler function will get two arguments: the response body and the address being geocoded. It should return an object with `lat` and `lng` properties when successful. Otherwise it should return a string error message, which will be passed to the `failure` event (see below). | ||
You can use a non-Google geocoder from within a Node script by customizing the `url` option and adding a custom function as the `handler` option. | ||
**Default:** "google" | ||
Your handler will be passed two string arguments: the body of the geocoder response and the address being geocoded. | ||
To use Mapbox instead, you should supply `"mapbox"` as the handler and a Mapbox API url with your API key, like: | ||
It should return a string error message if there's no lat/lng to use, or it should return an object with `lat` and `lng` properties. | ||
```js | ||
csvgeocode("input.csv","output.csv",{ | ||
url: "http://api.tiles.mapbox.com/v4/geocode/mapbox.places/{{a}}.json?access_token=MY_API_KEY", | ||
handler: "mapbox" | ||
}); | ||
``` | ||
For example, if you wanted to use [Mapbox's geocoder](https://www.mapbox.com/developers/api/geocoding/) instead, you could run it like this: | ||
Using a custom handler could look something like this: | ||
```js | ||
csvgeocode("input.csv","output.csv",{ | ||
url: "http://myspecialgeocoder.com?q={{a}}", | ||
handler: mySpecialHandler | ||
url: "http://api.tiles.mapbox.com/v4/geocode/mapbox.places/{{a}}.json?access_token=MY_API_KEY", //custom URL template | ||
handler: mapboxHandler //custom handler function | ||
}); | ||
function mySpecialHandler(body,address) { | ||
function mapboxHandler(body,address) { | ||
var parsed = JSON.parse(body); | ||
var result = JSON.parse(body); | ||
//Error, return a string | ||
if (parsed.error) { | ||
return "Some sort of error message."; | ||
} | ||
if (result.features === undefined) { | ||
//No match, return a string | ||
if (parsed.results.length === 0) { | ||
return "No results for: "+address; | ||
return response.message; | ||
//No results, return a string | ||
} else if (!result.features.length) { | ||
return "No match for " + address; | ||
} | ||
//Success, return a lat/lng object | ||
return { | ||
lat: parsed.results[0].lat, | ||
lng: parsed.results[0].lng | ||
lat: result.features[0].center[1], | ||
lng: result.features[0].center[0] | ||
}; | ||
} | ||
``` | ||
## Events | ||
While the geocoder is running, it will emit three events: `success`, `failure` and `complete`. | ||
`success` is emitted whenever a row in the input CSV successfully geocodes. | ||
`failure` is emitted whenever a row in the input CSV fails to geocode. | ||
`complete` is emitted when all rows are done, and includes a summary object with `failures`, `successes`, and `time` properties. | ||
You can listen to any of these events to monitor progress or trigger other events as needed. | ||
```js | ||
csvgeocoder("input.csv","output.csv") | ||
.on("failure",function(error){ | ||
//An address failed, there's an error message | ||
}) | ||
.on("success",function(address){ | ||
//An address was successfully geocoded | ||
}) | ||
.on("complete",function(summary){ | ||
/* | ||
Summary is an object like: | ||
{ | ||
failures: 1, //1 row failed | ||
successes: 49, //49 rows succeeded | ||
time: 8700 //it took 8.7 seconds | ||
} | ||
*/ | ||
}); | ||
``` | ||
## Notes | ||
Geocoding a long list of unsanitized addresses rarely goes perfectly the first time. Using csvgeocode, any addresses that don't succeed will have their lat/lng columns left blank. By listening for the `failure` event (or just browsing the results), you can figure out which ones failed and edit them as needed. Then you can run the result back through and only the failed rows will get re-geocoded. | ||
Geocoding a long list of unsanitized addresses rarely goes perfectly the first time. Using csvgeocode, any addresses that don't succeed will have their lat/lng columns left blank. By listening for the `row` event or browsing the final results, you can figure out which ones failed and edit them as needed. Then you can run the result back through and only the failed rows will get re-geocoded. | ||
@@ -284,3 +210,5 @@ ## To Do | ||
* Support a CSV with no header row where `lat`, `lng`, and `address` are numerical indices instead of column names. | ||
* Allow `address` to be an array of multiple fields that get concatenated (e.g. `["street","city","state","zip"]`) | ||
* Make `bounds` a separate option rather than something you have to hardcore into the URL. | ||
* Support the `handler` option for CLI too? | ||
* Support both POST and GET requests somehow. | ||
@@ -287,0 +215,0 @@ |
@@ -6,7 +6,6 @@ var misc = require("./misc"), | ||
request = require("request"), | ||
parse = require("csv-parse"), | ||
format = require("csv-stringify"), | ||
queue = require("queue-async"), | ||
extend = require("extend"), | ||
util = require("util"), | ||
csv = require("./csv"), | ||
EventEmitter = require("events").EventEmitter; | ||
@@ -18,4 +17,4 @@ | ||
var input = fs.createReadStream(inFile), | ||
output = process.stdout, | ||
var input = inFile, | ||
output = null, | ||
options = {}; | ||
@@ -25,3 +24,3 @@ | ||
if (typeof outFile === "string") { | ||
output = fs.createWriteStream(outFile); | ||
output = outFile; | ||
} else { | ||
@@ -31,8 +30,8 @@ options = outFile; | ||
} else if (arguments.length === 3) { | ||
output = fs.createWriteStream(outFile); | ||
output = outFile; | ||
options = userOptions; | ||
} | ||
//Default options | ||
options = extend(defaults,options); | ||
//Extend default options | ||
options = extend({},defaults,options); | ||
@@ -44,162 +43,198 @@ if (typeof options.handler === "string") { | ||
} else { | ||
throw new Error("Invalid value for 'handler' option. Must be 'google', 'mapbox', or a function."); | ||
throw new Error("Invalid value for 'handler' option. Must be 'google', 'mapbox', or a function."); | ||
} | ||
} else if (typeof options.handler !== "function") { | ||
throw new Error("Invalid value for 'handler' option. Must be 'google', 'mapbox', or a function."); | ||
throw new TypeError("Invalid value for 'handler' option. Must be 'google', 'mapbox', or a function."); | ||
} | ||
var geocoder = new Geocoder(input,output,options); | ||
if (output && typeof output !== "string") { | ||
throw new TypeError("Invalid value for output. Needs to be a string filename."); | ||
} | ||
return geocoder.run(); | ||
var geocoder = new Geocoder(); | ||
return geocoder.run(input,output,options); | ||
}; | ||
var Geocoder = function(input,output,options) { | ||
var Geocoder = function() { | ||
}; | ||
this.input = input; | ||
this.output = output; | ||
util.inherits(Geocoder, EventEmitter); | ||
Geocoder.prototype.run = function(input,output,options) { | ||
var cache = {}, //Cached results by address | ||
_this = this, | ||
time = (new Date()).getTime(); | ||
this.options = options; | ||
//TO DO: allow more concurrency? | ||
this.queue = queue(1); | ||
csv.read(input,csvParsed); | ||
this.cache = {}; //Cached results by address | ||
return this; | ||
}; | ||
function csvParsed(parsed) { | ||
util.inherits(Geocoder, EventEmitter); | ||
var q = queue(1); | ||
Geocoder.prototype.run = function() { | ||
//If there are unset column names, | ||
//try to discover them on the first data row | ||
if (options.lat === null || options.lng === null || options.address === null) { | ||
this.formatter = format({ header: true }); | ||
this.parser = parse({ columns: true }); | ||
this.time = (new Date()).getTime(); | ||
options = misc.discoverOptions(options,parsed[0]); | ||
this.formatter.pipe(this.output); | ||
if (options.address === null) { | ||
throw new Error("Couldn't auto-detect address column."); | ||
} | ||
//extend options with defaults | ||
this.input.pipe(this.parser) | ||
.on("data",function(row){ | ||
} | ||
this.emit("data",row); | ||
parsed.forEach(function(row){ | ||
q.defer(codeRow,row); | ||
}); | ||
//If there are unset column names, | ||
//try to discover them on the first data row | ||
if (this.options.lat === null || this.options.lng === null || this.options.address === null) { | ||
q.awaitAll(complete); | ||
this.options = misc.discoverOptions(this.options,row); | ||
} | ||
if (this.options.address === null) { | ||
throw new Error("Couldn't auto-detect address column."); | ||
} | ||
function codeRow(row,cb) { | ||
} | ||
if (row[options.address] === undefined) { | ||
_this.emit("row","Couldn't find address column '"+options.address+"'",row); | ||
return cb(null,row); | ||
} | ||
this.queue.defer(this.codeRow.bind(this),row); | ||
//Doesn't need geocoding | ||
if (!options.force && misc.isNumeric(row[options.lat]) && misc.isNumeric(row[options.lng])) { | ||
_this.emit("row",null,row); | ||
return cb(null,row); | ||
} | ||
//Address is cached from a previous result | ||
if (cache[row[options.address]]) { | ||
}.bind(this)) | ||
.on("end",function(){ | ||
this.queue.awaitAll(this.complete.bind(this)); | ||
}.bind(this)); | ||
row[options.lat] = cache[row[options.address]].lat; | ||
row[options.lng] = cache[row[options.address]].lng; | ||
return this; | ||
_this.emit("row",null,row); | ||
return cb(null,row); | ||
} | ||
} | ||
Geocoder.prototype.codeRow = function(row,cb) { | ||
request.get(misc.url(options.url,row[options.address]),function(err,response,body) { | ||
//Some other error | ||
if (err) { | ||
var cbgb = function(){ | ||
return cb(null,!!(row.lat && row.lng)); | ||
}; | ||
_this.emit("row",err.toString(),row); | ||
return cb(null,row); | ||
if (row[this.options.address] === undefined) { | ||
this.emit("failure","[ERROR] Couldn't find address column '"+this.options.address+"'"); | ||
return this.formatter.write(row,cbgb); | ||
} | ||
} else if (response.statusCode !== 200) { | ||
//Doesn't need geocoding | ||
if (!this.options.force && misc.isNumeric(row[this.options.lat]) && misc.isNumeric(row[this.options.lng])) { | ||
this.emit("success",row[this.options.address]); | ||
return this.formatter.write(row,cbgb); | ||
} | ||
_this.emit("row","HTTP Status "+response.statusCode,row); | ||
return cb(null,row); | ||
//Address is cached from a previous result | ||
if (this.cache[row[this.options.address]]) { | ||
} else { | ||
row[this.options.lat] = this.cache[row[this.options.address]].lat; | ||
row[this.options.lng] = this.cache[row[this.options.address]].lng; | ||
handleResponse(body,row,cb); | ||
this.emit("success",row[this.options.address]); | ||
return this.formatter.write(row,cbgb); | ||
} | ||
}); | ||
} | ||
request.get(misc.url(this.options.url,row[this.options.address]),function(err,response,body){ | ||
function handleResponse(body,row,cb) { | ||
var result; | ||
//Some other error | ||
if (err) { | ||
try { | ||
result = options.handler(body,row[options.address]); | ||
} catch (e) { | ||
_this.emit("row","Parsing error: "+e.toString(),row); | ||
} | ||
this.emit("failure","[ERROR]"+err.toString()); | ||
//Error code | ||
if (typeof result === "string") { | ||
} else if (response.statusCode !== 200) { | ||
row[options.lat] = ""; | ||
row[options.lng] = ""; | ||
this.emit("failure","[ERROR] HTTP Status "+response.statusCode); | ||
_this.emit("row",result,row); | ||
//Success | ||
} else if ("lat" in result && "lng" in result) { | ||
row[options.lat] = result.lat; | ||
row[options.lng] = result.lng; | ||
//Cache the result | ||
cache[row[options.address]] = result; | ||
_this.emit("row",null,row); | ||
//Unknown extraction error | ||
} else { | ||
try { | ||
result = this.options.handler(body,row[this.options.address]); | ||
} catch(e) { | ||
this.emit("failure","[ERROR] Parsing error: "+e.toString()); | ||
} | ||
_this.emit("row","Invalid return value from handler for response body: "+body,row); | ||
//Error code | ||
if (typeof result === "string") { | ||
} | ||
row[this.options.lat] = ""; | ||
row[this.options.lng] = ""; | ||
return setTimeout(function(){ | ||
cb(null,row); | ||
},options.delay); | ||
this.emit("failure",result); | ||
} | ||
//Success | ||
} else if ("lat" in result && "lng" in result) { | ||
row[this.options.lat] = result.lat; | ||
row[this.options.lng] = result.lng; | ||
function complete(e,results) { | ||
//Cache the result | ||
this.cache[row[this.options.address]] = result; | ||
this.emit("success",row[this.options.address]); | ||
var numSuccesses = results.filter(successful).length, | ||
numFailures = results.length - numSuccesses, | ||
summarize = function(){ | ||
_this.emit("complete",{ | ||
failures: numFailures, | ||
successes: numSuccesses, | ||
time: (new Date()).getTime() - time | ||
}); | ||
}; | ||
//Unknown extraction error | ||
if (!options.test) { | ||
if (typeof output === "string") { | ||
csv.write(output,results,summarize); | ||
} else { | ||
this.emit("failure","[ERROR] Invalid return value from handler for response body: "+body); | ||
output = output || process.stdout; | ||
csv.stringify(results,function(string){ | ||
try { | ||
output.write(string,summarize); | ||
} catch(e) { | ||
throw new TypeError("Second argument output needs to be a filename or a writeable stream."); | ||
} | ||
}); | ||
} | ||
} else { | ||
summarize(); | ||
} | ||
return this.formatter.write(row,function(){ | ||
setTimeout(cbgb,this.options.delay); | ||
}.bind(this)); | ||
} | ||
}.bind(this)); | ||
function successful(row) { | ||
return misc.isNumeric(row[options.lat]) && misc.isNumeric(row[options.lng]); | ||
} | ||
}; | ||
Geocoder.prototype.complete = function(err,results){ | ||
var failures = results.filter(function(d){ | ||
return !d; | ||
}).length; | ||
this.emit("complete",{ | ||
failures: failures, | ||
successes: results.length - failures, | ||
time: (new Date()).getTime() - this.time | ||
}); | ||
} |
@@ -13,7 +13,7 @@ module.exports = { | ||
if (response.status === "ZERO_RESULTS" || response.status === "OK") { | ||
return "[NO MATCH] "+address; | ||
return "NO MATCH"; | ||
} | ||
//Other error, return a string | ||
return "[ERROR] "+response.status; | ||
return response.status; | ||
@@ -26,5 +26,5 @@ }, | ||
if (response.features === undefined) { | ||
return "[ERROR] "+response.message; | ||
return response.message; | ||
} else if (!response.features.length) { | ||
return "[NO MATCH] "+address; | ||
return "NO MATCH"; | ||
} | ||
@@ -31,0 +31,0 @@ |
149
test/test.js
@@ -1,15 +0,136 @@ | ||
var geocode = require("../"); | ||
var assert = require("assert"), | ||
geocode = require("../"), | ||
queue = require("queue-async"); | ||
geocode("test.csv",{ | ||
url: "http://api.tiles.mapbox.com/v4/geocode/mapbox.places/{{a}}.json?access_token=pk.eyJ1IjoiamtlZWZlIiwiYSI6ImVCXzdvUGsifQ.5tFwEhRfLmH36EUxuvUQLA", | ||
handler: "mapbox" | ||
}) | ||
.on("success",function(address){ | ||
console.log("SUCCESS "+address); | ||
}) | ||
.on("failure",function(err){ | ||
console.warn(err); | ||
}) | ||
.on("complete",function(summary){ | ||
console.warn(summary); | ||
}); | ||
queue(1) | ||
.defer(basicTest) | ||
.defer(cacheTest) | ||
.defer(columnNamesTest) | ||
.defer(addColumnsTest) | ||
.defer(handlerTest) | ||
.defer(throwTest) | ||
.awaitAll(function(){}); | ||
function basicTest(cb) { | ||
geocode("basic.csv",{ | ||
test: true | ||
}) | ||
.on("row",function(err,row){ | ||
assert("address" in row && | ||
"lat" in row && "lng" in row,"address, lat, or lng missing from row"); | ||
if (row.address === "160 Varick St, New York NY") { | ||
assert.deepEqual(Math.round(+row.lat),41,"Unexpected lat for "+row.address); | ||
assert.deepEqual(Math.round(+row.lng),-74,"Unexpected lng for "+row.address); | ||
} else if (row.address === "CACHED GIBBERISH ADDRESS") { | ||
assert.deepEqual(+row.lat,99,"Expected cached lat of 99."); | ||
assert.deepEqual(+row.lng,99,"Expected cached lng of 99."); | ||
} | ||
if (row.address === "THIS IS NOT AN ADDRESS") { | ||
assert(err,"NO MATCH","Expected NO MATCH for "+row.address); | ||
} else { | ||
assert.deepEqual(err,null,"Expected null error for "+row.address); | ||
} | ||
}) | ||
.on("complete",function(summary){ | ||
assert.deepEqual(summary.failures,1,"Expected 1 failure"); | ||
assert.deepEqual(summary.successes,6,"Expected 6 successes"); | ||
cb(null); | ||
}); | ||
} | ||
function cacheTest(cb) { | ||
geocode("basic.csv",{ | ||
force: true, | ||
test: true | ||
}) | ||
.on("row",function(err,row){ | ||
if (row.address === "CACHED GIBBERISH ADDRESS") { | ||
assert.deepEqual(row.lat,"","Lat should be empty."); | ||
assert.deepEqual(row.lng,"","Lng should be empty."); | ||
assert.deepEqual(err,"NO MATCH","Expected NO MATCH for "+row.address); | ||
} | ||
}) | ||
.on("complete",function(summary){ | ||
assert.deepEqual(summary.failures,3,"Expected 3 failures"); | ||
assert.deepEqual(summary.successes,4,"Expected 4 successes"); | ||
cb(null); | ||
}); | ||
} | ||
function columnNamesTest(cb) { | ||
geocode("column-names.csv",{ | ||
lat: "LERTITUDE", | ||
lng: "LANGITUDE", | ||
test: true | ||
}) | ||
.on("row",function(err,row){ | ||
assert.deepEqual(row.lat,undefined); | ||
assert.deepEqual(row.lng,undefined); | ||
if (err) { | ||
assert.deepEqual(row.LERTITUDE,"","Expected blank LERTITUDE in column names test."); | ||
assert.deepEqual(row.LANGITUDE,"","Expected blank LANGITUDE in column names test."); | ||
assert.deepEqual(err,"NO MATCH","Expected NO MATCH in column names test."); | ||
} else { | ||
assert(row.LERTITUDE && row.LANGITUDE,"LERTITUDE and LANGITUDE should be set in column names test."); | ||
} | ||
}) | ||
.on("complete",function(summary){ | ||
cb(null); | ||
}); | ||
} | ||
function addColumnsTest(cb) { | ||
geocode("column-names.csv",{ | ||
test: true | ||
}) | ||
.on("row",function(err,row){ | ||
if (err) { | ||
assert.deepEqual(err,"NO MATCH","Expected NO MATCH in add columns test."); | ||
} else { | ||
assert("lat" in row && "lng" in row); | ||
} | ||
}) | ||
.on("complete",function(summary){ | ||
cb(null); | ||
}); | ||
} | ||
function handlerTest(cb) { | ||
geocode("basic.csv",{ | ||
force: true, | ||
handler: function(body,address){ | ||
return "CUSTOM ERROR"; | ||
}, | ||
test: true | ||
}) | ||
.on("row",function(err,row){ | ||
assert.deepEqual(err,"CUSTOM ERROR","Expected CUSTOM ERROR from custom handler."); | ||
}) | ||
.on("complete",function(summary){ | ||
assert.deepEqual(summary.successes,0,"Expected 0 successes"); | ||
cb(null); | ||
}); | ||
} | ||
function throwTest(cb) { | ||
assert.throws( | ||
function(){ | ||
geocode("basic.csv",{ | ||
test: true, | ||
handler: "dumb string" | ||
}); | ||
}, | ||
function(err) { | ||
if (err instanceof Error && /invalid value/i.test(err)) { | ||
return true; | ||
} | ||
}, | ||
"Expected invalid handler message" | ||
); | ||
cb(null); | ||
} |
Sorry, the diff of this file is not supported yet
Major refactor
Supply chain riskPackage has recently undergone a major refactor. It may be unstable or indicate significant internal changes. Use caution when updating to versions that include significant changes.
Found 1 instance in 1 package
No tests
QualityPackage does not have any tests. This is a strong signal of a poorly maintained or low quality package.
Found 1 instance in 1 package
25611
34.03%18
50%410
88.07%1
-50%225
-24.24%2
100%1
Infinity%