dataproofer-core-suite
Advanced tools
Core suite of tests for Dataproofer. These tests relate to common problems and data checks — namely, making sure data has not been truncated by looking for specific cut-off indicators.
Calculates the percentage of rows that are empty for each column
Parameters
rows Array an array of objects representing rows in the spreadsheetcolumnHeads Array an array of strings for column names of the spreadsheetReturns Object describing the result
Calculates the percentage of rows that contain special, non-typical Latin characters for each column Source: http://www.w3schools.com/charsets/ref_html_utf8.asp
Parameters
rows Array an array of objects representing rows in the spreadsheetcolumnHeads Array an array of strings for column names of the spreadsheetReturns Object describing the result
src/stringsHaveExactly255Characters.js:14-66
Determine the cells that have exactly 255 characters (SQL upper limit error). See ProPublica's bad data guide for further information https://github.com/propublica/guides/blob/master/data-bulletproofing.md#integrity-checks-for-every-data-set
Parameters
rows Array an array of objects representing rows in the spreadsheetcolumnHeads Array an array of strings for column names of the spreadsheetReturns Object describing the result
Indicates an bigint at its upper signed limit (MySQL or PostgreSQL) of 9,223,372,036,854,775,807 or its upper unsigned limit (MySQL) of 18,446,744,073,709,551,616.
Common database programs, like MySQL, have a cap on how big of a number it can save.
Please see the MySQL documentation or PostgreSQL documentation for more information.
Parameters
rows Array an array of objects representing rows in the spreadsheetcolumnHeads Array an array of strings for column names of the spreadsheetReturns Object describing the result
Indicates a integer at its upper signed limit is 2,147,483,647 (MySQL or PostgreSQL) or its upper unsigned limit (MySQL) of 4,294,967,295. Common database programs, like MySQL, have a cap on how big of a number it can save. Please see the MySQL documentation for more information.
Parameters
rows Array an array of objects representing rows in the spreadsheetcolumnHeads Array an array of strings for column names of the spreadsheetReturns Object describing the result
Indicates an smallint at its upper signed limit (MySQL or PostgreSQL) of 32,767 or its upper unsigned limit (MySQL) of 65,535.
Common database programs, like MySQL, have a cap on how big of a number it can save.
Please see the MySQL documentation or PostgreSQL documentation for more information.
Parameters
rows Array an array of objects representing rows in the spreadsheetcolumnHeads Array an array of strings for column names of the spreadsheetReturns Object describing the result
Indicates a summed integers at its upper limit of 2,097,152. Please see the Integrity Checks section of the ProPublica Data Bulletproofing Guide for more information.
Parameters
rows Array an array of objects representing rows in the spreadsheetcolumnHeads Array an array of strings for column names of the spreadsheetReturns Object describing the result
src/checkDuplicateRows.js:13-73
Check for any duplicate rows in the spreadsheet. Optionally
Parameters
rows Array an array of objects representing rows in the spreadsheetcolumnHeads Array an array of strings for column names of the spreadsheetinput Object accept user input, such as selected ColumnsReturns Object describing the result
src/numberOfRowsIs65k.js:12-31
Test to see if number of rows is exactly 65,536 rows (cutoff by Excel)
Parameters
rows Array an array of objects representing rows in the spreadsheetcolumnHeads Array an array of strings for column names of the spreadsheetReturns Object describing the result
git clone git@github.com:dataproofer/core-suite.git
cd core-suite
npm install
We use documentation.js, but have created a handy script for regenerating documentation.
npm run docs
Then open up and check your docs in DOCUMENTATION.md
FAQs
Core suite of tests for dataproofer
The npm package dataproofer-core-suite receives a total of 3 weekly downloads. As such, dataproofer-core-suite popularity was classified as not popular.
We found that dataproofer-core-suite demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 3 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Socket CEO Feross Aboukhadijeh discusses open source security challenges, including zero-day attacks and supply chain risks, on the Cyber Security Council podcast.
Security News
Research
Socket researchers uncover how threat actors weaponize Out-of-Band Application Security Testing (OAST) techniques across the npm, PyPI, and RubyGems ecosystems to exfiltrate sensitive data.
Security News
Research
A malicious npm campaign is targeting Ethereum developers by impersonating Hardhat plugins and the Nomic Foundation, stealing sensitive data like private keys.