Security News
PyPI’s New Archival Feature Closes a Major Security Gap
PyPI now allows maintainers to archive projects, improving security and helping users make informed decisions about their dependencies.
ph-municipalities
Advanced tools
List and write the `municipalities` of Philippines provinces or regions into JSON files
The ph-municipalities NPM package provides NPM scripts that allow interactive querying of Philippine municipalities included in one or more provinces or from a whole region, with an option of writing them to JSON files from the command line through a CLI-like application and classes for only parsing and listing municipality names and provinces from Excel file data sources.
It uses a PAGASA 10-day weather forecast Excel file in /app/data/day1.xlsx
(downloaded and stored as of this 20220808) from PAGASA's 10-Day Climate Forecast web page as the default data source.
Since the PAGASA 10-day weather forecast Excel files do not have region names, the parser scripts sync it with the PAGASA Seasonal weather forecast regions and province names defined in the manually-encoded
/app/config/regions.json
file to determine the region names. The classes also provide an option to specify a new region name configuration file on initialization.
It also asks users to key in the file download URL of a remote PAGASA 10-Day weather forecast Excel file should they want to use another Excel file for a new and updated data source.
INFO: When installing the package using
npm i ph-municipalities
, the default data source is inside/data/day1.xlsx
. All source codes and files are also inside the ph-municipalities root directory.
Extracted municipalities are written in JSON files following the format:
{
"metadata": {
"source": "https://pubfiles.pagasa.dost.gov.ph/pagasaweb/files/climate/tendayweatheroutlook/day1.xlsx",
"title": "List of PH Municipalities By Province and Region",
"description": "This dataset generated with reference to the excel file contents from the source URL on 20220808.",
"date_created": "Mon Aug 08 2022"
},
"data": {
"Albay": ["Bacacay", "Camalig", ... ],
"Camarines Norte": ["Basud", "Capalonga", ... ],
"Camarines Sur": ["Baao", "Balatan", ... ],
...
}
}
Pre-compiled windows binaries are available for download in the latest Releases download page.
We welcome contributions! Please see CONTRIBUTING.md for guidelines.
The following dependencies are used for this project. Feel free to use other dependency versions as needed.
/app/data
directory as data source."municipalityName (provinceName)"
"Project Areas"
plus two (2) blank rows before rows containing municipality and province names to enable strict testing and validation of the number of parsed data rows/app/data/day1.xlsx
sample file for more informationEXCEL_FILE_URL
variable on the Installation section.
ph-municipalities aims to provide a simple, organized, and flexible interface for viewing, querying, and listing Philippine provinces and municipalities using the PAGASA 10-day weather forecast Excel files as the data source. It aims to enable easy extension of Philippine regions/provinces/municipalities listing for parsing the PAGASA 10-day weather forecast data.
Its early stages were written as procedural functions within a private backend project for extracting 10-day weather forecast data from the PAGASA 10-day weather forecast Excel files. When the private project started gaining complexity, a need to separate the logic and management for listing the Philippine province and municipalities per region rose. Creating an independent, public OpenSource version listing the provinces and municipalities per region was decided after experiencing drawbacks and difficulties testing using similar OpenSource libraries (some of which are listed below) for that project.
ph-municipalities aim to contribute to the OpenSource community by listing ONLY Philippine provinces and municipalities' names, using PAGASA's 10-day weather forecast Excel files, which are publicly accessible to everyone.
ph-municipalities is entirely public and open-source, and it has no linkage to private and proprietary source codes and resources.
While ph-municipalities use a PAGASA 10-day weather forecast Excel file as a data source for its provinces and municipalities list, NO, it cannot parse and extract PAGASA 10-day weather forecast data.
ph-municipalities only have class methods for parsing, extracting, listing and querying provinces and municipalities names data from a PAGASA 10-day weather forecast Excel file. Refer to the ExcelFile class documentation for more information about its available methods.
Yes, several OpenSource libraries and projects similar to ph-municipalities exist, which you can use in its place to list Philippine provinces and municipalities.
Here is a list of several of these libraries and code repositories. Note, however, that these items use old and new data sources. These may not be for you if you require using provinces and municipalities' names data from the PAGASA 10-day weather forecast Excel files.
While ph-municipalites do not support parsing and extracting PAGASA 10-day weather forecast data, you can extend the ExcelFile
or ExcelFactory
classes with custom logic and codes to enable parsing and extracting PAGASA 10-day weather forecast data.
Since the ExcelFile
or ExcelFactory
are JavaScript Classes (Functions in disguise, not true OOP, but inheritance still works), you can extend them with class inheritance, overriding or creating new class methods to accommodate processing the PAGASA 10-day weather forecast data. Refer to the ph-municipalities class documentation to know more about the available classes, member variables, and methods.
An example of extending the classes to parse PAGASA 10-day weather forecast data may go along the lines of:
const { ExcelFile } = require('ph-municipalities')
class PAGASATendayParser extends ExcelFile {
/* Override constructor if neccessary
constructor (params) {
super(params)
// Custom class initialization logic here
}
*/
getWeatherData () {
// Note: this.#data contains the "raw" original Excel rows data as JSON with weather data
const weatherData = this.#data.reduce((list, row, index) => {
// Write logic to parse and extract weather data here
}, [])
return weatherData
}
}
const parser = new PAGASATendayParser()
weatherForecast = parser.getWeatherData()
The PAGASA 10-day Excel files only contain province and municipality names linked with weather data - it has no region information. Therefore, ph-municipalities link the province names in the 10-day Excel files to region/province names defined in the /app/config/regions.json
file to determine which region they belong to.
/app/config/regions.json
fileThis file contains region/province names mapping encoded manually with reference from the region and province names listed in the PAGASA Seasonal Forecast website's Forecast Analysis Rainfall table.
ph-municipalities expect similar region naming conventions from the PAGASA Seasonal Forecast and the PAGASA 10-Day Forecast since both are data products produced by PAGASA.
NOTE:
The region/province mapping defined in this file may become outdated as time passes. ph-municipalities users are encouraged to Use a Custom Configuration File, defining new region/province name mappings following the file's current format if they notice region-to-province inconsistencies in the generated municipality lists.
NO. By default, ph-municipalities use an outdated PAGASA 10-day Excel file for its local data source, downloaded on August 8, 2022. However, it also provides several ways for using updated PAGASA 10-day Excel files as data sources:
NOTE:
Overall, the provinces and municipality list rely on the latest contents of a PAGASA 10-day Excel file and manual configuration of region/province names mapping in the/app/config/regions.json
file (See Class Usage - Using a Custom Configuration File). It is not yet known and tested if these are in sync with the latest regions, provinces, and municipalities in the more standard and canon Philippine Standard Geographic Code (PSGC) data.
npm start
/ npm run list:region
npm run list:province
npm run build:win:region
npm run build:win:province
npm run build:win:all
npm run minify:region
npm run minify:province
npm run minify:all
npm run generate-docs
npm run docs:install
npm run docs:build
npm run lint
npm run lint:fix
npm test
npm run example
Clone this repository.
git clone https://github.com/ciatph/municipalities-by-province.git
Install dependencies.
cd app
npm install
Create a .env
file from the .env.example
file inside the /app
directory. Use the default values for the following environment variables.
INFO: If installed as an NPM package with
npm i ph-municipalities
, create the.env
file inside the NPM project's root directory.
Variable Name | Description |
---|---|
EXCEL_FILE_URL | (Optional) Remote excel file's download URL. If provided, the excel file will be downloaded and saved on the specified pathToFile local filesystem location during the ExcelFile class initialization.Read on Usage for more information. |
SHEETJS_COLUMN | Column name read by sheetjs in an excel file. This column contains the municipality and province names following the string pattern "municipalityName (provinceName)" Default value is __EMPTY |
SORT_ALPHABETICAL | Arranges the municipality names in alphabetical order. Default value is 1 . Set to 0 to use the ordering as read from the Excel file. |
SPECIAL_CHARACTERS | Key-value pairs of special characters or garbled text and their normalized text conversions, delimited by the ":" character.Multiple key-value pairs are delimited by the "," character.If a special character key's value is a an empty string, write it as i.e.,: "some-garbled-text:" |
IMAGE_URL | Raw URL of the README image file from this GitHub repository.NOTE: Only add this variable in the GitHub Secrets for publishing to the NPM registry since NPM does not allow displaying images by relative path. |
We can use Docker to run dockerized Node app for local development mode. The following methods require Docker and Docker compose correctly installed and set up on your development machine.
Set up the environment variables for the /app
directory. Visit the Installation section, step #3 for more information.
Stop and delete all docker instances for a fresh start.
NOTE: Running this script will delete all docker images, containers, volumes, and networks. Run this script if you feel like everything is piling but do not proceed if you have important work on other running Docker containers.
sudo chmod u+x scripts/docker-cleanup.sh
./scripts/docker-cleanup.sh
# Answer all proceeding prompts
Build and run the app using Docker.
docker compose -f docker-compose.dev.yml build
docker compose -f docker-compose.dev.yml up
docker compose -f docker-compose.dev.yml down
Edit and execute scripts within the running docker container from step #3.
docker exec -it ph-municipalities <NPM_SCRIPT>
docker exec -it ph-municipalities node ./src/new.js
Note: These NPM scripts run relative within the
/app
directory, when working on a git-cloned repository of the app. To run using only NodeJS, navigate first to the/app
directory and execute a target script, for example:
cd app
npm run list:region
npm start
/ npm run list:region
/app/data/day1.xlsx
by default./app/data/datasource.xlsx
if download URL in the class constructor is provided.npm i ph-municipalities
:
node .\node_modules\ph-municipalities\src\scripts\by_region.js
npm run list:province
/app/data/day1.xlsx
by default./app/data/datasource.xlsx
if download URL in the class constructor is provided.npm i ph-municipalities
:
node .\node_modules\ph-municipalities\src\scripts\by_province.js
npm run build:win:region
npm start
script into a stand-alone windows node16-win-x64
executable./dist/ph-regions-win.exe
. Click the executable file to run.npm run build:win:province
npm list:province
script into a stand-alone windows node16-win-x64
executable./dist/ph-provinces-win.exe
. Click the executable file to run.npm run build:win:all
npm start
and npm list:province
script into a stand-alone windows node16-win-x64
executables in one go./dist
directory.npm run minify:region
npm list:region
script and dependencies into a single script using ncc./dist/region
. Run the command to use the compiled script:node dist/region
npm run minify:province
npm list:province
script and dependencies into a single script using ncc./dist/province
. Run the command to use the compiled script:node dist/province
npm run minify:all
npm list:region
and npm list:province
scripts in one go./dist
directory.npm run generate-docs
Builds the class documentation into the /docs directory.
NOTE:
This script requires manual installation of thejsdoc@4.0.3
,minami@1.2.3
, andtaffydb@2.7.3
packages as devDependencies inside the /app directory. These libraries, only used for building the class documentation, were excluded from the final package.json to have fewer external dependencies.npm install --save-dev jsdoc@4.0.3 minami@1.2.3 taffydb@2.7.3
Installing these libraries will update the
package.json
andpackage-lock.json
files. Take care not to push changes caused by installation.
npm run docs:install
Runs the Bash script that installs the JSDoc and theme dependencies for building the class documentation only within the development Docker environment.
NOTE:
This script requires running from a Bash terminal - it won't work from a Windows command line terminal. It is reserved for building the documentation with Docker.
This script is used for building the class documentation from a local Docker environment along with the npm run docs:build
NPM script.
docker exec -u root -it ph-municipalities npm run docs:install
docker exec -u root -it ph-municipalities npm run docs:build
npm run docs:build
Runs the Bash script that builds the class documentation using JSDoc only within the development Docker environment.
NOTE:
This script requires running from a Bash terminal - it won't work from a Windows command line terminal. It is reserved for building the documentation with Docker.
This script is used for building the class documentation from a local Docker environment along with the npm run docs:install
NPM script.
docker exec -u root -it ph-municipalities npm run docs:install
docker exec -u root -it ph-municipalities npm run docs:build
npm run lint
Lint JavaScript source codes.
npm run lint:fix
Fix JavaScript lint errors.
npm test
Run tests defined in the /app/__tests__
directory.
npm run example
await
Below are example usages of the ExcelFile
class, run from the /app/src/examples directory. Check out the /app/src/examples/sample_usage.js
file for more examples.
This is a simple usage example of the ExcelFile
class.
const path = require('path')
const ExcelFile = require('../classes/excel')
// Use the the following if installed via npm
// const { ExcelFile } = require('ph-municipalities')
// Reads an existing excel file on /app/data/day1.xlsx
file = new ExcelFile({
pathToFile: path.join(__dirname, '..', '..', 'data', 'day1.xlsx'),
// fastload: false
})
// Call init() if fastload=false
// file.init()
// listMunicipalities() lists all municipalities
// for each province
const provinces = ['Albay','Masbate','Sorsogon']
const municipalitiesFromProvince = file.listMunicipalities({ provinces })
// writeMunicipalities() writes municipalities data to a JSON file
// and returns the JSON object
const json = file.writeMunicipalities({
provinces,
fileName: path.join(__dirname, 'municipalities.json'),
prettify: true
})
// shapeJsonData() returns the output of writeMunicipalities()
// without writing to a JSON file
const json2 = file.shapeJsonData(provinces)
// JSON data of the parsed excel file will is accessible on
// file.datalist
console.log(file.datalist)
// Set the contents of file.datalist
file.datalist = [
{ municipality: 'Tayum', province: 'Abra' },
{ municipality: 'Bucay', province: 'Abra' }]
const path = require('path')
const ExcelFile = require('../classes/excel')
// Use the the following if installed via npm
// const { ExcelFile } = require('ph-municipalities')
// Reads an existing excel file on /app/data/day1.xlsx
const file = new ExcelFile({
pathToFile: path.join(__dirname, '..', '..', 'data', 'day1.xlsx'),
fastload: true
})
// listRegions() lists all regions from the regions settings file
const regions = file.listRegions()
console.log('---regions', regions)
// List region data from query
const regionQuery = file.listRegions('region_name')
console.log(`---region full name`, regionQuery)
// listProvinces() lists all provinces of a region
// for each given province
const provinces = file.listProvinces('Region IX')
console.log(`---provinces of ${regions[0]}`, provinces)
const municipalitiesFromProvince = file.listMunicipalities({ provinces })
console.log(`---municipalities`, municipalitiesFromProvince)
Adding a url
field in the constructor parameter prepares the class to download a remote Excel file for the data source.
INFO: Run the
.init()
method after initializing a class with aurl
parameter to start the async file download.
require('dotenv').config()
const path = require('path')
const ExcelFile = require('../classes/excel')
// Use the the following if installed via npm
// const { ExcelFile } = require('ph-municipalities')
const main = async () => {
// Excel file will be downloaded to /app/src/examples/excelfile.xlsx
const file = new ExcelFile({
pathToFile: path.join(__dirname, 'excelfile.xlsx'),
url: process.env.EXCEL_FILE_URL
})
try {
// Download file
await file.init()
console.log(file.datalist)
} catch (err) {
console.log(err.message)
}
}
main()
require('dotenv').config()
const path = require('path')
const ExcelFile = require('../classes/excel')
// Use the the following if installed via npm
// const { ExcelFile } = require('ph-municipalities')
const main = () => {
try {
// Initialize an ExcelFile class instance.
const PHExcel = new ExcelFile({
pathToFile: path.join(__dirname, 'excelfile.xlsx'),
url: process.env.EXCEL_FILE_URL
})
// Download file
PHExcel.init()
// Listen to the instance's EVENTS.LOADED event.
PHExcel.events.on(PHExcel.EVENTS.LOADED, () => {
console.log('--Excel data loaded!', PHExcel.datalist)
})
} catch (err) {
console.log(`[ERROR]: ${err.message}`)
}
}
main()
The ph-municipalities ExcelFile
and ExcelFactory
classes use a default configuration file to define their regions and provinces in the /app/config/regions.json
file. The data in this file syncs with the PAGASA Seasonal and 10-Day Weather Forecast Excel files provinces and municipalities naming convention, encoded by hand as of August 24, 2024.
Follow the codes to define a custom regions config file, following the format of the /app/config/regions.json
file to customize region definitions.
Note: The custom config file's province/municipality names should match those in the 10-day Excel file.
This method is available from ph-municipalities version 1.3.3 and higher.
{
"metadata": {
"sources": [
"http://localhost:3000"
],
"title": "List of Random Provinces by Regions",
"description": "Sample regions config file",
"date_created": "20240824"
},
"data": [
{
"name": "Mondstat",
"abbrev": "MON",
"region_num": "1",
"region_name": "The City of Freedom",
"provinces": ["Brightcrown Mountains","Galesong Hill","Starfell Valley","Windwail Highland","Dragonspine"]
},
{
"name": "Inazuma",
"abbrev": "INZ",
"region_num": "3",
"region_name": "The Realm of Eternity",
"provinces": ["Narukami","Kannazuka","Yashiori","Watatsumi","Seirai","Tsurumi","Enkanomiya"]
},
...
]
}
require('dotenv').config()
const path = require('path')
const ExcelFile = require('../classes/excel')
// Use the the following if installed via npm
// const { ExcelFile } = require('ph-municipalities')
const config = require('./config.json')
// Load the local Excel file
const file = new ExcelFile({
pathToFile: path.join(__dirname, '..', '..', 'data', 'day1.xlsx'),
fastload: true,
// Use a custom settings file to define region info
settings: config
})
// Load provinces from the custom config file
const provinces = file.listProvinces('Inazuma')
// List the municipalities of defined provinces in the config file
// Note: Province/municipality names should match with those in the 10-day Excel file
const municipalities = file.listMunicipalities({ provinces })
console.log('---provinces', provinces)
console.log('\nProvince/municipality names should match with those in the 10-day Excel file')
console.log('---municipalities', municipalities)
npm run build:win:region
npm run build:win:province
# npm run build:win:all
/dist
directory to execute.
npm run minify:region
npm run minify:province
# npm run minify:all
/dist
directory to execute.
node dist/region
node dist/province
The class documentation uses JSDoc annotations where applicable in the JavaScript source codes inside the /src directory. There are two (2) options for building the class documentation.
Run docker for localhost development. Refer to the Docker for Localhost Development section for more information.
Install the dependencies for JSDoc. Suceeding builds will not need to install dependencies after an initial installation. Refer to the npm run docs:install script usage for more information.
docker exec -u root -it ph-municipalities npm run docs:install
Build the documentation. Refer to the npm run docs:build script usage for more information.
docker exec -u root -it ph-municipalities npm run docs:build
Install the dependencies for JSDoc. Refer to the npm run generate-docs
Script usage for more information.
Copy Bash scripts to the /app directory.
/scripts/docs-install.sh
and /scripts/docs-build.sh
Bash scripts to the /scripts directory created from the previous step.Copy static assets to the /app directory.
/docs/diagrams
directory inside the /app directory.README.md
file to the /app directory.Add appropriate user permission to the files.
chmod u+x scripts/docs-install.sh
chmod u+x scripts/docs-build.sh
Run the commands for building the documentation.
INFO: Use a GitBash terminal if you are working on a Windows OS machine.
./scripts/docs-install.sh
./scripts/docs-build.sh
This section describes several common errors and related fixes.
EACCESS: permission denied, open '/opt/app/region1.json'
"app,"
defined in the Dockerfile.docker run --rm -it ciatph/ph-municipalities:dev sh -c "id -u app"
./app
directory using the ph-municipalities user host UID from step #1.sudo chown -R <APP_UID>:<APP_UID> ./app
@ciatph
20220807
FAQs
List and write the `municipalities` of Philippines provinces or regions into JSON files
The npm package ph-municipalities receives a total of 3,038 weekly downloads. As such, ph-municipalities popularity was classified as popular.
We found that ph-municipalities demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 0 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
PyPI now allows maintainers to archive projects, improving security and helping users make informed decisions about their dependencies.
Research
Security News
Malicious npm package postcss-optimizer delivers BeaverTail malware, targeting developer systems; similarities to past campaigns suggest a North Korean connection.
Security News
CISA's KEV data is now on GitHub, offering easier access, API integration, commit history tracking, and automated updates for security teams and researchers.