ph-municipalities - npm Package Compare versions

Comparing version 1.0.9 to 1.0.10

package.json

 {
   "name": "ph-municipalities",
-  "version": "1.0.9",
+  "version": "1.0.10",
   "description": "List and write the `municipalities` of Philippines provinces or regions into JSON files",
@@ -5,0 +5,0 @@ "main": "index.js",

README.md

 ## ph-municipalities
 
-**ph-municipalities** have **npm scripts** that allow interactive querying of Philippines municipalities included in one or more provinces or from a whole region, with an option of writing them to JSON files from the command line.
+**ph-municipalities** have **NPM scripts** that allow interactive querying of Philippines municipalities included in one or more provinces or from a whole region, with an option of writing them to JSON files from the command line.
 
 It uses `/data/day1.xlsx` (downloaded and stored as of this 20220808) from PAGASA's [10-day weather forecast excel files](https://www.pagasa.dost.gov.ph/climate/climate-prediction/10-day-climate-forecast) as the default data source.
 
-It also asks users to key in the download URL of a remote excel file should they want to use another excel file for a new and updated data source.
+It also asks users to key in the download URL of a remote PAGASA 10-Day weather forecast excel file should they want to use another excel file for a new and updated data source.
 
@@ -84,4 +84,5 @@ Extracted municipalities are written in JSON files following the format:
    | EXCEL_FILE_URL    | (Optional) Remote excel file's download URL.<br>If provided, the excel file will be downloaded and saved on the specified `pathToFile` local filesystem location during the `ExcelFile` class initialization.<br>Read on [Usage](#usage) for more information. |
-   | SHEETJS_COLUMN    | Column name read by [sheetjs](https://sheetjs.com/) in an excel file.<br>This column contains the municipality and province names following the string pattern<br>`"municipalityName (provinceName)"`<br>Default value is `__EMPTY`                            |
-   | SORT_ALPHABETICAL | Arranges the municipality names in alphabetical order.<br>Default value is `1`. Set to `0` to use the ordering as read from the Excel file.                                                                                                                    |
+   | SHEETJS_COLUMN    | Column name read by [sheetjs](https://sheetjs.com/) in an excel file.<br>This column contains the municipality and province names following the string pattern<br>`"municipalityName (provinceName)"`<br>Default value is `__EMPTY`|
+   | SORT_ALPHABETICAL | Arranges the municipality names in alphabetical order.<br>Default value is `1`. Set to `0` to use the ordering as read from the Excel file. |
+   | SPECIAL_CHARACTERS | Key-value pairs of special characters or garbled text and their normalized text conversions, delimited by the `":"` character.<br>Multiple key-value pairs are delimited by the `","` character.<br>If a special character key's value is a an empty string, write it as i.e.,: `"some-garbled-text:"` |
 
@@ -88,0 +89,0 @@ ## Available Scripts

src/classes/excel/index.js

@@ -192,2 +192,48 @@ const https = require('https')
   /**
+   * Checks if a string contains special characters
+   * @param {String} str - String to check
+   * @returns {Bool}
+   */
+  static hasSpecialChars (str) {
+    /* eslint-disable no-control-regex */
+    const regex = /[^\x00-\x7F]/g
+    return regex.test(str)
+  }
+
+  /**
+   * Cleans/removes default-known special characters and garbled text defined in config from string.
+   * @param {String} str - String to clean
+   * @returns {String} - Clean string
+   */
+  static removeGarbledText (str) {
+    // Known garbled special text
+    let charMap = {
+      '├â┬▒': 'ñ', // Replace "├â┬▒" with "ñ"
+      â: '' // Remove "â"
+    }
+
+    // Other special characters from config
+    const specialChars = (process.env.SPECIAL_CHARACTERS?.split(',') ?? [])
+      .reduce((list, item) => {
+        const [key, value] = item.split(':')
+
+        return {
+          ...list,
+          ...((key || value) && { [key]: value ?? '' })
+        }
+      }, {})
+
+    charMap = {
+      ...charMap,
+      ...specialChars
+    }
+
+    for (const [key, value] of Object.entries(charMap)) {
+      str = str.replace(new RegExp(key, 'g'), value)
+    }
+
+    return str
+  }
+
+  /**
    * Extract the municipality name from a string following the pattern:
@@ -271,4 +317,8 @@ *    "municipalityName (provinceName)"
 
-        acc[item.province].push(item.municipality)
+        const cleanText = ExcelFile.hasSpecialChars(item.municipality)
+          ? ExcelFile.removeGarbledText(item.municipality)
+          : item.municipality
 
+        acc[item.province].push(cleanText)
+
         // Sort municipality names alphabetically
@@ -275,0 +325,0 @@ if (process.env.SORT_ALPHABETICAL === '1') {

New alerts

Environment variable access

Supply chain risk

Package accesses environment variables, which may be a sign of credential stuffing or data theft.

Found 4 instances in 1 package

Fixed alerts

Environment variable access

Supply chain risk

Package accesses environment variables, which may be a sign of credential stuffing or data theft.

Found 1 instance in 1 package

Improved metrics

Total package byte prevSize

664903

increased by0.23%

Lines of code

703

increased by6.35%

Number of lines in readme file

296

increased by0.34%

Number of low supply chain risk alerts

8

decreased by-20%

No dependency changes