hdx_python_scraper-2.5.4

.config

coveragerc

pre-commit-config.yaml

pytest.ini

ruff.toml

.github

workflows

publish.yaml

run-python-tests.yaml

.gitignore

CONTRIBUTING.md

LICENSE

PKG-INFO

README.md

documentation

.readthedocs.yaml

main.md

pydoc-markdown.yaml

pyproject.toml

requirements.txt

scraper

framework

__init__.py

_version.py

base_scraper.py

outputs

base.py

excelfile.py

googlesheets.py

json.py

runner.py

scrapers

aggregator.py

configurable_scraper.py

resource_downloader.py

rowparser.py

timeseries.py

utilities

fallbacks.py

reader.py

region_lookup.py

sector.py

sector_configuration.yaml

sources.py

writer.py

tests

config

project_configuration.yaml

fixtures

input

access_2pacx-1vrszjzuyvt9i-mkrq2hbxrul2lx2vihkthqm-lae8nyhqty70zqtcufs3pxbhzgat1l2bkoa4-daoap-pub-gid-574237756-single-true-output-csv.csv

additional-json.json

altworldindata_data-tagger-match-all-on-tagger-01-header-location-tagger-01-tag.csv

casualties_2pacx-1vqidedbzz0ehrc0b4fswip14r7mdtu1mpmwakuxupelsah2awcurkgalfduhjvyjul8vzzat3r1b5qg-pub-gid-0-single-true-output-csv.csv

cbpf-allocations-and-contributions.json

cbpf2-allocations-and-contributions.json

cerf-covid-19-allocations.json

cerf2-covid-19-allocations.json

cerf2_global_download-full-pfmb-allocations.csv

cerf_global_download-full-pfmb-allocations.csv

covax_2pacx-1vtvzu79pptfaa2syevoqfyrrjy63djwitqu0ffbxiqczoun9k9timwmrvfgg1rbsnlmgyugzseiaye2-pub-gid-992438980-single-true-output-csv.csv

covidtests_data-owid-covid-data.xlsx

education_closures_broken.xls

education_closures_school_closures.csv

education_enrolment_enrollment_data.xlsx

ethiopia-drought-related-key-figures.json

ethiopia-pin-targeted-reached-by-location-and-cluster.json

ethiopia_drought_affected_targeted_reached_by_cluster.csv

fallbacks.json

gam_download-unicef-who-wb-global-expanded-databases-severe-wasting.xlsx

gam_other_download-unicef-who-wb-global-expanded-databases-severe-wasting.xlsx

global-coordination-groups-beta.json

global-school-closures-covid19.json

hno_2017_sahel_nutrition.csv

hno_2017_sahel_people_in_need.xlsx

idmc-internally-displaced-persons-idps.json

idps_download-displacement-data.csv

idps_override_population-widget-id-264111-geo-id-693-population-group-54074999.json

idps_somalia_som_unhcr_prmn_displacement_dataset.xlsx

ipc_somalia_som_food_insecurity_oct_dec2022_projection.csv

kenya-drought-related-key-figures.json

kenya-pin-targeted-reached-by-location-and-cluster.json

kenya_drought_affected_targeted_reached_by_cluster.csv

key_figures_2pacx-1vrppqx8jtkkkrckmzfncmmtfecvcpkbp9pdhs1sqtuyacmbsx8tlaxpgblfce-lcehukregguxja-4s-pub-gid-0-single-true-output-csv.csv

key_figures_2pacx-1vrppqx8jtkkkrckmzfncmmtfecvcpkbp9pdhs1sqtuyacmbsx8tlaxpgblfce-lcehukregguxja-4s-pub-gid-1275038715-single-true-output-csv.csv

key_figures_2pacx-1vrppqx8jtkkkrckmzfncmmtfecvcpkbp9pdhs1sqtuyacmbsx8tlaxpgblfce-lcehukregguxja-4s-pub-gid-2015311116-single-true-output-csv.csv

key_figures_eth_2pacx-1vrppqx8jtkkkrckmzfncmmtfecvcpkbp9pdhs1sqtuyacmbsx8tlaxpgblfce-lcehukregguxja-4s-pub-gid-2015311116-single-true-output-csv.csv

key_figures_ken_2pacx-1vrppqx8jtkkkrckmzfncmmtfecvcpkbp9pdhs1sqtuyacmbsx8tlaxpgblfce-lcehukregguxja-4s-pub-gid-1275038715-single-true-output-csv.csv

key_figures_som_2pacx-1vrppqx8jtkkkrckmzfncmmtfecvcpkbp9pdhs1sqtuyacmbsx8tlaxpgblfce-lcehukregguxja-4s-pub-gid-0-single-true-output-csv.csv

ourworldindata_data-tagger-match-all-on-tagger-01-header-location-tagger-01-tag.csv

ourworldindata_other_data-tagger-match-all-on-tagger-01-header-location-tagger-01-tag.csv

oxcgrt_oxcgrt_csv.csv

oxford-covid-19-government-response-tracker.json

population.json

HDX Python scraper utilities to assemble data from multiple sources. 

See how Anthropic automated 95% of dependency reviews with Socket

hdx-python-scraper

HDX Python scraper utilities to assemble data from multiple sources

mcarans

(Experimental) An ambiguous license classifier was found.

What is an ambiguous license classifier?

Ambiguous License Classifier

Source files are encoded using a non-standard text encoding.

What is bad text encoding?

Bad text encoding

Package version is not a valid semantic version (semver).

What is bad semver?

Bad semver

Package has dependencies with an invalid semantic version. This could be a sign of beta, low quality, or unmaintained dependencies.

What is bad dependency semver?

Bad dependency semver

Source files contain bidirectional unicode control characters. This could indicate a Trojan source supply chain attack. See: trojansource.codes for more information.

What are bidirectional unicode control characters?

Bidirectional unicode control characters

This package has multiple bin scripts with the same name. This can cause non-deterministic behavior when installing or could be a sign of a supply chain attack.

What is bin script confusion?

Bin script confusion

Semantic versions published out of chronological order.

What is a chronological version anomaly?

Chronological version anomaly

Project maintainer's SSH key has been compromised.

What is a compromised SSH key?

Compromised SSH key

(Experimental) Copyleft license information was found.

What do I need to know about license files?

Copyleft License

Contains a Critical Common Vulnerability and Exposure (CVE).

What is a critical CVE?

Title

Critical CVE

Contains a high severity Common Vulnerability and Exposure (CVE).

What is a CVE?

High CVE

Uses debug, reflection and dynamic code execution features.

What is debug access?

Debug access

The maintainer of the package marked it as deprecated. This could indicate that a single version should not be used, or that the package is no longer maintained and any new vulnerabilities will not be fixed.

What is a deprecated package?

Deprecated

(Experimental) Contains a known deprecated SPDX license exception.

What is a deprecated SPDX exception?

Deprecated SPDX exception

(Experimental) License is deprecated which may have legal implications regarding the package's use.

What is a deprecated license?

Deprecated license

Package name is similar to other popular packages and may not be the package you want.

What is a typosquat?

Possible typosquat attack

Dynamic require can indicate the package is performing dangerous or unsafe dynamic code execution.

What is dynamic require?

Dynamic require

Package does not contain any code. It may be removed, is name squatting, or the result of a faulty package publish.

What is an empty package?

Empty package

Package accesses environment variables, which may be a sign of credential stuffing or data theft.

What is environment variable access?

Environment variable access

(Experimental) Something was found which is explicitly marked as unlicensed.

Explicitly Unlicensed Item

Package optionally loads a dependency which is not specified within any of the package.json dependency fields. It may inadvertently be importing dependencies specified by other packages.

What are extraneous dependencies?

Name

Extraneous dependency

Contains a dependency which resolves to a file. This can obfuscate analysis and serves no useful purpose.

What are file dependencies?

File dependency

Accesses the file system, and could potentially read sensitive data.

What is filesystem access?

Filesystem access

Package has a dependency with a floating version range. This can cause issues if the dependency publishes a new major version.

What are wildcard dependencies?

Wildcard dependency

Contains a dependency which resolves to a remote git URL. Dependencies fetched from git URLs are not immutable and can be used to inject untrusted code or reduce the likelihood of a reproducible install.

What are git dependencies?

Git dependency

Contains a dependency which resolves to a GitHub URL. Dependencies fetched from GitHub specifiers are not immutable can be used to inject untrusted code or reduce the likelihood of a reproducible install.

What are GitHub dependencies?

GitHub dependency

AI has identified unusual behaviors that may pose a security risk.

What is an AI-detected potential code anomaly?

AI-detected potential code anomaly

AI has identified this package as a potential typosquat of a more popular package. This suggests that the package may be intentionally mimicking another package's name, description, or other metadata.

What is AI-detected potential typosquatting?

AI-detected possible typosquat

AI has identified this package as malware. This is a strong signal that the package may be malicious.

What is AI-detected potential malware?

AI-detected potential malware

AI has determined that this package may contain potential security issues or vulnerabilities.

What are AI-detected potential security risks?

AI-detected potential security risk

Contains native code (e.g., compiled binaries or shared libraries). Including native code can obscure malicious behavior.

Why is native code a concern?

Native code

Contains high entropy strings. This could be a sign of encrypted data, leaked secrets or obfuscated code.

What are high entropy strings?

High entropy strings

Contains unicode homoglyphs which can be used in supply chain confusion attacks.

What are unicode homoglyphs?

Unicode homoglyphs

Contains a dependency which resolves to a remote HTTP URL which could be used to inject untrusted code and reduce overall package reliability.

What are http dependencies?

HTTP dependency

hdx-python-scraper - pypi Package File explorer