appthreat-vulnerability-db
Advanced tools
AppThreat's vulnerability database and package search library with a built-in sqlite based storage. OSV, CVE, GitHub, npm are the primary sources of vulnerabilities.
This repo is a vulnerability database and package search for sources such as AppThreat vuln-list, OSV, NVD, and GitHub. Vulnerability data are downloaded from the sources and stored in a sqlite based storage with indexes to allow offline access and efficient searches.
A good vulnerability database must have the following properties:
Multiple upstream sources are used by vdb to improve accuracy and reduce false negatives. SQLite database containing data in CVE 5.1 schema format is precompiled and distributed as files via ghcr to simplify download. With automatic purl prefix generation even for git repos, searches on the database can be performed with purl, cpe, or even http git url string. Every row in the database uses an open specification such as CVE 5.0 or Package URL (purl and vers) thus preventing the possibility of vendor lock-in.
1 - We exclude Linux and oss-fuzz feeds by default. Set the environment variable OSV_INCLUDE_FUZZ=true to include them.
2 - Malware feeds are included by default, thus increasing the db size slightly. Set the environment variable OSV_EXCLUDE_MALWARE=true to exclude them.
pip install appthreat-vulnerability-db>=6.2.0
To install vdb with optional dependencies such as oras use the [oras] or [all] dependency group.
pip install appthreat-vulnerability-db[all]
NOTE: VDB v6 is a major rewrite to use SQLite database. Current users of depscan v5 must continue using version 5.8.x
pip install appthreat-vulnerability-db==5.8.0
This package is ideal as a library for managing vulnerabilities. This is used by owasp-dep-scan, a free open-source dependency audit tool. However, there is a limited cli capability available with few features to test this tool directly.
To download a pre-built SQLite database (refreshed every 6 hours) containing all application vulnerabilities (~ 700MB). This step is recommended for all users.
# pip install appthreat-vulnerability-db[all]
vdb --download-image
You can execute this command daily or when a fresh database is required.
To perform containers and OS scans, download the full image (~ 7.5GB) which includes all application and OS vulnerabilities.
vdb --download-full-image
Using ORAS cli might be slightly faster.
export VDB_HOME=$HOME/vdb
oras pull ghcr.io/appthreat/vdbxz:v6 -o $VDB_HOME
tar -xvf *.tar.xz
rm *.tar.xz
Use any sqlite browser or cli tools to load and query the two databases.
data.index.vdb6 - index db with purl prefix and vers
data.vdb6 - Contains source data in CVE 5.1 format stored as a jsonb blob.
Cache application vulnerabilities
vdb --cache
The typical size of this database is over 700 MB.
Cache from just OSV
vdb --cache --only-osv
It is possible to customize the cache behavior by increasing the historic data period to cache by setting the following environment variables.
Cache application and OS vulnerabilities
vdb --cache-os
Note the size of the database with OS vulnerabilities is over 7.5 GB. It is possible to ignore/exlude specific OS distros using environment variables.
Example to ignore almalinux and ubuntu data from getting included, set the below environment variables:
export VDB_IGNORE_ALMALINUX=true
export VDB_IGNORE_UBUNTU=true
Refer to the variable LINUX_DISTRO_VULN_LIST_PATHS in config.py for the full list of distro strings supported.
usage: vdb [-h] [--clean] [--cache] [--cache-os] [--only-osv] [--only-aqua] [--only-ghsa] [--search SEARCH] [--list-malware] [--bom BOM_FILE]
AppThreat's vulnerability database and package search library with a built-in sqlite based storage.
options:
-h, --help show this help message and exit
--clean Clear the vulnerability database cache from platform specific user_data_dir.
--cache Cache vulnerability information in platform specific user_data_dir.
--cache-os Cache OS vulnerability information in platform specific user_data_dir.
--only-osv Use only OSV as the source. Use with --cache.
--only-aqua Use only Aqua vuln-list as the source. Use with --cache.
--only-ghsa Use only recent ghsa as the source. Use with --cache.
--search SEARCH Search for the package or CVE ID in the database. Use purl, cpe, or git http url.
--list-malware List latest malwares with CVE ID beginning with MAL-.
--bom BOM_FILE Search for packages in the CycloneDX BOM file.
--download-image Downloaded pre-created vdb image to platform specific user_data_dir. Application vulnerabilities only.
--download-full-image
Downloaded pre-created vdb image to platform specific user_data_dir. All vulnerabilities including OS.
--print-vdb-metadata Display metadata about the current vdb in user_data_dir.
It is possible to perform a range of searches using the cli.
vdb --search pkg:pypi/xml2dict@0.2.2
# Search based on a purl prefix
vdb --search pkg:pypi/xml2dict
# Full url and short form for swift
vdb --search "pkg:swift/github.com/vapor/vapor@4.39.0"
vdb --search "pkg:swift/vapor/vapor@4.89.0"
# Search by cpe
vdb --search "cpe:2.3:a:npm:gitblame:*:*:*:*:*:*:*:*"
# Search by colon separated values
vdb --search "npm:gitblame:0.0.1"
# Search by vulnerability id (CVE, GHSA, ALSA, DSA, etc.)
vdb --search CVE-2024-25169
# Search with wildcard for CVE
vdb --search CVE-2024-%
# Search by git url
vdb --search "https://github.com/electron/electron"
# Search by CycloneDX SBOM
vdb --bom bom.json
To list malware entries with the MAL- prefix, use the following command.
vdb --list-malware
Refer to cli.py for usage. Below is a sample code snippet.
from vdb.lib import config, db6 as db_lib, search
from vdb.lib.cve_model import CVE
# Perform search
results = search.search_by_any("purl or cpe or url", with_data=True)
# Download prebuilt database
from vdb.lib.orasclient import download_image
download_image(config.VDB_DATABASE_URL, config.DATA_DIR)
# Check if database needs updating
if db_lib.needs_update():
print("Database needs to be updated")
download_image(...)
MIT
FAQs
AppThreat's vulnerability database and package search library with a built-in sqlite based storage. OSV, CVE, GitHub, npm are the primary sources of vulnerabilities.
We found that appthreat-vulnerability-db demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.
Research
Security News
A threat actor's playbook for exploiting the npm ecosystem was exposed on the dark web, detailing how to build a blockchain-powered botnet.
Security News
NVD’s backlog surpasses 20,000 CVEs as analysis slows and NIST announces new system updates to address ongoing delays.