.gitignore

.pre-commit-config.yaml

CITATION.cff

CODE_OF_CONDUCT.md

CONTRIBUTING.md

Dockerfile

LICENSE

MANIFEST.in

README.md

THIRD_PARTY_LICENSES.txt

analytics

analytics.py

sagemaker_cost_rpm_plot.py

sagemaker_metrics_plot.py

scratchpad.ipynb

check_s3_content.sh

copy_s3_content.sh

create_manifest.py

debug.sh

docs

accuracy.md

advanced.md

analytics.md

announcement.md

benchmarking.md

benchmarking_multimodal_models_on_bedrock.md

benchmarking_on_bedrock.md

benchmarking_on_ec2.md

benchmarking_on_eks.md

benchmarking_on_sagemaker.md

build.md

byo_dataset.md

byo_rest_predictor.md

byoe.md

cli.md

customize_config_files.md

deepseek.md

ec2.md

evaluation.md

features.md

gettingstarted.md

ML-FMBT-cloudformation-launch-stack.png

accuracy_trajectory_per_payload.png

business_summary.png

config-structure.png

ec2-iam.png

ec2connect1.png

ec2connect10.png

ec2connect2.png

ec2connect3.png

ec2connect4.png

ec2connect5.png

ec2connect5a.png

ec2connect6.png

ec2connect7.png

ec2connect8.png

ec2connect9.png

ec2tag.png

fmbench-deployment1.png

fmbench-deployment2.png

fmbench-thumbnail.png

fmbench-website.png

fmbench.drawio

fmbt-small.png

fmbt.png

ground_truth_info.png

heatmap.png

hf_ds_keys.png

latency_vs_tokens.png

overall_candidate_model_majority_voting_accuracy.png

results.gif

website.png

index.md

manifest.md

misc

ec2_instance_creation_steps.md

eks_cluster-creation_steps.md

the-diy-version-w-gory-details.md

mm_copies.md

neuron.md

quarto.md

quickstart.md

releases.md

resources.md

results.md

run_as_container.md

simplified_config_files.md

website.md

workflow.md

ec2_template.yaml

fmbench

0_setup.ipynb

1_generate_data.ipynb

2_deploy_model.ipynb

3_run_inference.ipynb

4_get_evaluations.ipynb

5_model_metric_analysis.ipynb

6_cleanup.ipynb

__init__.py

Introducing Organization Notifications in Socket

github.com/aws-samples/foundation-model-benchmarking-tool

foundation-model-benchmarking-tool

An ambiguous license classifier was found.

What is an ambiguous license classifier?

Ambiguous License Classifier

Source files are encoded using a non-standard text encoding.

What is bad text encoding?

Bad text encoding

Package version is not a valid semantic version (semver).

What is bad semver?

Bad semver

Package has dependencies with an invalid semantic version. This could be a sign of beta, low quality, or unmaintained dependencies.

What is bad dependency semver?

Bad dependency semver

Source files contain bidirectional unicode control characters. This could indicate a Trojan source supply chain attack. See: trojansource.codes for more information.

What are bidirectional unicode control characters?

Bidirectional unicode control characters

This package has multiple bin scripts with the same name. This can cause non-deterministic behavior when installing or could be a sign of a supply chain attack.

What is bin script confusion?

Bin script confusion

This Chrome extension includes content scripts that execute JavaScript on specified websites.

What are Chrome extension content scripts?

Chrome: Content Script

This Chrome extension requests host permissions to access specific websites or domains.

What are Chrome extension host permissions?

Chrome: Host Permission

This Chrome extension requests permissions to access browser APIs, user data, or system features.

What are Chrome extension permissions?

Chrome: Permission

This Chrome extension requests wildcard host permissions that grant broad access to websites.

What are Chrome extension wildcard host permissions?

Chrome: Wildcard Host Permission

Semantic versions published out of chronological order.

What is a chronological version anomaly?

Chronological version anomaly

Project maintainer's SSH key has been compromised.

What is a compromised SSH key?

Compromised SSH key

What do I need to know about license files?

Copyleft License

Contains a Critical Common Vulnerability and Exposure (CVE).

What is a critical CVE?

Title

Critical CVE

Contains a high severity Common Vulnerability and Exposure (CVE).

What is a CVE?

High CVE

Uses debug, reflection and dynamic code execution features.

What is debug access?

Debug access

The maintainer of the package marked it as deprecated. This could indicate that a single version should not be used, or that the package is no longer maintained and any new vulnerabilities will not be fixed.

What is a deprecated package?

Deprecated

Contains a known deprecated SPDX license exception.

What is a deprecated SPDX exception?

Deprecated SPDX exception

License is deprecated which may have legal implications regarding the package's use.

What is a deprecated license?

Deprecated license

Package name is similar to other popular packages and may not be the package you want.

What is a typosquat?

Possible typosquat attack

Dynamic require can indicate the package is performing dangerous or unsafe dynamic code execution.

What is dynamic require?

Dynamic require

Package does not contain any code. It may be removed, is name squatting, or the result of a faulty package publish.

What is an empty package?

Empty package

Package accesses environment variables, which may be a sign of credential stuffing or data theft.

What is environment variable access?

Environment variable access

Something was found which is explicitly marked as unlicensed.

Explicitly Unlicensed Item

Package optionally loads a dependency which is not specified within any of the package.json dependency fields. It may inadvertently be importing dependencies specified by other packages.

What are extraneous dependencies?

Name

Extraneous dependency

Contains a dependency which resolves to a file. This can obfuscate analysis and serves no useful purpose.

What are file dependencies?

File dependency

Accesses the file system, and could potentially read sensitive data.

What is filesystem access?

Filesystem access

Package has a dependency with a floating version range. This can cause issues if the dependency publishes a new major version.

What are wildcard dependencies?

Wildcard dependency

Generic ad-hoc alert, uploaded by user or produced by system diagnostics

Alert title

Generic alert

An input argument to this GitHub Action is being exported as an environment variable. If a user of this action passes untrusted input, it could be used in an insecure manner by subsequent workflow steps.

What are GitHub Actions taint flows?

GitHub Actions: Input argument exported as environment variable

An input argument to this GitHub Action is being passed back as an output. If a user of this action passes untrusted input, it could be used in an insecure manner by consuming workflows.

GitHub Actions: Input argument passed back as output

An input argument to this GitHub Action flows into a dangerous sink (such as shell command execution). This could allow a malicious user to inject commands or exploit the action.

GitHub Actions: Input argument flows to dangerous sink

A GitHub context variable (such as issue title, PR description, or comment body) is being exported as an environment variable. These context values are user-controlled and could be exploited by subsequent workflow steps.

GitHub Actions: GitHub context variable exported as environment variable

A GitHub context variable (such as issue title, PR description, or comment body) is being passed back as an output. These context values are user-controlled and could be exploited by consuming workflows.

GitHub Actions: GitHub context variable passed back as output

A GitHub context variable (such as issue title, PR description, or comment body) flows into a dangerous sink (such as shell command execution). This is a critical security issue that could allow command injection or code execution attacks.

GitHub Actions: GitHub context variable flows to dangerous sink

github.com/aws-samples/foundation-model-benchmarking-tool - go Package File explorer