AlpacaEval : An Automatic Evaluator of Instruction-following Models. 

alpaca-eval

AlpacaEval : An Automatic Evaluator of Instruction-following Models

lxuechen

rtaori

YannDubs

Inherit

Block

Ignore

Monitor

Warn

License

Maintenance

Miscellaneous

Other

Quality

Supply chain risk

Vulnerability

We may hide one of these components if their impact is neutral, e.g. unknown reachability. When alerts are associated with multiple artifacts, the priority score is based on the artifact with the highest priority score.

Legal

Operational

Security

Admins only

Organization members

Dashboard

Grouping: Directs

Grouping: None

Artifact: <code>{{- fullArtifactDisplayName}}</code>

Dependency tree

Summary with created, updated, and cleared alerts

{{feature}} is not available on your current plan. Upgrade to {{tier}} to access this feature

Unlock powerful security features for your {{tier}}

Upgrade to {{tier}}

<upgradeLink>Upgrade</upgradeLink> to access this feature

Upgrade to access this feature

Rate your experience

Archive file: {{fileName}}

Achievements

Why Socket?

Blog

Customer Stories

Languages

Impersonation Mode Active

Update your browser to the latest version for the best experience.

Your web browser is outdated

Use another browser for the best experience.

Your web browser is unsupported

{{count, number}} commits last two months

{{count, number}} critical severity alert

{{count, number}} critical severity alerts

{{count, number}} dependency vulnerability

{{count, number}} dependency vulnerabilities

{{count, number}} development dependencies

{{count, number}} transitive dependencies

{{count, number}} version published last month

{{count, number}} versions published last month

{{count, number}} version published in last two months

{{count, number}} versions published in last two months

{{count, number}} version published last week

{{count, number}} versions published last week

{{count, number}} version published last year

{{count, number}} versions published last year

Socket undergoes annual SOC2 Type II audits. Socket customers can request a copy of the full SOC2 Type II report.

Work with a dedicated Customer Success Manager to help you get the most out of Socket.

Restrict logins to the organization to SSO, preventing other login methods (e.g. GitHub or email)

Have members who are not admins in the organization. Restrict access with Role-based Access Control (RBAC).

Integrates with top AI coding assistants — including Claude Code, Claude Desktop, Gemini Code, and other MCP-compatible agents — to provide real-time dependency insights, vulnerability context, and security scoring directly inside your AI-powered coding environment.

Get a clear view of your current supply chain health. Track active alerts, dependency risk, and security trends across your org—all in one place on the Socket Dashboard and via API.

Allows organization owners and admins to access detailed records of actions taken by members of their organization, complete with timestamps.

Automatically generate fixes for vulnerable dependencies with safe, tested upgrades.

Prevent compromised packages from infiltrating your supply chain by monitoring for changes in real-time. Block 70+ types of open source threats, including malware, typo-squatting, and backdoors.

Shift left by catching risky or low-quality dependencies before they're merged. Use CI gating, PR comments, and visual warnings to guide developers early and enforce security guardrails.

Removes known vulnerabilities instantly without upgrading dependencies. Delivers safe, human-reviewed, AI-validated patches that live in your repo, eliminate risky updates, and keep builds stable while reducing supply chain exposure.

Socket opens pull requests for each patch so developers can review and merge fixes with no extra work.

Receive updated patches as new CVEs and package releases appear, with no extra work.

Remove a vulnerability instantly by clicking “Patch” in the dashboard or CLI. This applies a secure, prebuilt patch to remove a vulnerability without changing dependency versions.

Focus on vulnerabilities your code can actually hit, then fix them instantly with patches.

Use patches vetted by experienced maintainers so you can remediate confidently. Patches ship with automated behavioral tests to ensure nothing breaks.

Integrate with Slack. Microsoft Teams is coming soon.

Integrate with Vanta. Drata is coming soon.

Scans browser extensions for risk and malware — supports Chrome, Firefox, Edge, and Safari.

Analyzes IDE extensions for security risks — supports OpenVSX and VS Code.

Stops known malware and risky dependencies before they’re downloaded — protecting developer machines and CI/CD pipelines in real time.

Monitor blocked installs, package risks, and usage across all teams directly in your Socket dashboard or via the Socket API.

Configure Socket Firewall to run with private registries or as a chained HTTP proxy for full control of dependency ingress.

Customize whether to block, warn, or allow packages based on your security and license policies.

Works with JavaScript/TypeScript (npm, Yarn, pnpm), Python (pip, uv), and Rust (cargo).

Deploy Socket Firewall in your own infrastructure, with flexible modes for wrapper-only or full network proxy operation.

Filters out CVEs in functions your code doesn’t call — reducing up to 90% of false positives and irrelevant CVEs. Tier 1 reachability provides the highest quality reachability data. Easy to setup. Need to add a CLI command to CI/CD or GitHub Action to the repository.

Customize parts of the comment sent to developers in GitHub pull requests

See how your security changes over time with 30 days of historical insights. Track alert trends, monitor dependency changes, and export data via API for custom dashboards.

Surfaces security issues directly in IDEs to help developers fix faster while coding. VS Code extension includes a bundled MCP server by default.

Surfaces issues and fixes in the IDE, helping developers catch and resolve problems as they code.

Socket scans for vulnerabilities in your dependencies, including CVEs, security advisories, and more.

Socket supports JavaScript, TypeScript, Python, Go, Java, Ruby, .NET, Scala, Kotlin, Rust, and more — with continuous expansion to cover the most popular ecosystems.

Automatically enforce license policies across all repositories, blocking non-compliant dependencies and ensuring your entire organization stays compliant.

Local AI connector that exposes dependency scoring and security context to your tools via a standardized protocol.

Define and enforce custom license policies using repo labels — based on factors like team, criticality, environment, or any criteria you choose.

Define and enforce custom security policies using repo labels — based on factors like team, criticality, environment, or any criteria you choose.

Ranks issues by exploitability, reachability, fixability, and custom priorities set via repo labels.

Faster replies to support questions via email and Slack.

Get dedicated support from technical experts and engineers in a private Slack channel.

Use Socket Web Extension to educate developers at the earliest possible point in time about package risks before they install them on their local development systems. Supports Chrome, Edge, and Firefox.

Import or export Software Bills of Materials (SBOMs) to track and audit dependencies.

Scans AI models on PyPI and HuggingFace for malicious code, backdoors, and other security risks.

Scans GitHub Actions for malicious code, backdoors, and other security risks — protecting your CI/CD workflows from supply-chain attacks.

Automate user provisioning and deprovisioning with SCIM.

Use Socket with a self-hosted source code management (GitHub Enterprise Server, GitLab Enterprise, Bitbucket Server, and Azure DevOps Server..

Inegrate with Elastic, Google BigQuery, Panther, and Splunk.

Socket is SOC2 Type II compliant and undergoes annual audits.

Connects to public registries to scan container images from Docker Hub, ECR, ACR, GCR, and more.

Applies to all scans — whether triggered via API, CLI, IDE, or SCM integrations.

Block commits and pushes with secrets using git hooks—before they ever hit your repo.

Detect a wide range of secrets with over 800 built-in detectors covering API keys, credentials, tokens, and more—right out of the box.

The Socket CLI allows teams to prevent risky dependencies from being installed, and to get dependency insights from the command line.

Connects to your source code management system to scan repos, review PRs or MRs, and enrich the developer workflow with security insights.

Socket supports SAML SSO for single sign-on and identity management.

Paid add-on. Provides real-time access to curated vulnerability and malware threat data for use in your own tools and workflows.

Integrate with Jira. Linear is coming soon.

Analyze the reachability of vulnerabilities in your dependencies.

Get up to speed quickly with hands-on training and guided onboarding. Our team helps your developers and security teams hit the ground running.

Receive a Service Level Agreement for uptime of the Socket application.

High-quality comprehensive vulnerability database which integrates proprietary data from Socket, NVD, GitHub, OSV, OpenSSF, and many other sources.

Receive updates via Webhooks anytime Socket has new scan results or alert state has changed.

Create custom tabs in the Alerts page to filter and organize your alerts.

Socket will analyze your SBOMs up to a maximum number of dependencies.

API that allows you to analyze a project, lookup package scores and alerts, and more.

Create API tokens to authenticate with the Socket API and CLI.

A developer is anyone who’s made a commit to repositories scanned by Socket, excluding external contributors to public repos.

A member is anyone added to your Socket organization dashboard with a role of "member" or higher, giving them access to team settings, security policy, org-wide alerts, and integrations.

Repositories restricted to your organization. Socket scans them securely using the SCM integration, or through the CLI or API.

Repositories that are publicly visible. Socket scans them securely using the SCM integration, or through the CLI or API.

Organize repos by team, project, or importance. Apply custom policies and adjust alert priority based on labels.

Access the threat feed that powers Socket.

For growing teams with enhanced scale, security, and support.

Unlimited scale, self-hosting, and priority support for large teams.

For open-source projects, individuals, and small teams.

This is a legacy plan that is no longer available for new customers. Upgrade to our Team or Enterprise plans for enhanced features and support.

There is a package with a similar name that is downloaded much more often.

There are packages with similar names that are downloaded much more often.

This may not be the package you want!

Instance

Alert Locations

Package

Readme not found for selected version

Provide a detailed description of the threat...

Socket fights vulnerabilities and provides visibility, defense-in-depth, and proactive supply chain protection for JavaScript, Python, and Go dependencies.

Socket - Secure your dependencies. Ship with confidence.

Upgrade to access this feature and <pricingLink>many more features</pricingLink>.

This feature is not available on your current plan

Welcome to Socket

{{count, number}} package

{{count, number}} packages

{{- packageName}} - {{ecosystem}} Package Dependencies

{{packageDescription}}Version: {{versionString}} was published by {{publisherName}}. Start using Socket to analyze {{- packageName}} and its dependencies to secure your app from supply chain attacks.

We could not scan this package. Some page functionalities have been disabled

Source code not available

Sorry, it seems this package was removed from the registry

Package was removed

This package version has been unpublished, mostly likely due to security reasons

alpaca-eval - pypi Package Dependencies