New Report Warns of LLM-Enhanced Cyber Threats: Polymorphic Malware, Customer Service Jailbreaking, and Highly Personalized Spearphishing

Recent advancements in Large Language Models (LLMs) are making it possible to automate tasks that were once considered too complex, including those with malicious intent. In a collaborative technology exploration project, the Netherlands Organization for Applied Scientific Research (TNO) and the National Cyber Security Centre (NCSC-NL) examined how LLMs could influence the cyber threat landscape over the next three to five years.

The report focuses on the current and near-future capabilities of LLMs and their potential to enhance cyber threats. While it does speculate on some future possibilities, it is grounded in the present realities of what LLMs can already do or are likely to do in the near term.

Part of the discovery process was to determine whether LLMs create evolutionary threats (building on existing threats) or revolutionary threats (creating fundamentally new threats).

The exploration identified three evolutionary changes to the threat landscape researchers anticipate will be performed more efficiently by LLMs or at a larger scale:

• LLMs used for larger scale impersonation attacks that can be personalized more,
• LLMs used by moderate experts as a buddy/co-pilot that suggest attack paths,
• LLMs used by companies publicly could become vulnerable to LLM-specific attacks.

Researchers also identified three attack techniques that would be the first signs of a revolutionary change in how threat actors use LLMs:

• LLMs used by non-experts for producing malicious/exploit code,
• LLMs used by any actor for autonomous exploitation, or
• LLMS used for automated identification of vulnerabilities in software.

This report is based on a survey of recent research but goes further by employing the MITRE ATT&CK framework to systematically map out how LLMs could be used across different phases of cyber attacks, from reconnaissance to impact, identifying both confirmed uses and potential blind spots.

Monitoring Offensive Advantages of LLMs in the Future Threat Landscape#

The NCSC and TNO used the Signposts of Change model, a framework they adapted with a set of indicators that signal potential changes and developments in cyber attack tactics and techniques due to the integration of LLMs.

One example includes the ease of creating polymorphic malware, which frequently mutates its features to evade detection from traditional security solutions. LLMs would enable the generation of varied and sophisticated code that would allow malware to alter its appearance, posing a significant challenge to those defending against the dynamic nature of this threat.

Researchers expect to see non-experts generating malicious code and polymorphic malware with the help of LLMs, and indicators would include:

• An explosive increase in bad exploit code
• Academic papers proving the concept
• Non-experts claim a lot more vulnerability disclosure bounties with high CVSS scores

The exploratory report highlights several examples of how the Signposts of Change model would be used to track LLM-enhanced threats.

Another threat possibility they are monitoring is jailbreaking customer service LLMs, which refers to the exploitation of LLMs that are connected to internal company databases. Adversaries can manipulate these models to bypass their intended functionalities and extract sensitive information or perform unauthorized actions. This could involve tricking the LLM into providing confidential data, executing unintended commands, or otherwise compromising the security and integrity of the system.

Highly targeted spearphishing is one usage of LLMs for cyber attacks that has been found in current scientific literature. There are ample ways LLMs can enhance offensive security tasks by speeding up the process of collecting information specific to the targeted organization. The report included samples of the kinds of attacks they anticipate:

An example attack can also use job openings to disrupt the work of human resources employees. Since LLMs are capable of incorporating information relevant to a process of creative thinking, they can be used to automate the preparation of fake motivation letters or CVs, possibly including malicious content in these documents (including emails). Considering multi-modal models and generative capabilities of some of the models used for image creation, such content can be enriched with forged images or other content which makes the document more credible. In such a setting, the work to filter out counterfeit profiles might be too cumbersome for the team responsible for hiring process. A decision that a forgery profile is credible could open path to other forms of malicious activities.

The TNO and NCSC-NL report offers a glimpse at potential scenarios where the cyber security industry may be challenged by the transformative impact of LLMs. These advancements hold great promise for legitimate applications but the industry should expect their potential for misuse in cyber attacks.

Spearphishing has the potential to become much more precise, and malware may be democratized to the extent that anyone can create exploit code by leveraging LLMs. As organizations integrate LLMs into their operations, the risk of these tools being hijacked for malicious purposes, such as jailbreaking customer service bots, becomes a pressing concern. This lower barrier to entry for cybercriminal activities is one of the revolutionary changes outlined in this report that could fundamentally alter the cybersecurity landscape in the next three to five years.