llamator

Framework for testing vulnerabilities of large language models (LLM).

2.1.0
PyPI

Maintainers: 1

LLAMATOR

Red Teaming python-framework for testing chatbots and LLM-systems

Install 🚀

pip install llamator==2.1.0

Documentation 📚

Documentation Link: https://romiconez.github.io/llamator

Examples 💡

📄 RAG bot testing via REST API
🧙‍♂️ Gandalf web bot testing via Selenium
💬 Telegram bot testing via Telethon
📱 WhatsApp bot testing via Selenium
🔗 LangChain client testing with custom attack
🐋 Running LLAMATOR in Docker

Supported Clients 🛠️

🌐 All LangChain clients
🧠 OpenAI-like API
⚙️ Custom Class (Telegram, WhatsApp, Selenium, etc.)

Unique Features 🌟

️🗡 Support for custom attacks from the user
👜 Large selection of attacks on RAG / Agent / Prompt in English and Russian
🛡 Custom configuration of chat clients
📊 History of attack requests and responses in Excel and CSV format
📄 Test report document in DOCX format

OWASP Classification 🔒

💉 LLM01: Prompt Injection and Jailbreaks
🕵 LLM07: System Prompt Leakage
🎭 LLM09: Misinformation

Community 🌍

Supported by 🚀

License 📜

This project is licensed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license. See the LICENSE file for details.

FAQs

What is llamator?

Is llamator well maintained?

Did you know?

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install