🚧pyllments🚧 [Construction In Progress]
Speedrun your Prototyping
uv pip install pyllments
More info on installation
Documentation
Build Visual AI Workflows with Python
A batteries-included LLM workbench for helping you integrate complex LLM flows into your projects - built fully in and for Python.
- Rapid prototyping complex workflows
- GUI/Frontend generation(With Panel components)
- Simple API deployment(Using FastAPI + Optional serving with Uvicorn)
Pyllments is a Python library that empowers you to build rich, interactive applications powered by large language models. It provides a collection of modular Elements, each encapsulating its data, logic, and UI—that communicate through well-defined input and output ports. With simple flow-based programming, you can effortlessly compose, customize, and extend components to create everything from dynamic GUIs to scalable API-driven services.
It comes prepackaged with a set of parameterized application you can run immediately from the command line like so:
pyllments recipe run branch_chat --height 900 --width 700
See Recipes Here
Elements:
- Easily integrate into your own projects
- Have front end components associated with them, which allows you to build your own composable GUIs to interact with your flows
- Can individually or in a flow be served as an API (with limitless endpoints at any given part of the flow)
Chat App Example
With history, a custom system prompt, and an interface.

https://github.com/user-attachments/assets/1f9998bf-2dfe-49c6-aec7-0a918ee57246
In this example, we'll build a simple chat application using four core Elements from the Pyllments library. Elements are modular components that handle specific functions and connect via ports to form a data flow graph.
-
ChatInterfaceElement
Manages the chat GUI, including the input box for typing messages, a send button, and a feed to display the conversation. It emits user messages and renders incoming responses.
-
HistoryHandlerElement
Maintains a running log of all messages (user and assistant) and tool responses. It can emit the current message history to be used as context for the LLM.
-
ContextBuilderElement
Combines multiple inputs into a structured list of messages for the LLM. This includes a predefined system prompt (e.g., setting the AI's personality), the conversation history, and the latest user query.
-
LLMChatElement
Connects to a Large Language Model (LLM) provider, sends the prepared message list, and generates a response. It also offers a selector to choose different models or providers.
Here's what happens step by step in the chat flow:
- User Input: You type a message in the
ChatInterfaceElement
and click send. This creates a MessagePayload
with your text and the role 'user'.
- History Update: Your message is sent via a port to the
HistoryHandlerElement
, which adds it to the conversation log.
- Context Building: The
ContextBuilderElement
receives your message and constructs the full context. It combines:
- A fixed system prompt (e.g., "You are Marvin, a Paranoid Android."),
- The message history from
HistoryHandlerElement
(if available, up to a token limit),
- Your latest query.
- LLM Request: This combined list is sent through a port to the
LLMChatElement
, which forwards it to the selected LLM (like OpenAI's GPT-4o-mini).
- Response Handling: The LLM's reply is packaged as a new
MessagePayload
with the role 'assistant' and sent back to the ChatInterfaceElement
to be displayed in the chat feed.
- History Update (Again): The assistant's response is also sent to the
HistoryHandlerElement
, updating the log for the next interaction.
- Cycle Repeats: The process loops for each new user message, building context anew each time.
chat_flow.py
from pyllments import flow
from pyllments.elements import ChatInterfaceElement, LLMChatElement, HistoryHandlerElement
chat_interface_el = ChatInterfaceElement()
llm_chat_el = LLMChatElement()
history_handler_el = HistoryHandlerElement(history_token_limit=3000)
context_builder_el = ContextBuilderElement(
input_map={
'system_prompt_constant': {
'role': "system",
'message': "You are Marvin, a Paranoid Android."
},
'history': {'ports': [history_handler_el.ports.message_history_output]},
'query': {'ports': [chat_interface_el.ports.user_message_output]},
},
emit_order=['system_prompt_constant', '[history]', 'query']
)
chat_interface_el.ports.user_message_output > history_handler_el.ports.message_input
context_builder_el.ports.messages_output > llm_chat_el.ports.messages_input
llm_chat_el.ports.message_output > chat_interface_el.ports.message_emit_input
chat_interface_el.ports.assistant_message_output > history_handler_el.ports.message_emit_input
@flow
def interface():
width = 950
height = 800
interface_view = chat_interface_el.create_interface_view(width=int(width*.75))
model_selector_view = llm_chat_el.create_model_selector_view(
models=config.custom_models,
model=config.default_model
)
history_view = history_handler_el.create_context_view(width=int(width*.25))
main_view = pn.Row(
pn.Column(
model_selector_view,
pn.Spacer(height=10),
interface_view,
),
pn.Spacer(width=10),
history_view,
styles={'width': 'fit-content'},
height=height
)
return main_view
CLI Command
pyllments serve chat_flow.py
For more in-depth material, check our Getting Started Tutorial
Elements are building blocks with a consistent interface
https://github.com/user-attachments/assets/771c3063-f7a9-4322-a7f4-0d47c7f50ffa
Elements can create and manage easily composable front-end components called Views
https://github.com/user-attachments/assets/8346a397-0308-4aa1-ac9b-5045b701c754
Using their Ports
interface, Elements can be connected in endless permutations.
https://github.com/user-attachments/assets/1693a11a-ccdb-477b-868a-e93c0de3f52a
Attach API endpoints to any part of the flow
https://github.com/user-attachments/assets/13126317-859c-4eb9-9b11-4a24bc6d0f07
RAG Application Example
https://gist.github.com/DmitriyLeybel/8e8aa3cd10809f15a05b6b91451722af
