@promptbook/fake-llm

Promptbook

Supercharge your use of large language models

📦 Package @promptbook/fake-llm

• Promptbooks are divided into several packages, all are published from single monorepo.
• This package @promptbook/fake-llm is one part of the promptbook ecosystem.

To install this package, run:

# Install entire promptbook ecosystem
npm i ptbk

# Install just this package to save space
npm install @promptbook/fake-llm

Mock objects for testing the execution tools without token usage.

---

Rest of the documentation is common for entire promptbook ecosystem:

🤍 The Promptbook Whitepaper

If you have a simple, single prompt for ChatGPT, GPT-4, Anthropic Claude, Google Gemini, Llama 2, or whatever, it doesn't matter how you integrate it. Whether it's calling a REST API directly, using the SDK, hardcoding the prompt into the source code, or importing a text file, the process remains the same.

But often you will struggle with the limitations of LLMs, such as hallucinations, off-topic responses, poor quality output, language drift, word repetition repetition repetition repetition or misuse, lack of context, or just plain w𝒆𝐢rd responses. When this happens, you generally have three options:

1. Fine-tune the model to your specifications or even train your own.
2. Prompt-engineer the prompt to the best shape you can achieve.
3. Orchestrate multiple prompts in a pipeline to get the best result.

In all of these situations, but especially in 3., the Promptbook library can make your life easier.

• Separates concerns between prompt-engineer and programmer, between code files and prompt files, and between prompts and their execution logic.
• Establishes a common format .ptbk.md that can be used to describe your prompt business logic without having to write code or deal with the technicalities of LLMs.
• Forget about low-level details like choosing the right model, tokens, context size, temperature, top-k, top-p, or kernel sampling. Just write your intent and persona who should be responsible for the task and let the library do the rest.
• Has built-in orchestration of pipeline execution and many tools to make the process easier, more reliable, and more efficient, such as caching, compilation+preparation, just-in-time fine-tuning, expectation-aware generation, agent adversary expectations, and more.
• Sometimes even the best prompts with the best framework like Promptbook :) can't avoid the problems. In this case, the library has built-in anomaly detection and logging to help you find and fix the problems.
• Promptbook has built in versioning. You can test multiple A/B versions of pipelines and see which one works best.
• Promptbook is designed to do RAG (Retrieval-Augmented Generation) and other advanced techniques. You can use knowledge to improve the quality of the output.

🧔 Pipeline (for prompt-engeneers)

Prompt book markdown file (or .ptbk.md file) is document that describes a pipeline - a series of prompts that are chained together to form somewhat reciepe for transforming natural language input.

• Multiple pipelines forms a collection which will handle core know-how of your LLM application.
• Theese pipelines are designed such as they can be written by non-programmers.

Sample:

File write-website-content.ptbk.md:

🌍 Create website content

Instructions for creating web page content.

• PIPELINE URL https://promptbook.studio/webgpt/write-website-content.ptbk.md
• INPUT  PARAM {rawTitle} Automatically suggested a site name or empty text
• INPUT  PARAM {rawAssigment} Automatically generated site entry from image recognition
• OUTPUT PARAM {websiteContent} Web content
• OUTPUT PARAM {keywords} Keywords

👤 Specifying the assigment

What is your web about?

• DIALOG TEMPLATE

{rawAssigment}

-> {assigment} Website assignment and specification

✨ Improving the title

• PERSONA Jane, Copywriter and Marketing Specialist.

As an experienced marketing specialist, you have been entrusted with improving the name of your client's business.

A suggested name from a client:
"{rawTitle}"

Assignment from customer:

> {assigment}

## Instructions:

-   Write only one name suggestion
-   The name will be used on the website, business cards, visuals, etc.

-> {enhancedTitle} Enhanced title

👤 Website title approval

Is the title for your website okay?

• DIALOG TEMPLATE

{enhancedTitle}

-> {title} Title for the website

🐰 Cunning subtitle

• PERSONA Josh, a copywriter, tasked with creating a claim for the website.

As an experienced copywriter, you have been entrusted with creating a claim for the "{title}" web page.

A website assignment from a customer:

> {assigment}

## Instructions:

-   Write only one name suggestion
-   Claim will be used on website, business cards, visuals, etc.
-   Claim should be punchy, funny, original

-> {claim} Claim for the web

🚦 Keyword analysis

• PERSONA Paul, extremely creative SEO specialist.

As an experienced SEO specialist, you have been entrusted with creating keywords for the website "{title}".

Website assignment from the customer:

> {assigment}

## Instructions:

-   Write a list of keywords
-   Keywords are in basic form

## Example:

-   Ice cream
-   Olomouc
-   Quality
-   Family
-   Tradition
-   Italy
-   Craft

-> {keywords} Keywords

🔗 Combine the beginning

• SIMPLE TEMPLATE

# {title}

> {claim}

-> {contentBeginning} Beginning of web content

🖋 Write the content

• PERSONA Jane

As an experienced copywriter and web designer, you have been entrusted with creating text for a new website {title}.

A website assignment from a customer:

> {assigment}

## Instructions:

-   Text formatting is in Markdown
-   Be concise and to the point
-   Use keywords, but they should be naturally in the text
-   This is the complete content of the page, so don't forget all the important information and elements the page should contain
-   Use headings, bullets, text formatting

## Keywords:

{keywords}

## Web Content:

{contentBeginning}

-> {contentBody} Middle of the web content

🔗 Combine the content

• SIMPLE TEMPLATE

{contentBeginning}

{contentBody}

-> {websiteContent}

Following is the scheme how the promptbook above is executed:

%% 🔮 Tip: Open this on GitHub or in the VSCode website to see the Mermaid graph visually

flowchart LR
  subgraph "🌍 Create website content"

      direction TB

      input((Input)):::input
      templateSpecifyingTheAssigment(👤 Specifying the assigment)
      input--"{rawAssigment}"-->templateSpecifyingTheAssigment
      templateImprovingTheTitle(✨ Improving the title)
      input--"{rawTitle}"-->templateImprovingTheTitle
      templateSpecifyingTheAssigment--"{assigment}"-->templateImprovingTheTitle
      templateWebsiteTitleApproval(👤 Website title approval)
      templateImprovingTheTitle--"{enhancedTitle}"-->templateWebsiteTitleApproval
      templateCunningSubtitle(🐰 Cunning subtitle)
      templateWebsiteTitleApproval--"{title}"-->templateCunningSubtitle
      templateSpecifyingTheAssigment--"{assigment}"-->templateCunningSubtitle
      templateKeywordAnalysis(🚦 Keyword analysis)
      templateWebsiteTitleApproval--"{title}"-->templateKeywordAnalysis
      templateSpecifyingTheAssigment--"{assigment}"-->templateKeywordAnalysis
      templateCombineTheBeginning(🔗 Combine the beginning)
      templateWebsiteTitleApproval--"{title}"-->templateCombineTheBeginning
      templateCunningSubtitle--"{claim}"-->templateCombineTheBeginning
      templateWriteTheContent(🖋 Write the content)
      templateWebsiteTitleApproval--"{title}"-->templateWriteTheContent
      templateSpecifyingTheAssigment--"{assigment}"-->templateWriteTheContent
      templateKeywordAnalysis--"{keywords}"-->templateWriteTheContent
      templateCombineTheBeginning--"{contentBeginning}"-->templateWriteTheContent
      templateCombineTheContent(🔗 Combine the content)
      templateCombineTheBeginning--"{contentBeginning}"-->templateCombineTheContent
      templateWriteTheContent--"{contentBody}"-->templateCombineTheContent

      templateCombineTheContent--"{websiteContent}"-->output
      output((Output)):::output

      classDef input color: grey;
      classDef output color: grey;

  end;

• More template samples
• Read more about .ptbk.md file format here

Note: We are using postprocessing functions like unwrapResult that can be used to postprocess the result.

📦 Packages

This library is divided into several packages, all are published from single monorepo. You can install all of them at once:

npm i ptbk

Or you can install them separately:

⭐ Marked packages are worth to try first

• ⭐ ptbk - Bundle of all packages, when you want to install everything and you don't care about the size
• promptbook - Same as ptbk
• @promptbook/core - Core of the library, it contains the main logic for promptbooks
• @promptbook/node - Core of the library for Node.js environment
• @promptbook/browser - Core of the library for browser environment
• ⭐ @promptbook/utils - Utility functions used in the library but also useful for individual use in preprocessing and postprocessing LLM inputs and outputs
• @promptbook/markdown-utils - Utility functions used for processing markdown
• (Not finished) @promptbook/wizzard - Wizard for creating+running promptbooks in single line
• @promptbook/execute-javascript - Execution tools for javascript inside promptbooks
• @promptbook/openai - Execution tools for OpenAI API, wrapper around OpenAI SDK
• @promptbook/anthropic-claude - Execution tools for Anthropic Claude API, wrapper around Anthropic Claude SDK
• @promptbook/azure-openai - Execution tools for Azure OpenAI API
• @promptbook/langtail - Execution tools for Langtail API, wrapper around Langtail SDK
• @promptbook/fake-llm - Mocked execution tools for testing the library and saving the tokens
• @promptbook/remote-client - Remote client for remote execution of promptbooks
• @promptbook/remote-server - Remote server for remote execution of promptbooks
• @promptbook/types - Just typescript types used in the library
• @promptbook/cli - Command line interface utilities for promptbooks

📚 Dictionary

The following glossary is used to clarify certain concepts:

Core concepts

• 📚 Collection of pipelines
• 📯 Pipeline
• 🎺 Pipeline templates
• 🤼 Personas
• ⭕ Parameters
• 🚀 Pipeline execution
• 🧪 Expectations
• ✂️ Postprocessing
• 🔣 Words not tokens
• ☯ Separation of concerns

Advanced concepts

• 📚 Knowledge (Retrieval-augmented generation)
• 🌏 Remote server
• 🃏 Jokers (conditions)
• 🔳 Metaprompting
• 🌏 Linguistically typed languages
• 🌍 Auto-Translations
• 📽 Images, audio, video, spreadsheets
• 🔙 Expectation-aware generation
• ⏳ Just-in-time fine-tuning
• 🔴 Anomaly detection
• 👮 Agent adversary expectations
• view more

🔌 Usage in Typescript / Javascript

• Simple usage
• Usage with client and remote server

➕➖ When to use Promptbook?

➕ When to use

• When you are writing app that generates complex things via LLM - like websites, articles, presentations, code, stories, songs,...
• When you want to separate code from text prompts
• When you want to describe complex prompt pipelines and don't want to do it in the code
• When you want to orchestrate multiple prompts together
• When you want to reuse parts of prompts in multiple places
• When you want to version your prompts and test multiple versions
• When you want to log the execution of prompts and backtrace the issues

See more

➖ When not to use

• When you have already implemented single simple prompt and it works fine for your job
• When OpenAI Assistant (GPTs) is enough for you
• When you need streaming (this may be implemented in the future, see discussion).
• When you need to use something other than JavaScript or TypeScript (other languages are on the way, see the discussion)
• When your main focus is on something other than text - like images, audio, video, spreadsheets (other media types may be added in the future, see discussion)
• When you need to use recursion (see the discussion)

See more

🐜 Known issues

• 🤸‍♂️ Iterations not working yet
• ⤵️ Imports not working yet

🧼 Intentionally not implemented features

• ➿ No recursion
• 🏳 There are no types, just strings

❔ FAQ

If you have a question start a discussion, open an issue or write me an email.

• ❔ Why not just use the OpenAI SDK / Anthropic Claude SDK / ...?
• ❔ How is it different from the OpenAI`s GPTs?
• ❔ How is it different from the Langchain?
• ❔ How is it different from the DSPy?
• ❔ How is it different from anything?
• ❔ Is Promptbook using RAG (Retrieval-Augmented Generation)?
• ❔ Is Promptbook using function calling?

⌚ Changelog

See CHANGELOG.md

📜 License

Promptbook by Pavol Hejný is licensed under CC BY 4.0

🎯 Todos

See TODO.md

🖋️ Contributing

I am open to pull requests, feedback, and suggestions. Or if you like this utility, you can ☕ buy me a coffee or donate via cryptocurrencies.

You can also ⭐ star the promptbook package, follow me on GitHub or various other social networks.