QLLM (QuantaLogic LLM) is a powerful and flexible Command Line Interface (CLI) for interacting with multiple Large Language Model (LLM) providers. Built with ❤️ by @quantalogic, QLLM simplifies the process of leveraging state-of-the-art language models in your projects and workflows.
⭐ If you find QLLM useful, consider giving us a star on GitHub! It helps us reach more developers and improve the tool. ⭐
QLLM is a command-line tool that allows you to interact with various Large Language Models (LLMs) through a unified interface. It supports multiple providers, enabling you to leverage different AI models for tasks such as text generation, analysis, and interactive conversations.
Install QLLM globally using npm:
npm install -g qllm
This makes the qllm command available system-wide.
Ask a question:
qllm ask "Write a 100-word story about a time traveler" --max-tokens 150
Start an interactive chat session:
qllm chat --max-tokens 150 --provider anthropic --model haiku
Stream a response:
qllm stream "Explain quantum computing" --max-tokens 200
View configuration:
qllm config --show
QLLM offers four main commands:
ask: Ask a single question and get a responsechat: Start an interactive chat sessionstream: Stream a response in real-timeconfig: View or update configuration settingsEach command supports various options to customize the behavior of the LLM:
--max-tokens <number>: Maximum number of tokens to generate (default: 256)--temperature <float>: Controls randomness (0-1, default: 0.7)--top-p <float>: Nucleus sampling parameter (0-1, default: 1)--top-k <number>: Top-k sampling parameter (1-1000, default: 250)--system <string>: System message to set context--file <path>: Path to input file--output <path>: Path to output file--format <type>: Output format (json, markdown, text)--provider <name>: LLM provider (anthropic, openai)--model <name>: Specific model to useImagine you're baking a cake, but instead of a regular recipe, you're using a magical oven that can create any cake you want. The parameters in AI language models are like the settings on this magical oven.
- The "max-tokens" is like setting a timer for how long the oven should run - it determines how much text the AI will generate.
- The "temperature" is like adjusting the oven's heat - higher values make the AI more creative but potentially less accurate, while lower values make it more predictable.
- "Top-p" and "top-k" are like filters that decide which ingredients the oven can use - they help control the variety of words the AI chooses from.
- The "system" message is like giving the oven a theme for the cake, setting the overall context.
- The "file" and "output" options are like choosing where to get your ingredients and where to put the finished cake.
- The "format" is like deciding if you want your cake sliced, whole, or as cupcakes - it determines how the AI's response is presented.
- Finally, the "provider" and "model" are like choosing which brand and type of magical oven you want to use - different ones might be better for different kinds of cakes.
All these settings work together to help you get the exact kind of AI-generated text you're looking for, just like how different oven settings help you bake the perfect cake.
Explanation of top-p and top-k:
Top-p and Top-k Sampling
These are methods to control text generation in language models:
Top-p (Nucleus Sampling)
- Selects from tokens whose cumulative probability exceeds threshold p
- Dynamic: adapts to probability distribution
- Example: p = 0.9 considers tokens up to 90% cumulative probability
Top-k Sampling
- Selects from k most probable tokens
- Fixed: always considers same number of options
- Example: k = 50 considers only top 50 most likely tokens
Key Differences
- Top-p is more flexible, adjusting to context
- Top-k uses a fixed cutoff regardless of probabilities
- Lower values: more focused output
- Higher values: more diverse, creative responses
Often used together to balance coherence and creativity in text generation.
Use the config command to manage your QLLM settings:
qllm config --show
qllm config --set-profile <profile_name>
qllm config --set-region <aws_region>
qllm config --set-model <model_name>
qllm config --set-provider <provider_name>
Available model aliases for AWS Bedrock Anthropic:
sonnet: Claude 3 Sonnetsonnet35: Claude 3.5 Sonnethaiku: Claude 3 Haiku (default)opus: Claude 3 Opusqllm ask "Write a 100-word story about a time traveler" --max-tokens 150
qllm stream "Explain the following code: function fibonacci(n) { if (n <= 1) return n; return fibonacci(n - 1) + fibonacci(n - 2); }" --format markdown
qllm chat --system "You are a helpful coding assistant. Provide code examples and explanations."
qllm ask --file input.txt --output results.json --format json
qllm ask "Summarize the benefits of exercise" --max-tokens 100 --temperature 0.9 --format markdown
qllm/
├── src/
│ ├── commands/ # Implementation of CLI commands
│ ├── config/ # Configuration management
│ ├── helpers/ # Utility functions
│ ├── providers/ # LLM provider integrations
│ ├── utils/ # General utility functions
│ └── qllm.ts # Main entry point
├── .env # Environment variables
├── package.json # Project dependencies and scripts
└── README.md # Project documentation
@anthropic-ai/bedrock-sdk: SDK for Anthropic's Bedrock modelscommander: Command-line interface creationdotenv: Environment variable managementprompts: Interactive command-line user interfaceswinston: Logging library(See package.json for a full list of dependencies and their versions)
We welcome contributions to QLLM! Here's how you can help:
git checkout -b feature/your-feature-namegit commit -m 'Add some feature'git push origin feature/your-feature-namePlease ensure your code adheres to our coding standards:
We use Jest for unit testing. To run the test suite:
npm test
When adding new features, please include appropriate test coverage. Test files should be placed in a __tests__ directory adjacent to the code being tested.
QLLM is designed to be used as a local CLI tool. To create a packaged version for distribution:
npm pack
This will create a .tgz file that can be installed globally:
npm install -g qllm-x.y.z.tgz
After installation, you can use QLLM commands directly from your terminal.
aws configure.For more issues, please check our GitHub Issues page or submit a new issue.
Q: How do I update QLLM?
A: Run npm update -g qllm to update to the latest version.
Q: Can I use QLLM in my scripts? A: Yes, QLLM can be easily integrated into shell scripts or other automation tools.
Q: Is my data secure when using QLLM? A: QLLM does not store any of your prompts or responses. However, please review the privacy policies of the LLM providers you're using.
The changelog is available at CHANGELOG
This project is licensed under the Apache License 2.0. See the LICENSE file for details.
Made with ❤️ by the QLLM team. Happy prompting!
[0.6.2] - 2024-07-03
FAQs
QLLM CLI: A versatile CLI tool for interacting with multiple AI/LLM providers. Features include chat sessions, one-time queries, image handling, and conversation management. Streamlines AI development with easy provider/model switching and configuration.
The npm package qllm receives a total of 116 weekly downloads. As such, qllm popularity was classified as not popular.
We found that qllm demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 0 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
OpenSSF has published OSPS Baseline, an initiative designed to establish a minimum set of security-related best practices for open source software projects.
Security News
Michigan TypeScript founder Dimitri Mitropoulos implements WebAssembly runtime in TypeScript types, enabling Doom to run after processing 177 terabytes of type definitions.
Security News
vlt's new "reproduce" tool verifies npm packages against their source code, outperforming traditional provenance adoption in the JavaScript ecosystem.