Changelog
=========
Version 3.12.0
--------------
1. **Python**
- Fix the hashing algorithm to exclude the unstable `hash()` output from
digests thus fixed an issue with handling a `None` suffix in digests,
improving the stability and accuracy of digest computations.
- Section addressing has been adjusted to start from 1 instead of 0 for
readability.
- Made `eval-sections` the default command, reflecting a focus on ease of use
and user interaction.
2. **Environment**
- Underscores have been removed from auxiliary directory file names.
- Add a link to the arXiv [paper](https://arxiv.org/abs/2501.10738).
3. **Vim**
- Added new options for ignoring specific LaTeX sections with the `lignore`
feature, enhancing customization for LaTeX users.
- Updated the Vim async tag pattern `BG` -> `LR`.
Version 3.11.0
--------------
1. **Python**
- The interpreters have been refactored into their own submodule, improving
code organization.
- Added preliminary support for
[shell](./python/litrepl/interpreters/shell.py) interpreters, allowing the
system to handle shell commands.
- Introduced new commands: `print-grammar` to output the grammar and
`print-regexp` for regular expression management.
- Introduced adjustable code section labels, removing the default `lcode`
sections support.
- Improved debugging and configuration support in IPython by ensuring
expanded paths and correct handling of the DEBUG setting.
- Support for automatic file type detection with `--filetype=auto` has been
implemented.
- The support for `--irreproducible-exitcode` argument was added
2. **Vim**
- Fixed cursor handling issues in `LEvalMon`, improving text navigation and
evaluation feedback within Vim.
- Improve error reporting performed by the `LOpenErr` command.
Version 3.10.1
--------------
Minor environment changes
Version 3.10.0
--------------
1. **Python**
- Code refactoring for better code management and efficiency: moved main
logic in `litrepl` from `python/bin/litrepl` to a new file
[python/litrepl/main.py](./python/litrepl/main.py).
- Improved handling of subprocess management for the Python and AI
interpreters, ensuring proper cleanup and output management. Additionally,
calls to `os.system` were mostly replaced with native Python calls.
- Added robust error handling and message retrieval for the interpreter
interface, providing clearer errors when the interpreter fails.
2. **Vim**
- Introduced the default environment variable check for `LITREPL_WORKDIR`,
`LITREPL_PYTHON_AUXDIR`, `LITREPL_AI_AUXDIR` etc. environment variables.
- Updated Vim functions to consider both buffer and global settings,
improving flexibility in command generation (`LitReplGet` helper function).
- Updated commands related to AI tasks like `:LAIStyle`, `:LAIFile`, and
`:LAICode` to handle rephrasing and AI interaction more efficiently.
- Adjusted internal Vim functions to handle text width settings and cursor
positionings.
3. **Environment**
- Replaced `test.sh` with [runtests.sh](./sh/runtests.sh) in the build and test
process for enhanced test execution flow.
- Allowed more environment configuration by introducing more settings that
default from environment variables, streamlining setup customization.
- Updated Nix build script ([default.nix](./default.nix)) to reflect changes
in the test script location and hash updates for source verification.
- Added this changelog and the AI-powered changelog helper script
[diffchanges.sh](./sh/diffchanges.sh).
Version 3.9.0
-------------
1. **Python**
- Added assertions to ensure a successful attachment to the interpreter,
enhancing error handling robustness.
2. **Vim**
- Enhanced [vim/plugin/litrepl.vim](./vim/plugin/litrepl.vim) by preserving
caret positions and restructuring comments for function alignment.
- Significant expansions in
[vim/plugin/litrepl_extras.vim](./vim/plugin/litrepl_extras.vim) with
multiple new functions for region handling, AI task management, and
interaction across different scopes, substantially improving user
experience with AI functionalities in Vim. Several new commands such as
`LAITell`, `LAICont`, `LAIStyle`, `LAIFile`, and `LAICode` are introduced
to support a wide range of code and AI-related tasks.
3. **Environment**
- Added a new markdown file `./doc/screencast.md` with examples for
evaluating Python code in Vim, offering insights into the screencast
documentation process.
- Adjusted `screencast.sh` to be executable and made improvements to
accommodate a target file parameter.
- Introduced a new test function `test_vim_ai_query` in `test.sh`,
facilitating new test coverage for AI query functionality.
⌨️ LitRepl ⌨️
[Changelog](./CHANGELOG.md) | [Installation](#-installation) | [Usage](#basic-execution) | [Gallery](#-gallery)
**Litrepl** is a command-line processor for *Markdown* or *LaTeX* documents with
**literate programming** code sections. Instructed by its arguments, it
evaluates and updates sections via background interpreters. Interpreters can
stay active for a **read-eval-paste-loop** style. The repository includes a Vim
plugin to demonstrate editor integration.

_Notes:_
* _[literate programming](https://en.wikipedia.org/wiki/Literate_programming)_
* _[read-eval-print-loop coding](https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop)_
|
Preprint: (2025, Sergei Mironov)
Litrepl: Literate Paper Processor Promoting Transparency More Than Reproducibility
arXiv:2501.10738
|
🔥 Features
-----------
* **Document formats**
Markdown _(Example [[MD]](./doc/example.md))_ **|**
[LaTeX](https://www.latex-project.org/)
_(Examples [[TEX]](./doc/example.tex)[[PDF]](./doc/example.pdf))_
* **Interpreters**
[Sh](https://en.wikipedia.org/wiki/Bourne_shell) **|**
[Bash](https://www.gnu.org/software/bash/) **|**
[Python](https://www.python.org/) **|**
[IPython](https://ipython.org/) **|**
[Aicli](https://github.com/sergei-mironov/aicli)
* **Editor integration**
[Vim](https://www.vim.org/scripts/script.php?script_id=6117) _(plugin source included)_
✅ Requirements
---------------
* **POSIX-compatible OS**, typically a Linux. The tool relies on POSIX
operations, notably pipes, and depends on certain Shell commands.
* **lark-parser** and **psutil** Python packages.
* **[Socat](http://www.dest-unreach.org/socat/)** (Optional) Needed for
`litrepl repl` and Vim's `LTerm` commands to work.
📚 Contents
-----------
⚙️ Installation
This repository hosts the Litrepl tool, a standalone command-line application
and an interface plugin for the Vim editor. The author's preferred installation
method is using Nix, but if you choose not to use it, you'll need to install one
or both components separately. Below, we outline several common installation
methods.
Release versions, from Pypi and Vim.org
pip install litrepl
- Download the
litrepl.vim
from the vim.org
script page and put it into
your ~/.vim/plugin
folder.
Latest versions, from Git, using Pip and Vim-Plug
- Install the
litrepl
Python package with pip:
$ pip install --user git+https://github.com/sergei-mironov/litrepl
$ litrepl --version
- Install the Vim plugin by adding the following line between the
plug#begin
and plug#end
lines of your .vimrc
file:
Plug 'https://github.com/sergei-mironov/litrepl' , { 'rtp': 'vim' }
Note: rtp
sets the custom vim-plugin source directory of the plugin.
Latest versions, from source, using Nix
The repository offers a suite of Nix expressions designed to optimize
installation and development processes on systems that support Nix. Consistent
with standard practices in Nix projects, the flake.nix file
defines the source dependencies, while the default.nix file
identifies the build targets.
For testing, the vim-demo
expression is a practical choice. It includes a
pre-configured Vim setup with several related plugins, including Litrepl. To
build this target, use the command nix build '.#vim-demo'
. Once the build is
complete, you can run the editor with ./result/bin/vim-demo
.
To add the Litrepl tool to your system profile, first include the Litrepl flake
in your flake inputs. Then, add litrepl-release
to
environment.systemPackages
or to your custom environment.
To include the Litrepl Vim plugin, add vim-litrepl-release
to the vimPlugins
list within your vim_configurable
expression.
Regardless of the approach, Nix will manage all necessary dependencies
automatically.
Nix are used to open the development shell, see the Development
section.
Latest versions, from source, using Pip
The Litrepl application might be installed with pip install .
run from the
project root folder. The Vim plugin part requires hand-copying
./vim/plugin/litrepl.vim
and ./vim/plugin/litrepl_extras.vim
to the ~/.vim
config folder.
The Nix-powered installation methods install the Socat tool automatically. For
other installation methods, use your system pacakge manager to install it. For
example, Ubuntu users might run sudo apt-get install socat
.
The Python interpreter is usually installed by default, along with the pip
installer. To install ipython
, you can use pip install ipython
. For the
aicli interpreter, you can run pip install sm_aicli
, or refer to the project page for additional installation
methods.
🚀 Usage
General Concepts
The Litrepl tool identifies code and result sections within a text document. It
processes the code by sending it to the appropriate interpreters and populates
the result sections with their responses. The interpreters remain active in the
background, ready to handle new inputs.
Litrepl supports subsets of Markdown and LaTeX formatting in order to
recognize the sections. Some aspects of the recognized grammars, such as section
labels, could be configured.
Basic Execution
Litrepl searches for verbatim code sections followed by zero or more result
sections. In Markdown documents, the Python code is any triple-quoted section
with a pre-configured label such as python
. The result is any triple-quoted
result
section. In LaTeX documents, sections are marked with
\begin{...}\end{...}
environments correspondingly.
The primary command for evaluating formatted documents is litrepl eval-sections
. Consider a hello.md document:
``` python
print('Hello Markdown!')
```
``` result
```
You pass it to Litrepl using:
$ cat hello.md | litrepl eval-sections > result.md
The result.md
will have all sections filled in correctly.
``` python
print('Hello Markdown!')
```
``` result
Hello Markdown!
```
Similarly, in a LaTeX document, we format code and result sections as follows:
\begin{python}
print('Hello LaTeX!')
\end{python}
\begin{result}
Hello LaTeX!
\end{result}
- LaTeX documents require a preamble introducing python/result environments to
the TeX processor. For more information, see Formatting LaTeX
documents.
By default, Litrepl tried to guess the format of the input document. Use the
--filetype=(latex|markdown)
option to set the format explicitly:
$ cat doc.md | litrepl --filetype=markdown eval-sections
$ cat doc.tex | litrepl --filetype=latex eval-sections
- The main Vim command for code section evaluation is
:LEval
. By default, it
executes the section at the cursor. To execute all sections in a document, use
:LEval all
.
Selecting Sections for Execution
By default, litrepl eval-sections
evaluates all sections in a document. To
evaluate only specific sections, the range argument should be specified. The
overall syntax is litrepl eval-sections [RANGE]
, where RANGE
can be:
N
: Represents a specific code section to evaluate, with the following
possible formats:
- A number starting from
0
. $
symbol, indicating the last section.L:C
, referring to the line and column position. Litrepl calculates the
section number based on this position.
N..N
: Represents a range of sections, determined using the rules mentioned
above.
Some examples:
$ litrepl eval-sections '0'
$ litrepl eval-sections '3..$'
$ litrepl eval-sections '34:1..$'
- The Vim command
:LEval
accepts similar syntax and also recognizes specific
keywords all
, above
, and below
. These keywords allow you to evaluate all
sections, only those above the cursor, or just the ones below the cursor,
respectively.
Managing Interpreter Sessions
Each interpreter session uses an auxiliary directory where Litrepl stores
filesystem pipes and other runtime data.
By default, the auxiliary directory path is derived from the working directory
name (for Vim, this defaults to the directory of the current file).
This behavior can be configured by:
- Setting the working directory with
LITREPL_WORKDIR
environment
variable or --workdir=DIR
command-line argument (this may also affect the
current directory of the interpreters), or - Explicitly setting the auxiliary directory with
LITREPL_<CLASS>_AUXDIR
environment variable or --<class>-auxdir=DIR
command-line argument, where
<class>
stands for either python
, ai
or sh
.
The commands litrepl start CLASS
, litrepl stop [CLASS]
, and litrepl restart [CLASS]
are used to manage interpreter sessions. They accept the interpreter
type to operate on or (for some commands) the keyword all
to apply the command
to all interpreters. Add the --<class>-interpteter=CMDLINE
to adjust the
command line to run, but be careful - Litrepl adds more arguments to configure
prompts and verbosity to some interpreters, notably to the pythons.
$ litrepl --python-interpreter=ipython start python
$ litrepl --sh-interpreter=/opt/bin/mybash start sh
$ litrepl restart all
$ litrepl stop
- The equivalent Vim commands are
:LStart CLASS
, :LStop [CLASS]
, and
:LRestart [CLASS]
. For the corresponding Vim configuration variables, see
the reference section below.
The litrepl status [CLASS]
command queries the information about the currently
running interpreters. The command reveals the process PID and the command-line
arguments. For stopped interpreters, the last exit codes are also listed.
Specifying CLASS
prints the status for this class of interpreters only.
$ litrepl status
python 3900919 - python3 -m IPython --config=/tmp/litrepl_1000_a2732d/python/litrepl_ipython_config.py --colors=NoColor -i
ai 3904696 - aicli --readline-prompt=
- The corresponding Vim command is
:LStatus
. No CLASS
argument is currently
supported.
Asynchronous Processing
Litrepl can generate an output document before the interpreter has finished
processing. If the evaluation takes longer than a timeout, Litrepl leaves a
marker, enabling it to continue from where it was stopped during future runs.
The --timeout=SEC[,SEC]
option allows you to set timeouts. The first number
specifies the initial execution timeout in seconds, while the optional second
number sets the timeout for subsequent attempts. By default, both timeouts are
set to infinity.
For instance, executing litrepl --timeout=3.5 eval-sections
on the
corresponding program yields:
``` python
from tqdm import tqdm
from time import sleep
for i in tqdm(range(10)):
sleep(1)
```
``` result
30%|███ | 3/10 [00:03<00:07, 1.00s/it]
[BG:/tmp/nix-shell.vijcH0/litrepl_1000_a2732d/python/litrepl_eval_5503542553591491252.txt]
```
Upon re-executing the document, Litrepl resumes processing from the marker. Once
evaluation concludes, it removes the marker from the output section.
The command litrepl interrupt
sends an interrupt signal to the interpreter,
prompting it to return control sooner (with an exception).
- The equivalent Vim commands are
:LEvalAsync
(defaulting to a 0.5-second
timeout) and :LInterrupt
. - The Vim plugin also provides the
:LEvalMon
command, which facilitates
continuous code evaluation with no delay. Interrupting this with Ctrl+C will
make Litrepl return control to the editor, leaving the evaluation ongoing in
the background.
Attaching Interpreter Sessions
The command litrepl repl [CLASS]
where CLASS
specifies interpreter class:
python
(the default) ai
or sh
, attaches to interpreter sessions. For this
command to work, socat tool needs to be
installed on your system. Litrepl blocks the pipes for the time of interaction
so no evaluation is possible while the repl session is active. For Python and
Shell interpreters, the command prompt is disabled which is a current technical
limitation. Use Ctrl+D
to safely detach the session. For example:
$ litrepl repl python
Opening the interpreter terminal (NO PROMPTS, USE `Ctrl+D` TO DETACH)
W = 'Hello from repl'
^D
$
Use litrepl eval-code [CLASS]
to direct code straight to the interpreter,
bypassing any section formatting steps. In contrast to the repl
command,
eval-code
mode features prompt detection, allowing the tool to display the
interpreter's response and detach while keeping the session open.
For example, after manually defining the W
variable in the example above, it
can be queried as in a typical IPython session.
$ echo 'W' | litrepl eval-code
'Hello from repl'
The eval-code
command can be utilized for batch processing and managing
sessions, in a manner similar to how the expect
tool is used.
- The equivalent Vim commands are
:LRepl [CLASS]
or :LTerm [CLASS]
. Both
commands open Vim terminal window.
Experimental AI Features
Litrepl experimentally supports Aicli
terminal allowing users to query external language models. In order to try it,
install the interpreter and use ai
as the name for code sections. For
low-speed models it might be convenient to use :LEvalMon
command to monitor
the text generation in real time.
``` ai
/model gpt4all:"./_models/Meta-Llama-3-8B-Instruct.Q4_0.gguf"
Hi chat! What is your name?
```
``` result
I'm LLaMA, a large language model trained by Meta AI. I'm here to help answer
any questions you might have and provide information on a wide range of topics.
How can I assist you today?
```
All Aicli /
-commands like the /model
command above are passed as-is to the
interpreter. The /ask
command is added automatically at the of each section,
so make sure that ai
secions have self-contained questions.
As a pre-processing step, Litrepl can paste text from other sections of the
document in place of special reference markers. The markers have the following
format:
>>RX<<
, where X
is a number - references a section number X
(starting
from zero).^^RX^^
, where X
is a number - references the section X
times above the
current one.vvRXvv
, where X
is a number - references the section X
times below the
current one.
``` ai
AI, what do you think the following text means?
^^R1^^
```
``` result
Another interesting piece of text!
This is an example of a chatbot introduction or "hello message." It appears to
be written in a friendly, approachable tone, with the goal of establishing a
connection with users.
```
Application Scenarios
Command Line, Foreground Evaluation
When performing batch processing of documents, it might be necessary to initiate
a new interpreter session solely for the evaluation's duration rather than
re-using the currently running session. The --foreground
option can be used to
activate this mode.
$ cat document.md.in | litrepl --foreground eval-sections > document.md
Command Line, Detecting Python Exceptions
Another frequently requested feature is the ability to report unhandled
exceptions. Litrepl can be configured to return a non-zero exit code in such
scenarios.
$ cat document.md
``` python
raise Exception("D'oh!")
```
$ cat document.md | litrepl --foreground --exception-exit=200 eval-sections
$ echo $?
200
In this example, the --foreground
option instructs Litrepl to start a new
interpreter session, stopping it upon completion. The --exception-exit=200
option specifies the exit code to be returned in the event of unhandled
exceptions.
GNU Make, Evaluating Code Sections in Project Documentation
A typical Makefile recipe for updating documentation is structured as follows:
SRC = $(shell find -name '*\.py')
.stamp_readme: $(SRC) Makefile
cp README.md _README.md.in
cat _README.md.in | \
litrepl --foreground --exception-exit=100 \
--python-interpreter=ipython \
--sh-interpreter=- \
eval-sections >README.md
touch $@
.PHONY: readme
readme: .stamp_readme
Here, $(SRC)
is expected to include the filenames of dependencies. With this
recipe, we can run make readme
to evaluate the python sections. By passing -
wealso tell Litrepl to ignore shell sections.
Vim, Setting Up Keybindings
The litrepl.vim
plugin does not define any keybindings, but users could do it
by themselves, for example:
nnoremap <F5> :LEval<CR>
nnoremap <F6> :LEvalAsync<CR>
Vim, Inserting New Sections
The litrepl.vim
plugin doesn't include tools for creating section formatting,
however they can be added easily if required. Below, we demonstrate how to
define the :C
command inserting new python
sections.
command! -buffer -nargs=0 C normal 0i``` python<CR>```<CR><CR>``` result<CR>```<Esc>4k
Vim, Running the Initial Section After Interpreter Restart
Below we demonstrate how to define the :LR
command for running first section
after the restart.
command! -nargs=0 LR LRestart | LEval 0
Vim, Evaluating Selected Text
Litrepl vim plugin defines LitReplEvalSelection
function which runs the
selection as a virtual code section. The section type is passed as the function
argument. For example, calling LitReplEvalSelection('ai')
will execute the
selection as if it is an ai
code section. The execution result is pasted right
after the selection as a plain text. LitReplEvalSelection('python')
would pipe
the selection through the current Python interpreter.
To use the feature, define a suitable key binding (Ctrl+K
in this example),
vnoremap <C-k> :call LitReplEvalSelection('ai')<CR>
Now write a question to the AI in any document, select it and hit Ctrl+K.
Hi model. What is the capital of New Zealand?
Upon the keypress, Litrepl pipes the selection through the AI interpreter - the
aicli
at the time of this writing - and paste the response right after the
last line of the original selection.
Hi model. What is the capital of New Zealand?
The capital of New Zealand is Wellington.
Internally, the plugin just uses eval-code
Litrepl command.
Vim, Calling for AI on a visual selection
The repository includes litrepl_extras.vim, which
defines extra tools for interacting with AI. These tools are based on the single
low-level LitReplAIQuery()
function.
The function enables the creation of an AI chat query possibly incorporating the
current file and any selected text. The AI model's response then returned
alongside with the Litrepl error code.
Based on this function, the following two middle-level functions are defined:
LitReplTaskNew(scope, prompt)
LitReplTaskContinue(scope, prompt)
Both functions take the prompt, produce the AI model response and decide where
to insert it. However, the key difference is that the first function determines
the target location based on user input (like cursor position or selection),
while the second function re-applies the previously used position, allowing
users to make changes easilly.
Finally, a number of high-level commands have been established. Each of these
commands receives an input string that directs the model on what action to take.
The user input can contain /S
or /F
tokens, which are replaced with the
values of the visual selection and the current file, respectively.
Command | Description | Incorporates | Updates |
---|
LAI | Passes the prompt as-is | Input, Selection | Cursor, Selection |
LAICont | Passes the prompt as-is | Input, Selection | Last |
LAIStyle | Asks to improve language style | Input, Selection | Selection |
LAICode | Asks to modify a code snippet | Input, Selection | Cursor, Selection |
LAITell | Asks to describe a code snippet | Input, Selection | Terminal* |
LAIFile | Asks to change a whole file | Input, Selection, File | File |
LAITell
shows the response in the AI terminal instead of inserting it into
the document.
As with the selection evaluation mode, the aicli
interpreter stays
active in the background, maintaining the log of the conversation.
Direct interaction with the interpreter functions as expected. The LTerm ai
command opens the Vim terminal as usual, enabling communication with a model
through aicli
text commands.
In-Depth Reference
Vim Commands and Command-Line Attributes
Vim ![]() | Command line ![]() | Description |
---|
:LStart [T] | litrepl start [T] | Start the background interpreter |
:LStop [T] | litrepl stop [T] | Stop the background interpreter |
:LRestart [T] | litrepl restart [T] | Restart the background interpreter |
:LStatus [T] | litrepl status [T] <F | Print the background interpreter status |
:LEval [N] | lirtepl eval-sections L:C <F | Evaluate the section under the cursor synchronously |
:LEval above | lirtepl eval-sections '0..N' <F | Evaluate sections above and under the cursor synchronously |
:LEval below | lirtepl eval-sections 'N..$' <F | Evaluate sections below and under the cursor synchronously |
:LEval all | lirtepl eval-sections <F | Evaluate all code sections in a document |
:LEvalAsync N | lirtepl --timeout=0.5,0 eval-sections N <F | Start or continue asynchronous evaluation of the section under the cursor |
:LInterrupt N | lirtepl interrupt N <F | Send SIGINT to the interpreter evaluating the section under the cursor and update |
:LEvalMon N | while .. do .. done | Start or continue monitoring asynchronous code evaluation |
N/A | lirtepl eval-code <P | Evaluate the given code verbatim |
:LTerm [T] | lirtepl repl [T] | Connect to the interpreter using GNU socat |
:LOpenErr | litrepl ... 2>F | View errors |
:LVersion | litrepl --version | Show version |
Where
T
Type of the interpreter: python
, ai
or sh
(some commands also accept all
)F
Path to a Markdown or LaTeX fileP
Path to a Python scriptN
Number of code section to evaluate, starting from 0.L:C
denotes line:column of the cursor.
Command Line Arguments and Vim Variables
Vim setting ![]() | CLI argument ![]() | Description |
---|
set filetype | --filetype=T | Input file type: latex |markdown |
let g:litrepl_python_interpreter=B | --python-interpreter=B | The Python interpreter to use |
let g:litrepl_ai_interpreter=B | --ai-interpreter=B | The AI interpreter to use |
let g:litrepl_sh_interpreter=B | --sh-interpreter=B | The shell interpreter to use |
let g:litrepl_python_auxdir=D | --python-auxdir=D | The auxiliary files directory used by Python interpreter |
let g:litrepl_ai_auxdir=D | --ai-auxdir=D | The auxiliary files directory used by AI interpreter |
let g:litrepl_sh_auxdir=D | --sh-auxdir=D | The auxiliary files directory used by a shell interpreter |
let g:litrepl_workdir=D | --workdir=D | The auxiliary files directory used by AI interpreter |
let g:litrepl_debug=0/1 | --debug=0/1 | Print debug messages to the stderr |
let g:litrepl_timeout=FLOAT | --timeout=FLOAT | Timeout to wait for the new executions, in seconds, defaults to inf |
T
Type of the document: tex
or markdown
(the default).B
Interpreter command to use, -
or auto
(the default). -
value
disabled this type of interpreters; auto
asks litrep to guess the best
available interpreter.D
Filesystem directoryFLOAT
Should be formatted as 1
or 1.1
or inf
. Note: command line
argument also accepts a pair of timeouts.
Command Line Arguments Summary
usage: litrepl [-h] [-v] [--filetype STR] [--python-markers STR[,STR]]
[--ai-markers STR[,STR]] [--sh-markers STR[,STR]]
[--python-interpreter EXE] [--ai-interpreter EXE]
[--sh-interpreter EXE] [--python-auxdir DIR] [--ai-auxdir DIR]
[--sh-auxdir DIR] [--timeout F[,F]] [--propagate-sigint]
[-d INT] [--verbose] [-C DIR] [--pending-exitcode INT]
[--irreproducible-exitcode INT] [--exception-exitcode INT]
[--foreground] [--map-cursor LINE:COL:FILE]
[--result-textwidth NUM]
{start,stop,restart,status,parse,parse-print,eval-sections,eval-code,repl,interrupt,print-regexp,print-grammar}
...
positional arguments:
{start,stop,restart,status,parse,parse-print,eval-sections,eval-code,repl,interrupt,print-regexp,print-grammar}
Commands to execute
start Start the background interpreter.
stop Stop the background interpreters.
restart Restart the background interpreters.
status Print background interpreter's status.
parse Parse the input file without futher processing
(diagnostics).
parse-print Parse and print the input file back
(diagnostics).
eval-sections Parse stdin, evaluate the specified sections (by
default - all available sections), print the
resulting file to stdout.
eval-code Evaluate the code snippet.
repl Connect to the background terminal using GNU
socat.
interrupt Send SIGINT to the background interpreter.
print-regexp Print regexp matching start of code sections for
the given file type.
print-grammar Print the resulting grammar for the given
filetype.
options:
-h, --help show this help message and exit
-v, --version Print version.
--filetype STR Specify the type of input formatting
(markdown|[la]tex|auto).
--python-markers STR[,STR] TODO
--ai-markers STR[,STR] TODO
--sh-markers STR[,STR] TODO
--python-interpreter EXE Python interpreter command line, or `auto`.
Defaults to the LITREPL_PYTHON_INTERPRETER
environment variable if set, otherwise "auto".
Litrepl determines "python" or "ipython" type
according to the value.
--ai-interpreter EXE `aicli` interpreter command line or `auto`.
Defaults to the LITREPL_AI_INTERPRETER
environment variable if set, otherwise "auto".
--sh-interpreter EXE Shell interpreter command line or `auto`.
Defaults to the LITREPL_SH_INTERPRETER
environment variable if set, otherwise "auto".
--python-auxdir DIR This directory stores Python interpreter pipes.
It defaults to LITREPL_PYTHON_AUXDIR if set;
otherwise, it's created in the system's
temporary directory, named after the current
working directory.
--ai-auxdir DIR This directory stores AI interpreter pipes. It
defaults to LITREPL_AI_AUXDIR if set; otherwise,
it's created in the system's temporary
directory, named after the current working
directory.
--sh-auxdir DIR This directory stores AI interpreter pipes. It
defaults to LITREPL_SH_AUXDIR if set; otherwise,
it's created in the system's temporary
directory, named after the current working
directory.
--timeout F[,F] Timeouts for initial evaluation and for pending
checks, in seconds. If the latter is omitted, it
is considered to be equal to the former one.
--propagate-sigint If set, litrepl will catch and resend SIGINT
signals to the running interpreter. Otherwise it
will just terminate itself leaving the
interpreter as-is.
-d INT, --debug INT Enable (a lot of) debug messages.
--verbose Be more verbose (used in status).
-C DIR, --workdir DIR Set the working directory before execution. By
default, it uses LITREPL_WORKDIR if set,
otherwise remains the current directory. This
affects the directory of a new interpreter and
the --<interpreter>-auxdir option.
--pending-exitcode INT Return this error code if whenever a section
hits timeout.
--irreproducible-exitcode INT
Return this error code if a section outputs a
different result than the one that is already
present in the document.
--exception-exitcode INT Return this error code at exception, if any.
Note: this option might not be defined for some
interpreters. It takes affect only for newly-
started interpreters.
--foreground Start a separate session and stop it when the
evaluation is done. All --*-auxdir settings are
ignored in this mode.
--map-cursor LINE:COL:FILE Calculate the new position of a cursor at
LINE:COL and write it to FILE.
--result-textwidth NUM Wrap result lines longer than NUM symbols.
🏗️ Development Guidelines
This project uses Nix as its main development
framework. The file flake.nix manages the source-level
dependencies required by Nix, whereas default.nix specifies
common build targets, including PyPI and Vim packages, demo Vim configurations,
development shells, and more.
Building Targets
To build individual Nix expressions, execute the command nix build '.#NAME'
,
replacing NAME
with the actual name of the Nix expression you want to build.
If the build is successful, Nix places the results of the last build in a
symbolic link located at ./result
.
The list of Nix build targets includes:
litrepl-release
- Litrepl script and Python liblitrepl-release-pypi
- Litrepl script and Python libvim-litrepl-release
- Vim with locally built litrepl pluginvim-litrepl-release-pypi
- Vim with litrepl plugin built from PYPIvim-test
- A minimalistic Vim with a single litrepl pluginvim-demo
- Vim configured to use litrepl suitable for recording screencastsvim-plug
- Vim configured to use litrepl via the Plug managershell-dev
- The development shellshell-screencast
- The shell for recording demonstrations, includes vim-demo
.
See local.collection
attribute-set in the default.nix for the
full list of defined targetr.
For example, to build a version of Vim pre-configured for demo, run
$ nix build '.#vim-demo'
$ ./result/bin/vim-demo
Development Environments and Setup
The default development shell is defined in the ./default.nix
as a Nix
expression named shell
which is the default name for development shells.
Running
$ nix develop
will ask Nix to install the development dependencies and open shell.
Tools for Screencast Recording
Another shell which might be useful is shell-screencast
. This would build the
full set of Litrepl tools and makes sure that the screencasting software is
available. To enter it, specify its Nix-flake path as follows:
$ nix develop '.#shell-screencast'
In the opened shell, run the screencast.sh
and wait a second, until the script
arranges demo and recorder wondows.
$ screencast.sh
screencast.sh
accepts an optional parameter specifying the template file to
open for the recording session.
Common Development Scenarios
The top-level Makefile encodes common development scenarios:
[ LitREPL-DEV ] $ make help
make[1]: Entering directory '/home/grwlf/proj/litrepl.vim'
LitREPL is a macroprocessing Python library for Litrate programming and code execution
Build targets:
help: Print help
test: Run the test script (./sh/runtests.sh)
readme: Update code sections in the README.md
wheel: Build Python wheel (the DEFAULT target)
vimbundle: Build Vim bundle
version: Print the version
dist: Build Python and Vim packages
upload: Upload Python wheel to Pypi.org (./_token.pypi is required)
make[1]: Leaving directory '/home/grwlf/proj/litrepl.vim'
The runtests.sh
script runs all tests by default, but accepts command-line
arguments for running specific tests.
[ LitREPL-DEV ] $ runtests.sh --help
Usage: runtest.sh [-d] [-i I(,I)*] [-t T(,T)*]
Arguments:
-d Be very verbose
-i I, --interpreters=I Run tests requiring interpreters matching the grep expression I
Run -i '?' to list all available interpreters.
-t T, --tests=T Run tests whose names match the grep expression T
Run -t '?' to list all available tests.
Examples:
runtests.sh -t '?' -i '?'
runtests.sh -i ipython
runtests.sh -t 'test_eval_code|test_status' -i python
🎥 Gallery
Basic usage (Show GIF)

AI capabilities (Show GIF)

Vimtex integration (Show Video)
We utilize LitRepl alongside the Vimtex plugin to edit and
preview LaTeX documents instantly.
💡 Technical Insights
The following events should normally happen after users type the :LitEval1
command:
- On the first run, LitREPL starts the Python interpreter in the background.
Its standard input and output are redirected into UNIX pipes in the current
directory.
- LitREPL runs the whole document through the express Markdown/Latex parser
determining the start/stop positions of code and result sections. The cursor
position is also available and the code from the right code section can
reach the interpreter.
- The process which reads the interpreter's response is forked out of the main
LitREPL process. The output goes to the temporary file.
- If the interpreter reports the completion quickly, the output is pasted to
the resulting document immediately. Otherwise, the temporary results are
pasted.
- Re-evaluating sections with temporary results causes LitREPL to update
these results.
🚫 Known Limitations
- Formatting: Nested code sections are not supported.
Formatting: Special symbols in the Python output could invalidate the
document.- Interpreter: Extra newline is required after Python function definitions.
- Interpreter: Stdout and stderr are joined together.
Interpreter: Evaluation of a code section locks the editor.- Interpreter: Tweaking
os.ps1
/os.ps2
prompts of the Python interpreter
could break the session. Interpreter: No asynchronous code execution.Interpreter: Background Python interpreter couldn't be interrupted
Related Tools and Projects
Edititng:
Code execution:
Useful Vim plugins:
Useful tools:
Considerations for Third-Party Tools