LitREPL is a command-line tool that brings together the benefits of literate programming and read-eval-print-loop coding. LitREPL comes bundled with an interface Vim plugin, integrating it into the editor.
lark-parser, psutil (Required).GNU socat (Optional)This repository includes the Litrepl tool in Python and an interface Vim plugin.
The Python part might be installed with pip install . run from the project
folder. The Vim part requires hand-copying ./vim/plugin/litrepl.vim to the
~/.vim config folder or using any Vim plugin manager, e.g. Vim-Plug.
pip install litrepllitrepl.vim from the vim.org
script page and put it into
your ~/.vim/plugin folder.
litrepl Python package with pip:
$ pip install --user git+https://github.com/sergei-mironov/litrepl
$ litrepl --version
plug#begin and plug#end lines of your .vimrc file:
Plug 'https://github.com/sergei-mironov/litrepl' , { 'rtp': 'vim' }
rtp sets the custom vim-plugin source directory of the plugin.
The repository also includes a set of Nix expressions that automate installation on Nix-enabled systems.
Nix supports
configurable Vim expressions.
To enable the Litrepl plugin, add the vim-litrepl.vim-litrepl-release to the
list of Vim plugins and put this version of vim into your Nix profile. Litrepl
and its dependencies will be installed automatically.
{ litrepl }:
...
vim_configurable.customize {
name = "vim";
vimrcConfig.packages.myVimPackage = with pkgs.vimPlugins; {
start = [
...
litrepl.vim-litrepl-release
...
];
};
}
Note: vim-demo expression from the default.nix provides
an example Vim configuration. Use nix build '.#vim-demo' to build it and then
./result/bin/vim-demo to run the editor.
See the Development section for more details.
The tool sends verbatim sections from a document to external interpreters, receiving the evaluated results in return. Litrepl currently supports two flavors of Python and the Aicli interpreter.
Litrepl recognises verbatim code sections followed by zero or more result
sections. In Markdown documents, the code is any triple-quoted section labeled
as python. The result is any triple-quoted result section. In LaTeX
documents, sections are marked with \begin{python}\end{python} and
\begin{result}\end{result} environments correspondingly.
litrepl eval-sections is the main command evaluating the formatted document.
To run the evaluation, send the file to the input of the shell command.
For example:
$ cat file.md
``` python
print('Hello Markdown!')
```
``` result
```
$ cat file.md | litrepl eval-sections
.. would produce a Markdown document containing the properly filled result section.
``` python
print('Hello Markdown!')
```
``` result
Hello Markdown!
```
Below we also show what the relevant LaTeX part would look like:
\begin{python}
print('Hello LaTeX!')
\end{python}
\begin{result}
Hello LaTeX!
\end{result}
:LEval.:LEval accepts optional argument
denoting the range: all, above (the cursor), below (the cursor), section
number, etc.--filetype=tex for Tex
documents. Vim plugin does this automatically based on the filetype
variable.litrepl start, litrepl stop and litrepl restart manage the interpreter
sessions. The commands also accepts the type of the interpreter to operation on.
IPython interpreter is assumed by default.
litrepl status queries the information about the interpreters running in
the background. The command reveals the process PID and the command-line arguments.
$ litrepl status
# Format:
# TYP PID EXITCODE CMD
python 3900919 - python3 -m IPython --config=/tmp/litrepl_1000_a2732d/python/litrepl_ipython_config.py --colors=NoColor -i
ai 3904696 - aicli --readline-prompt=
:LStart, :LStop, :LRestart and
:LStatusLitrepl can produce output document earlier than the interpreter reports the completion. In cases where the evaluation takes longer to finish, LitREPL will leave a marker that allows it to pick up where it left off on subsequent executions.
litrepl --timeout=3.5 eval-sections changes the reading timeout from the
default infinity the specified number of seconds. The output would be:
``` python
from tqdm import tqdm
from time import sleep
for i in tqdm(range(10)):
sleep(1)
```
``` result
30%|███ | 3/10 [00:03<00:07, 1.00s/it]
[BG:/tmp/nix-shell.vijcH0/litrepl_1000_a2732d/python/litrepl_eval_5503542553591491252.txt]
```
When re-executing this document, LitREPL will resume the reading. Once the evaluation is complete, it will remove the continuation marker from the output section.
litrepl interrupt will send interrupt signal to the interpreter so it return
the control earlier (with an exception).
:LEvalAsyn (with the timeout set to 0.5
seconds by default) and :LInterrupt.:LEvalMon command that enables repeated code evaluation
without any delay. Interrupting this process using Ctrl+C will cause Litrepl
to return control to the editor while leaving the evaluation in the
background.litrepl repl "manually" attaches to the interpreter session allowing us to
examine its internal state:
$ litrepl repl
Opening the interpreter terminal (NO PROMPTS, USE `Ctrl+D` TO DETACH)
W = 'Hello from repl'
^D
>>> symbols are going to appear.:LTermlitrepl eval-code might be used to pipe the code through the interpreter. The
W variable now resides in memory so we can query it as we would do in a
regular IPython session.
$ echo 'W' | litrepl eval-code
'Hello from repl'
Litrepl experimentally supports
Aicli allowing users to
query local LLMs. In order to try it, install the interpreter and use ai as
the name for code sections. For low-speed models it would be convenient to use
:LEvalMon command for evaluation.
``` ai
/model "~/.local/share/nomic.ai/GPT4All/Meta-Llama-3-8B-Instruct.Q4_0.gguf"
Hi chat! What is your name?
```
``` result
I'm LLaMA, a large language model trained by Meta AI. I'm here to help answer
any questions you might have and provide information on a wide range of topics.
How can I assist you today?
```
For AI sections, Litrepl can paste text from other sections of the document in place of reference markers. The markers have the following format:
>>RX<<, where X is a number - references a section number X (starting from zero).^^RX^^, where X is a number - references the section X times above the current one.vvRXvv, where X is a number - references the section X times below the current one.``` ai
AI, what do you think the following text means?
^^R1^^
```
``` result
Another interesting piece of text!
This is an example of a chatbot introduction or "hello message." It appears to
be written in a friendly, approachable tone, with the goal of establishing a
connection with users.
```
| Vim | Command line | Description |
|---|---|---|
:LStart [T] | litrepl start [T] | Start the background interpreter |
:LStop [T] | litrepl stop [T] | Stop the background interpreter |
:LRestart [T] | litrepl restart [T] | Restart the background interpreter |
:LStatus [T] | litrepl status [T] <F | Print the background interpreter status |
:LEval [N] | lirtepl eval-sections L:C <F | Evaluate the section under the cursor synchronously |
:LEval above | lirtepl eval-sections '0..N' <F | Evaluate sections above and under the cursor synchronously |
:LEval below | lirtepl eval-sections 'N..$' <F | Evaluate sections below and under the cursor synchronously |
:LEval all | lirtepl eval-sections <F | Evaluate all code sections in a document |
:LEvalAsync N | lirtepl --timeout=0.5,0 eval-sections N <F | Start or continue asynchronous evaluation of the section under the cursor |
:LInterrupt N | lirtepl interrupt N <F | Send SIGINT to the interpreter evaluating the section under the cursor and update |
:LEvalMon N | while .. do .. done | Start or continue monitoring asynchronous code evaluation |
| N/A | lirtepl eval-code <P | Evaluate the given code verbatim |
:LTerm | lirtepl repl [T] | Connect to the interpreter using GNU socat |
:LOpenErr | litrepl ... 2>F | Get the errors |
:LVersion | litrepl --version | Show version |
Where
T type of the interpreter: python or ai (some commands also accept all)F Path to a Markdown or LaTeX fileP Path to a Python scriptN number of code section to evaluate, starting from 0.L:C denotes line:column of the cursor.| Vim setting | CLI argument | Description |
|---|---|---|
set filetype | --filetype=D | Input file type: latex|markdown |
let g:litrepl_python_interpreter=B | --python-interpreter=B | The Python interpreter to use: python|ipython|auto (the default) |
let g:litrepl_ai_interpreter=B | --ai-interpreter=B | The AI interpreter to use: aicli|auto (the default) |
let g:litrepl_debug=0/1 | --debug=0/1 | Print debug messages to the stderr |
let g:litrepl_timeout=FLOAT | --timeout=FLOAT | Timeout to wait for the new executions, in seconds, defaults to inf |
D type of the document: tex or markdown (the default).B interpreter binary to use, defaults to auto which guesses the best one.FLOAT should be formatted as 1 or 1.1 or inf. Note: command line
argument also accepts a pair of timeouts.usage: litrepl [-h] [-v] [--filetype STR] [--python-interpreter EXE]
[--ai-interpreter EXE] [--timeout F[,F]] [--propagate-sigint]
[-d INT] [--verbose] [--python-auxdir DIR] [--ai-auxdir DIR]
[-C DIR] [--pending-exit INT] [--exception-exit INT]
[--foreground] [--map-cursor LINE:COL:FILE]
[--result-textwidth NUM]
{start,stop,restart,status,parse,parse-print,eval-sections,eval-code,repl,interrupt}
...
positional arguments:
{start,stop,restart,status,parse,parse-print,eval-sections,eval-code,repl,interrupt}
Commands to execute
start Start the background interpreter.
stop Stop the background interpreters.
restart Restart the background interpreters.
status Print background interpreter's status.
parse Parse the input file without futher processing
(diagnostics).
parse-print Parse and print the input file back
(diagnostics).
eval-sections Parse stdin, evaluate the sepcified sections (by
default - all available sections), print the
resulting file to stdout.
eval-code Evaluate the code snippet.
repl Connect to the background terminal using GNU
socat.
interrupt Send SIGINT to the background interpreter.
options:
-h, --help show this help message and exit
-v, --version Print version.
--filetype STR Specify the type of input formatting
(markdown|[la]tex).
--python-interpreter EXE Python interpreter to use (python|ipython|auto)
--ai-interpreter EXE AI interpreter to use (aicli|auto).
--timeout F[,F] Timeouts for initial evaluation and for pending
checks, in seconds. If the latter is omitted, it
is considered to be equal to the former one.
--propagate-sigint If set, litrepl will catch and resend SIGINT
signals to the running interpreter. Otherwise it
will just terminate itself leaving the
interpreter as-is.
-d INT, --debug INT Enable (a lot of) debug messages.
--verbose Be more verbose (used in status).
--python-auxdir DIR Directory to store Python interpreter pipes. By
default, it is created in the system temporary
directory with the name derived from current
working directory.
--ai-auxdir DIR Directory to store AI interpreter pipes. By
default, it is created in the system temporary
directory with the name derived from current
working directory.
-C DIR, --workdir DIR Set the new current working directory before
run. Note that changing directory has the
following effects: (1) changes directory of the
newly started interpreter (2) influence the
default --<interp>-auxdir.
--pending-exit INT Return this error code if whenever a section
hits timeout.
--exception-exit INT Return this error code at exception, if any.
Note: this option might not be defined for some
interpreters. It takes affect only for newly-
started interpreters.
--foreground Start a separate session and stop it when the
evaluation is done.
--map-cursor LINE:COL:FILE Calculate the new position of a cursor at
LINE:COL and write it to FILE.
--result-textwidth NUM Wrap result lines longer than NUM symbols.
To evaluate code section in a document:
$ cat doc/example.md | litrepl eval-sections >output.md
To evaluate a Python script:
$ cat script.py | litrepl eval-code
Note that both commands above share the same background interpreter session.
For batch processing of documents, it may be necessary to have an on-demand interpreter session available, which would exist solely for the duration of the evaluation process.
$ cat document.md.in
``` python
raise Exception("D'oh!")
```
$ cat document.md.in | litrepl --foreground --exception-exit=200 eval-sections >document.md
$ echo $?
200
Here, the --foreground argument tells Litrepl to run a new interpreter session
and then stop it before exiting, --exception-exit=200 sets the exit code
returned in the case of unhandled exceptions.
A typical Makefile recipe updating README.md might look as follows:
.stamp_readme: $(SRC) Makefile
cp README.md _README.md.in
cat _README.md.in | \
litrepl --foreground --exception-exit=100 --result-textwidth=100 \
eval-sections >README.md
touch $@
.PHONY: readme
readme: .stamp_readme
Where $(SRC) is supposed to contain valuable source filenames.
The plugin does not define any keybindings, but users could do it by themselves, for example:
nnoremap <F5> :LEval<CR>
nnoremap <F6> :LEvalAsync<CR>
Below we define :C command inserting new sections.
command! -buffer -nargs=0 C normal 0i``` python<CR>```<CR><CR>``` result<CR>```<Esc>4k
We define the :LR command running first section after the restart.
command! -nargs=0 LR LRestart | LEval 0
Thanks to IPython features, we can use exclamation to run shell commands directly from Python code sections.
``` python
!cowsay "Hello, Litrepl!"
```
``` result
_________________
< Hello, Litrepl! >
-----------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
```
Litrepl vim plugin defines LitReplEvalSelection function which runs the
selection as a virtual code section. The section type is passed as the function
argument. For example, calling LitReplEvalSelection('ai') will execute the
selection as if it is an ai code section. The execution result is pasted right
after the selection as a plain text. LitReplEvalSelection('python') would pipe
the selection through the current Python interpreter.
To use the feature, define a suitable key binding (Ctrl+K in this example),
vnoremap <C-k> :call LitReplEvalSelection('ai')<CR>
Now write a question to the AI in any document, select it and hit Ctrl+K.
Hi model. What is the capital of New Zealand?
Upon the keypress, Litrepl pipes the selection through the AI interpreter - the
aicli at the time of this writing - and paste the response right after the
last line of the original selection.
Hi model. What is the capital of New Zealand?
The capital of New Zealand is Wellington.
Internally, the plugin just uses eval-code Litrepl command.
This project uses Nix as a primary development framework. flake.nix handles the source-level Nix dependencies while the default.nix defines the common build targets including Pypi and Vim packages, demo Vim configurations, development shells, etc.
The default development shell is defined in the ./default.nix as a Nix
expression named shell which is the default name for development shells.
Running
$ nix develop
will ask Nix to install the development dependencies and open the shell.
Another shell which might be useful is shell-screencast. This would build the
full set of Litrepl tools and makes sure that the screencasting software is
available. To enter it, specify its Nix-flake path as follows:
$ nix develop '.#shell-screencast'
To build individual Nix expressions, run nix build '.#NAME' passing the
name of Nix-expression to build. If succeeded, Nix publishes the last build'
results under the ./result symlink.
$ nix build '.#vim-demo'
$ ./result/bin/vim-demo # Run the pre-configured demo instance of Vim
The list of Nix build targets includes:
litrepl-release - Litrepl script and Python liblitrepl-release-pypi - Litrepl script and Python libvim-litrepl-release - Vim with locally built litrepl pluginvim-litrepl-release-pypi - Vim with litrepl plugin built from PYPIvim-test - A minimalistic Vim with a single litrepl pluginvim-demo - Vim configured to use litrepl suitable for recording screencastsvim-plug - Vim configured to use litrepl via the Plug managershell-dev - The development shellshell-screencast - The shell for recording demonstrations, includes vim-demo.See Nix flakes manual for other Nix-related details.
The top-level Makefile encodes common development workflows:
[LitREPL-develop] $ make help
LitREPL is a macroprocessing Python library for Litrate programming and code execution
Build targets:
help: Print help
test: Run the test script (./sh/test.sh)
wheel: Build Python wheel (the DEFAULT target)
version: Print the version
upload: Upload wheel to Pypi.org (./_token.pypi is required)
Basic usage
Note: the below screencasts are outdated.
Using LitREPL in combination with the Vimtex plugin to edit Latex documents on the fly.
Asynchronous code execution
The following events should normally happen after users type the :LitEval1
command:
os.ps1/os.ps2 prompts of the Python interpreter
could break the session.Edititng:
Code execution:
Useful Vim plugins:
Useful tools:
FAQs
Litrepl evaluates code from LaTeX/Markdown verbatim sections.
We found that litrepl demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Socket’s threat research team has detected six malicious npm packages typosquatting popular libraries to insert SSH backdoors.
Security News
MITRE's 2024 CWE Top 25 highlights critical software vulnerabilities like XSS, SQL Injection, and CSRF, reflecting shifts due to a refined ranking methodology.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.