python-ripgrep

python-ripgrep

A Python wrapper for ripgrep, providing fast and efficient text searching capabilities.

Description

python-ripgrep is a Python package that wraps the functionality of ripgrep, a line-oriented search tool that recursively searches directories for a regex pattern. This package allows you to harness the power and speed of ripgrep directly from your Python code.

Features

• Fast text searching using ripgrep's algorithms
• Recursive directory searching
• Regular expression support
• Customizable search parameters

Installation

You can install python-ripgrep using pip:

pip install python-ripgrep

Usage

Here's a basic example of how to use python-ripgrep:

from python_ripgrep import search

# Perform a simple search, returning a
# list of string results grouped by file.
results = search(
    patterns=["pattern"],
    paths=["path/to/search"],
    globs=["*.py"],
)

# Process the results
for result in results:
    print(result)

API Reference

The main components of python-ripgrep are:

• search: The primary function for performing searches
• files: A function for listing files that would be searched (--files equivalent)
• PySortMode and PySortModeKind: Enums for specifying sort modes

For detailed API documentation, please refer to the source code comments.

Implementation Details

Direct Rust Integration

Unlike many other ripgrep bindings for Python, python-ripgrep doesn't shell out to the ripgrep command-line tool. Instead, it reimplements core ripgrep logic in Rust and provides a direct interface to Python. This approach offers several advantages:

• Performance: By avoiding the overhead of creating a new process and parsing stdout, this implementation can be more efficient, especially for large-scale searches or when called frequently.

• Fine-grained control: The library can expose more detailed control over the search process and return structured data directly to Python.

• Better integration: It allows for tighter integration with Python code, making it easier to incorporate into larger Python applications.

Current Limitations

As of now, the library implements a subset of ripgrep's functionality. The main search options currently supported are:

• patterns: The search patterns to use
• paths: The paths to search in
• globs: File patterns to include or exclude
• sort: Sort mode for search results
• max_count: Maximum number of matches to show

Implemented Flags

The following is a checklist of ripgrep flags that have been implemented in this Python wrapper:

• patterns: Search patterns
• paths: Paths to search (default: current directory)
• globs: File patterns to include or exclude (default: all non-ignored files)
• heading: (Optional) Whether to show file names above matching lines
• sort: (Optional) Sort mode for search results
• max_count: (Optional) Maximum number of matches to show per file
• after_context: (Optional) Number of lines to show after each match
• before_context: (Optional) Number of lines to show before each match
• separator_field_context: (Optional) Separator between fields in context lines
• separator_field_match: (Optional) Separator between fields in matching lines
• separator_context: (Optional) Separator between context lines

The following flags from ripgrep are not yet implemented in this wrapper:

• -C, --context: Show lines before and after each match
• --color: Controls when to use color in output
• -c, --count: Only show the count of matching lines
• --debug: Show debug messages
• --dfa-size-limit: Limit for regex DFA size
• -E, --encoding: Specify the text encoding of files to search
• -F, --fixed-strings: Treat patterns as literal strings
• -i, --ignore-case: Case insensitive search
• -v, --invert-match: Invert matching
• -n, --line-number: Show line numbers
• -x, --line-regexp: Only show matches surrounded by line boundaries
• -M, --max-columns: Don't print lines longer than this limit
• --mmap: Memory map searched files when possible
• -U, --multiline: Enable matching across multiple lines
• --no-ignore: Don't respect ignore files
• --no-unicode: Disable Unicode-aware search
• -0, --null: Print NUL byte after file names
• -o, --only-matching: Print only matched parts of a line
• --passthru: Print both matching and non-matching lines
• -P, --pcre2: Use the PCRE2 regex engine
• -p, --pretty: Alias for --color=always --heading -n
• -r, --replace: Replace matches with the given text
• -S, --smart-case: Smart case search
• -s, --case-sensitive: Case sensitive search
• --stats: Print statistics about the search
• -a, --text: Search binary files as if they were text
• -t, --type: Only search files matching TYPE
• -T, --type-not: Do not search files matching TYPE
• -u, --unrestricted: Reduce the level of "smart" searching
• -V, --version: Print version information
• -w, --word-regexp: Only show matches surrounded by word boundaries
• -z, --search-zip: Search in compressed files

Note that this list may not be exhaustive and some flags might have partial implementations or behave differently from the original ripgrep. Refer to the source code for the most up-to-date information on implemented features.

Extending Functionality

To add more ripgrep options to the library, you'll need to modify both the Rust and Python sides of the codebase:

• Update the PyArgs struct in src/ripgrep_core.rs to include the new option.
• Modify the pyargs_to_hiargs function in the same file to convert the new Python argument to the corresponding ripgrep argument.
• Update the Python wrapper code to expose the new option to Python users.

For example, to add a new option case_sensitive:

• Add to PyArgs:

  pub case_sensitive: Option<bool>,
  

• In pyargs_to_hiargs, add:

  if let Some(case_sensitive) = py_args.case_sensitive {
      low_args.case_sensitive = case_sensitive;
  }
  

• Update the Python wrapper to include the new option.

Remember to handle any necessary type conversions between Python and Rust in the pyargs_to_hiargs function.

Development

This project uses maturin for building the Python package from Rust code. To set up a development environment:

• Ensure you have Rust and Python installed
• Install maturin: pip install maturin
• Clone the repository
• Run maturin develop to build and install the package locally

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

[License information to be added]

Acknowledgements

This project is based on ripgrep by Andrew Gallant.

This project is built and maintained by the Exponent team.