ultralog

UltraLog - High-performance Logging System

UltraLog is a high-performance logging system that supports both local file logging and remote API logging.

Key Features

• Thread-Safe: Supports concurrent writes from multiple threads/clients with segmented locks
• Flexible Configuration: Extensive logging parameters via CLI or code
• Automatic Rotation: File rotation with size limits and backup counts
• Formatted Output: Consistent log formatting with timestamps
• Lifecycle Management: Proper resource cleanup on shutdown
• Large Message Handling: Optimized for messages up to 10KB with chunking
• Memory Efficient: Pre-allocated buffers and memory monitoring
• Priority Queueing: Smart prioritization of small messages

Performance

UltraLog is designed for high performance logging with minimal overhead. Below are benchmark results from testing 100,000 log messages:

Single Thread Performance (INFO level)

Message Size	Throughput (logs/sec)	Memory Usage
Small (50B)	425,000 (+33%)	1.5MB (-25%)
Medium (500B)	410,000 (+36%)	5.2MB (-31%)
Large (5KB)	250,000 (+33%)	45MB (-26%)
Very Large (10KB)	180,000	80MB

Multi-Thread Performance (10 threads)

Logger	Throughput (logs/sec)	Memory Usage
UltraLog	450,000 (+41%)	0.7MB (-28%)
Standard logging	50,087	0.00MB
Loguru	46,630	0.00MB

Key Performance Advantages

• Extreme Throughput: 9x faster than standard logging (450k logs/sec)
• Efficient Memory: 25-30% lower memory usage
• Large Message Support: Optimized handling of messages up to 10KB
• Smart Batching: Dynamic batch sizes based on message size
• Priority Handling: Small messages get processed faster
• Memory Monitoring: Real-time memory usage tracking

Installation

git clone https://github.com/birchkwok/ultralog.git
cd ultralog
pip install -e .

Basic Usage

Local Mode (Default)

from ultralog import UltraLog

# Basic initialization
logger = UltraLog(name="MyApp")

# Logging examples
logger.debug("Debug message")
logger.info("Application started")
logger.warning("Low disk space")
logger.error("Failed to connect")
logger.critical("System crash")

# Explicit cleanup (optional)
logger.close()

Remote Mode

from ultralog import UltraLog

# Remote configuration
logger = UltraLog(
    name="MyApp",
    server_url="http://your-server-ip:8000",
    auth_token="your_secret_token"
)

# Same logging interface
logger.info("Remote log message")

Log Formatting

UltraLog provides flexible log formatting similar to Python's built-in logging module.

Default Format

The default format follows loguru-style: %(asctime)s | %(levelname)-8s | %(module)s:%(func)s:%(line)s - %(message)s

Example output: 2025-04-19 07:50:22.139 | INFO | __main__:<module>:1 - Application started

Format Placeholders

Placeholder	Description
%(asctime)s	Timestamp (YYYY-MM-DD HH:MM:SS.microseconds)
%(levelname)s	Log level (DEBUG, INFO, WARNING, etc.)
%(module)s	Module name where log was called
%(func)s	Function name where log was called
%(line)s	Line number where log was called
%(message)s	The log message

Custom Formats

You can customize the format by passing a fmt parameter to the LogFormatter:

from ultralog import UltraLog

# Custom format logger
logger = UltraLog(
    name="MyApp",
    fmt="[%(levelname)s] %(name)s - %(asctime)s - %(message)s"
)

Available Placeholders

Placeholder	Description
%(asctime)s	Timestamp (YYYY-MM-DD HH:MM:SS.microseconds)
%(levelname)s	Log level (DEBUG, INFO, WARNING, etc.)
%(name)s	Logger name
%(message)s	The log message

Dynamic Format Changes

You can change the log format dynamically after initialization:

logger = UltraLog(name="MyApp")

# Initial format
logger.info("First message")  # Uses default format

# Change format
logger.set_format("%(levelname)s - %(message)s")
logger.info("Second message")  # Uses new simple format

# Change to detailed format
logger.set_format("[%(asctime)s] %(levelname)-8s %(name)s: %(message)s")
logger.info("Third message")  # Uses detailed format

Format Examples

• Simple format:

  fmt="%(levelname)s: %(message)s"
  

  Output: INFO: Application started

• Detailed format:

  fmt="[%(asctime)s] [%(levelname)-8s] %(name)-15s: %(message)s"
  

  Output: [2025-04-18 21:17:16.205283] [INFO ] MyApp : Application started

• JSON format:

  fmt='{"time": "%(asctime)s", "level": "%(levelname)s", "logger": "%(name)s", "msg": "%(message)s"}'
  

  Output: {"time": "2025-04-18 21:17:16.205283", "level": "INFO", "logger": "MyApp", "msg": "Application started"}

Server Configuration

Run the server with custom parameters:

python -m ultralog.server \
  --log-dir /var/log/myapp \
  --log-file app.log \
  --log-level DEBUG \
  --max-file-size 10485760 \
  --backup-count 5 \
  --console-output \
  --auth-token your_secure_token

Advanced Configuration

UltraLog Initialization Parameters

Parameter	Type	Default	Description
name	str	"UltraLogger"	Logger name prefix
fp	str	None	Local log file path
level	str	"INFO"	Minimum log level
truncate_file	bool	False	Truncate existing log file
with_time	bool	True	Include timestamps
max_file_size	int	10MB	Max file size before rotation
backup_count	int	5	Number of backup files
console_output	bool	False	Print to console
force_sync	bool	False	Force synchronous writes
enable_rotation	bool	True	Enable log rotation
file_buffer_size	int	256KB	File write buffer size
batch_size	int	None	Remote batch size
flush_interval	float	None	Remote flush interval
server_url	str	None	Remote server URL
auth_token	str	None	Remote auth token

Development

Running Tests

pytest tests/

Building Package

python -m build

API Documentation

Interactive API docs available at: http://localhost:8000/docs when server is running

Best Practices

• For production:

  • Use proper log rotation settings
  • Set appropriate log levels
  • Use secure authentication tokens
  • Monitor log file sizes

• For remote logging:

  • Implement retry logic in your application
  • Consider batch sizes for high throughput
  • Monitor network connectivity

• General:

  • Use meaningful logger names
  • Include context in log messages
  • Regularly review log retention policy