rhoshift

RHOAI Tool Kit

A comprehensive toolkit for managing and upgrading Red Hat OpenShift AI (RHOAI) installations with parallel installation support.

📋 Table of Contents

• Features
• Project Structure
• Installation
• Usage
• Logging
• Configuration
• Development
• Troubleshooting
• Contributing

✨ Features

• Install single or multiple OpenShift operators
• Parallel installation for faster deployments
• Configurable timeouts and retries
• Comprehensive logging system
• Supports:
  • Serverless Operator
  • Service Mesh Operator
  • Authorino Operator
  • RHOAI Operator

📁 Project Structure

rhoshift/
├── rhoshift/              # Main package directory
│   ├── __init__.py
│   ├── main.py           # CLI entry point
│   ├── cli/              # Command-line interface
│   │   ├── __init__.py
│   │   ├── args.py      # Argument parsing
│   │   └── commands.py  # Command implementations
│   ├── logger/          # Logging utilities
│   │   ├── __init__.py
│   │   └── logger.py    # Logging configuration
│   └── utils/           # Core utilities
│       ├── __init__.py
│       ├── constants.py # Constants and configurations
│       ├── operator.py  # Operator management
│       └── utils.py     # Utility functions
├── run_upgrade_matrix.sh  # Upgrade matrix execution script
├── upgrade_matrix_usage.md # Upgrade matrix documentation
├── pyproject.toml        # Project dependencies and configuration
└── README.md            # This document

📋 Components

Core Components

• CLI: Command-line interface for operator management
• Logger: Logging configuration and utilities (logs to /tmp/rhoshift.log)
• Utils: Core utilities and operator management logic

RHOAI Components

• RHOAI Upgrade Matrix: Utilities for testing RHOAI upgrades
• Upgrade Matrix Scripts: Execution and documentation for upgrade testing

Maintenance Scripts

• Cleanup Scripts: Utilities for cleaning up operator installations
• Worker Node Scripts: Utilities for managing worker node configurations

🚀 Installation

• Clone the repository:

git clone https://github.com/mwaykole/O.git
cd O

• Install dependencies:

pip install -e .

• Verify installation:

rhoshift --help

💻 Usage

Basic Commands

# Install single operator
rhoshift --serverless

# Install multiple operators
rhoshift --serverless --servicemesh

# Install RHOAI with raw configuration
rhoshift --rhoai --rhoai-channel=<channel> --rhoai-image=<image> --raw=True

# Install RHOAI with Serverless configuration
rhoshift --rhoai --rhoai-channel=<channel> --rhoai-image=<image> --raw=False --all

# Install all operators
rhoshift --all

# Create DSC and DSCI with RHOAI operator installation
rhoshift --rhoai --deploy-rhoai-resources

# Clean up all operators
rhoshift --cleanup

Advanced Options

# Custom oc binary path
rhoshift --serverless --oc-binary /path/to/oc

# Custom timeout (seconds)
rhoshift --all --timeout 900

# Verbose output
rhoshift --all --verbose

Upgrade Matrix Testing

To run the upgrade matrix tests, you can use either method:

• Using the shell script:

./run_upgrade_matrix.sh [options] <current_version> <current_channel> <new_version> <new_channel>

• Using the Python command:

run-upgrade-matrix [options] <current_version> <current_channel> <new_version> <new_channel>

Options:

• -s, --scenario: Run specific scenario(s) (serverless, rawdeployment, serverless,rawdeployment)
• --skip-cleanup: Skip cleanup before each scenario
• --from-image: Custom source image path
• --to-image: Custom target image path

Example:

# Using shell script
./run_upgrade_matrix.sh -s serverless -s rawdeployment 2.10 stable 2.12 stable

# Using Python command
run-upgrade-matrix -s serverless -s rawdeployment 2.10 stable 2.12 stable

📝 Logging

The toolkit uses a comprehensive logging system:

• Logs are stored in /tmp/rhoshift.log
• Console output shows INFO level and above
• File logging captures DEBUG level and above
• Automatic log rotation (10MB max size, 5 backup files)
• Colored output in supported terminals

To view logs:

tail -f /tmp/rhoshift.log

🔧 Configuration

Environment Variables

• LOG_FILE_LEVEL: Set file logging level (default: DEBUG)
• LOG_CONSOLE_LEVEL: Set console logging level (default: INFO)

Command Options

• --oc-binary: Path to oc CLI (default: oc)
• --retries: Max retry attempts (default: 3)
• --retry-delay: Delay between retries in seconds (default: 10)
• --timeout: Command timeout in seconds (default: 300)

🛠️ Development

Prerequisites

• Python 3.8 or higher
• OpenShift CLI (oc)
• Access to an OpenShift cluster

Running Tests

pytest tests/

🔍 Troubleshooting

Common Issues

• Operator Installation Fails

  • Check cluster access: oc whoami
  • Verify operator catalog: oc get catalogsource
  • Check logs: tail -f /tmp/rhoshift.log

• Permission Issues

  • Ensure you have cluster-admin privileges
  • Check namespace permissions

• Timeout Errors

  • Increase timeout: --timeout 900
  • Check cluster resources

🤝 Contributing

• Fork the repository
• Create a feature branch
• Commit your changes
• Push to the branch
• Create a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.