isoai

Iso AI

Unlock the Value of GenAI with Iso!

isoai Package Documentation

Table of Contents

1. Overview
2. Installation
3. Modules
   • isobits
     • GPT_bitnet
     • torch_bitnet
     • replace_linear_layers
     • LLama_bitnet
     • MoE_bitnet
   • isodeploy
     • containerize
     • oscontainerization
   • isotrace
4. Usage Examples
   • Replacing Linear Layers with TorchBitNet
   • Using BitNetLLAMA
   • Using MoE Transformer
   • Containerizing Your Model
   • Containerizing with OS Detection
   • Tracing Variables
5. Contributing
6. Support
7. License
8. Acknowledgments

Overview

The isoai package is designed to facilitate the integration, transformation, deployment, and tracing of AI models. It provides a suite of tools that make it easy to work with different types of neural network architectures and deploy them in a containerized environment.

Installation

To install the isoai package, use the following command:

pip install isoai

Modules

isobits

GPT_bitnet

The GPT_bitnet module contains implementations of transformers using the BitNet architecture.

• Transformer: Main transformer class.
• TransformerBlock: Individual transformer block class.

torch_bitnet

The torch_bitnet module provides implementations for RMSNorm and TorchBitNet layers.

• RMSNorm: Root Mean Square Layer Normalization.
• TorchBitNet: Implementation of BitNet layers using PyTorch.

replace_linear_layers

The replace_linear_layers module provides functionality to replace traditional linear layers in a model with TorchBitNet layers.

• replace_linears_with_torchbitnet: Function to replace linear layers with TorchBitNet layers.

LLama_bitnet

The LLama_bitnet module includes the BitNetLLAMA model, which leverages the BitNet architecture.

• BitNetLLAMA: Implementation of the LLAMA model using BitNet.

MoE_bitnet

The MoE_bitnet module implements the Mixture of Experts (MoE) architecture using BitNet.

• MoETransformer: Transformer model with MoE.
• MoETransformerBlock: Individual MoE transformer block.

isodeploy

containerize

The containerize module helps in containerizing models into Docker containers.

• Containerize: Main class for containerizing models.

oscontainerization

The oscontainerization module extends containerization functionality with automatic OS detection.

• OSContainerize: Class for containerizing models with OS detection.

isotrace

The isotrace module provides tools for tracing and logging variables within the code.

• Autotrace: Class for automatic tracing and logging of variables.

Usage Examples

Replacing Linear Layers with TorchBitNet

import torch
from isoai.isobits.GPT_bitnet import Transformer
from isoai.isobits.replace_linear_layers import replace_linears_with_torchbitnet

class ModelArgs:
    def __init__(self):
        self.vocab_size = 30522
        self.dim = 768
        self.n_heads = 12
        self.n_kv_heads = 12
        self.max_seq_len = 512
        self.norm_eps = 1e-5
        self.multiple_of = 64
        self.ffn_dim_multiplier = 4
        self.n_layers = 12
        self.max_batch_size = 32

args = ModelArgs()
tokens = torch.randint(0, args.vocab_size, (2, args.max_seq_len))
transformer = Transformer(args)
output = transformer(tokens, start_pos=0)
print("Original Transformer Description: ", transformer)
replace_linears_with_torchbitnet(transformer, norm_dim=10)
print("Bitnet Transformer Description: ", transformer)

Using BitNetLLAMA

import torch
from isoai.isobits.LLama_bitnet import BitNetLLAMA

class ModelArgs:
    def __init__(self):
        self.vocab_size = 30522
        self.dim = 768
        self.n_heads = 12
        self.n_kv_heads = 12
        self.max_seq_len = 512
        self.norm_eps = 1e-5
        self.multiple_of = 64
        self.ffn_dim_multiplier = 4
        self.max_batch_size = 32
        self.n_layers = 12

args = ModelArgs()
tokens = torch.randint(0, args.vocab_size, (2, args.max_seq_len))
bitnet_llama = BitNetLLAMA(args)
output = bitnet_llama(tokens, start_pos=0)
print(output.shape)

Using MoE Transformer

import torch
from isoai.isobits.MoE_bitnet import MoETransformer

class ModelArgs:
    def __init__(self):
        self.vocab_size = 30522
        self.dim = 768
        self.n_heads = 12
        self.n_kv_heads = 12
        self.max_seq_len = 512
        self.norm_eps = 1e-5
        self.multiple_of = 64
        self.ffn_dim_multiplier = 4
        self.max_batch_size = 32
        self.n_layers = 12
        self.num_experts = 4  # Number of experts in MoE layers

args = ModelArgs()
tokens = torch.randint(0, args.vocab_size, (2, args.max_seq_len))
moe_transformer = MoETransformer(args)
output = moe_transformer(tokens, start_pos=0)
print(output.shape)

Containerizing Your Model

from isoai.isodeploy.containerize import Containerize

model_path = "isoai"
output_path = "Dockerfile"

containerize = Containerize(model_path)
containerize.run(output_path)

Containerizing with OS Detection

from isoai.isodeploy.oscontainerization import OSContainerize

model_path = "isoai"
output_path = "Dockerfile"

containerize = OSContainerize(model_path)
containerize.run(output_path)

Tracing Variables

from isoai.isotrace.autotrace import Autotrace

search_path = "isoai"
output_file = "output.json"

autotrace = Autotrace(search_path)
autotrace.run(output_file)