Insyt Secure
A proprietary service for securely interacting with databases, APIs, and other services within a secure environment. Visit https://insyt.co for more information.
Quick Start
Installation
pip install insyt-secure
Installing on a Fresh Ubuntu Instance
Follow these steps to set up insyt-secure on a fresh Ubuntu installation:
sudo apt update
sudo apt install python3-pip
sudo apt install python3-venv python3-full
mkdir myproject
cd myproject
python3 -m venv venv
source venv/bin/activate
pip install insyt-secure[mysql,postgres]
pip install python-dotenv
Notes:
- To run the service, use the following command:
insyt-secure --projects "your-project-id-123:your-api-key-xyz"
or run the service in the background:
nohup insyt-secure --projects "your-project-id-123:your-api-key-xyz" &
- Remember to activate the virtual environment each time you want to use this package in a new terminal session.
source venv/bin/activate
- To deactivate the virtual environment when you're done:
deactivate
- To stop the service, use the following command:
pkill -f insyt-secure
- To check the status of the service, use the following command:
ps aux | grep insyt-secure
Advanced Installation Options
Insyt Secure provides flexible installation options to include only the dependencies you need:
pip install "insyt-secure[postgres]"
pip install "insyt-secure[mongodb]"
pip install "insyt-secure[postgres,mongodb,redis]"
pip install "insyt-secure[pinecone]"
pip install "insyt-secure[aws]"
pip install "insyt-secure[kafka]"
pip install "insyt-secure[all]"
Available extension categories:
- SQL databases:
postgres
, mysql
, mssql
, oracle
, clickhouse
, snowflake
, duckdb
- NoSQL databases:
mongodb
, redis
, cassandra
, neo4j
, elasticsearch
, couchdb
- Vector databases:
pinecone
, qdrant
, milvus
, weaviate
, chroma
, faiss
- Cloud services:
aws
, azure
- Messaging systems:
kafka
, pulsar
, rabbitmq
Broader categories are also available: rdbms
, nosql
, vector
, cloud
, messaging
Basic Usage
insyt-secure --projects "your-project-id-123:your-api-key-xyz"
Multi-Project Support
Insyt Secure supports managing one or more projects simultaneously:
insyt-secure --projects "project-id-1:api-key-1,project-id-2:api-key-2"
Each project connection is managed independently, allowing for simultaneous interactions with multiple Insyt projects, each with its own topic subscriptions and credential management.
Getting Help
insyt-secure --help
The help command provides detailed information about all parameters, their defaults, and usage examples directly in your terminal.
Advanced Options
insyt-secure \
--projects "your-project-id-123:your-api-key-xyz" \
--max-workers 10 \
--execution-timeout 60 \
--allowed-ips "192.168.1.1,10.0.0.1:3456"
Logging Options
By default, logs are user-friendly and redact sensitive information. You can customize logging behavior:
insyt-secure --projects "your-project-id:your-api-key" --debug
insyt-secure --projects "your-project-id:your-api-key" --verbose
insyt-secure --projects "your-project-id:your-api-key" --json-logs
insyt-secure --projects "your-project-id:your-api-key" --show-sensitive
You can also control the log level via environment variables:
INSYT_LOG_LEVEL=DEBUG insyt-secure --projects "your-project-id:your-api-key"
INSYT_PROJECTS="project-id-1:api-key-1,project-id-2:api-key-2" insyt-secure
Cross-Platform Compatibility
Insyt Secure is designed to work seamlessly on all major platforms:
- Windows: Fully supported natively, no additional configuration needed
- macOS: Fully supported
- Linux: Fully supported
The service uses paho-mqtt with a platform-agnostic implementation to ensure consistent behavior across all operating systems.
Command Line Parameters
--projects | Comma-separated list of project-id:api-key pairs for managing one or more projects | - |
--max-workers | Maximum number of concurrent code executions per project | 5 |
--execution-timeout | Default maximum execution time in seconds per code snippet | 30 |
--allowed-ips | Comma-separated list of allowed IPs/hostnames | All allowed |
--verbose | Enable more verbose logs | False |
--debug | Enable debug level logging | False |
--json-logs | Output logs in JSON format | False |
--show-sensitive | Show sensitive information in logs | False |
Credentials Management
This service automatically retrieves and manages connection credentials:
- When the service starts, it gets credentials from the Insyt API for each configured project
- If the connection drops or credentials expire, it automatically requests new credentials
- Each project's credentials are managed independently
Project Management
Insyt Secure supports managing one or more projects simultaneously:
Configuration Options
You can configure projects in two ways:
-
Command line parameter:
insyt-secure --projects "project-id-1:api-key-1,project-id-2:api-key-2"
-
Environment variable:
export INSYT_PROJECTS="project-id-1:api-key-1,project-id-2:api-key-2"
insyt-secure
Benefits of Multi-Project Mode
- Resource Efficiency: All projects share a single process but maintain independent connections
- Simplified Management: Manage multiple projects from a single service instance
- Independent Reconnection: Each project can reconnect independently if its connection fails
- Shared DNS Cache: All projects benefit from the DNS cache for improved resilience
API Usage for Multiple Projects
You can also use the project management programmatically:
from insyt_secure.project_manager import ProjectManager
import asyncio
async def run_projects():
manager = ProjectManager()
await manager.add_project("project-id-1", "api-key-1", "https://api.account.insyt.co/")
await manager.add_project("project-id-2", "api-key-2", "https://api.account.insyt.co/")
await manager.start()
while True:
await asyncio.sleep(60)
if __name__ == "__main__":
asyncio.run(run_projects())
Use Cases
Data Processing Microservice
Perfect for running data transformation code that connects to various data sources:
insyt-secure --projects "your-project-id:your-api-key" --max-workers 15
Secure Environment for Code Testing
Create a sandboxed environment with restricted network access:
insyt-secure --projects "your-project-id:your-api-key" \
--allowed-ips "10.0.0.1,192.168.1.100" --execution-timeout 20
Multi-Project Service
Run a single service that executes code for multiple projects:
insyt-secure --projects "PRJ-A235466:api-key-1,PRJ-A235477:api-key-2,ai-models:api-key-3" \
--max-workers 10 --execution-timeout 60
Containerized Deployment
docker run -d --name insyt-secure \
insyt-secure insyt-secure \
--projects "your-project-id:your-api-key"
System Requirements and Dependencies
Insyt Secure is designed with a modular dependency structure to minimize installation size and resource usage. Below is a breakdown of what's included in each installation option:
Core Dependencies (included in base install)
The base installation includes:
- HTTP client capabilities via
httpx
- MQTT connectivity via
paho-mqtt
- Basic data science libraries: NumPy, Pandas, SciPy
- JSON and date handling utilities
Optional Dependencies
SQL Database Connectors
postgres
: High-performance PostgreSQL client (asyncpg)
mysql
: MySQL client libraries
mssql
: Microsoft SQL Server connectivity
oracle
: Oracle database connectivity
clickhouse
: ClickHouse analytics database client
snowflake
: Snowflake data warehouse client
duckdb
: Embedded analytical database
NoSQL Database Connectors
mongodb
: MongoDB client with async support
redis
: Redis client
cassandra
: Apache Cassandra and ScyllaDB clients
neo4j
: Neo4j graph database client
elasticsearch
: Elasticsearch search engine client
couchdb
: CouchDB document database client
Vector Database Connectors
pinecone
: Pinecone vector database client
qdrant
: Qdrant vector search engine client
milvus
: Milvus vector database client
weaviate
: Weaviate vector search engine
chroma
: ChromaDB for AI embeddings
faiss
: Facebook AI Similarity Search
Cloud Services
aws
: AWS SDK (boto3) with S3, Dynamo, etc.
azure
: Azure clients for Cosmos DB, Blob Storage, etc.
Messaging Systems
kafka
: Apache Kafka client
pulsar
: Apache Pulsar client
rabbitmq
: RabbitMQ client
Performance Considerations
The base installation already includes the core ML libraries (numpy, pandas, etc.). If you're installing on a resource-constrained environment, consider using only the specific connector extensions you need rather than the broader categories.
For production deployments, we recommend specifying exact dependencies rather than using broader categories:
pip install "insyt-secure[postgres,redis]"
pip install "insyt-secure[rdbms,nosql]"
Platform-Specific Installation
Insyt Secure is designed to work on all major platforms without modification:
Windows
pip install insyt-secure
insyt-secure --projects "your-project-id-123:your-api-key-xyz"
macOS/Linux
pip install insyt-secure
insyt-secure --projects "your-project-id-123:your-api-key-xyz"
Docker
echo 'FROM python:3.10-slim
RUN pip install insyt-secure
ENTRYPOINT ["insyt-secure"]' > Dockerfile
docker build -t insyt-secure .
docker run insyt-secure --projects "your-project-id-123:your-api-key-xyz"
Platform-Specific Considerations
- Windows: Works natively without WSL, using a cross-platform MQTT implementation
- MacOS/Linux: All features fully supported
- Docker: Ideal for deployment in containerized environments