github.com/arturoeanton/nflow-runtime
Workflow execution engine for nFlow. This project executes workflows created in the nFlow visual designer, providing a secure environment with resource limits and sandboxing.
go get github.com/arturoeanton/nflow-runtime
config.toml[database_nflow]
driver = "postgres"
dsn = "user=postgres dbname=nflow sslmode=disable"
[redis]
host = "localhost:6379"
password = ""
[vm_pool]
# VM pool for high performance
max_size = 200 # Maximum VMs in pool (increased for 4x performance)
preload_size = 100 # VMs preloaded at startup
# Resource limits (security)
max_memory_mb = 128 # Maximum memory per VM
max_execution_seconds = 30 # Maximum execution time
max_operations = 10000000 # Maximum JS operations
# Sandbox settings
enable_filesystem = false # Filesystem access
enable_network = false # Network access
enable_process = false # Process access
[tracker]
enabled = false # Execution tracking (performance impact)
verbose_logging = false # Detailed tracker logs
[monitor]
enabled = true # Monitoring endpoints
health_check_path = "/health"
metrics_path = "/metrics"
[debug]
enabled = false # Debug endpoints (development only)
auth_token = "" # Authentication token
allowed_ips = "" # Allowed IPs (e.g., "192.168.1.0/24")
[mail]
enabled = false
smtp_host = "smtp.gmail.com"
smtp_port = 587
[rate_limit]
enabled = false # IP-based rate limiting
ip_rate_limit = 100 # Requests per IP per window
ip_window_minutes = 1 # Time window in minutes
[security]
# Static JavaScript analysis
enable_static_analysis = false # Detect dangerous patterns before execution
block_on_high_severity = true # Block scripts with severe issues
# Sensitive data encryption
enable_encryption = false # Auto-encrypt sensitive data
encryption_key = "" # 32-byte key for AES-256
encrypt_sensitive_data = true # Detect and encrypt emails, SSN, API keys, etc.
# Log sanitization
enable_log_sanitization = false # Mask sensitive data in logs
log_masking_char = "*" # Character for masking
log_show_type = true # Show masked data type
# Normal mode
./nflow-runtime
# Verbose mode (detailed logging)
./nflow-runtime -v
Server will be available at http://localhost:8080
import (
"github.com/arturoeanton/nflow-runtime/engine"
"github.com/arturoeanton/nflow-runtime/process"
)
func main() {
// Initialize configuration
configRepo := engine.GetConfigRepository()
config := engine.ConfigWorkspace{
// ... configuration
}
configRepo.SetConfig(config)
// Initialize database
db, err := engine.GetDB()
if err != nil {
log.Fatal(err)
}
engine.InitializePlaybookRepository(db)
// Initialize process manager
process.InitializeRepository()
// Create Echo server
e := echo.New()
e.Any("/*", run)
e.Start(":8080")
}
Each VM has configurable limits to prevent DoS attacks:
JavaScript executes in a restricted environment:
eval() blockedFunction constructor blockedBefore execution, each script is analyzed to detect:
eval() or new Function()require('fs'))child_process)nflow-runtime/
├── engine/ # Main execution engine
│ ├── engine.go # Workflow execution logic
│ ├── vm_manager.go # VM pool for high performance
│ ├── vm_limits.go # Resource limit management
│ ├── vm_sandbox.go # Sandbox implementation
│ ├── js_context_wrapper.go # Echo context wrapper for JS
│ └── config_repository.go # Repository pattern for config
├── process/ # Process management
│ └── process_repository.go # Thread-safe repository
├── endpoints/ # API endpoints
│ ├── debug_endpoints.go # Debug endpoints
│ └── monitor_endpoints.go # Health & metrics
├── logger/ # Logging system
│ └── logger.go # Structured logger with levels
├── security/ # Security module
│ ├── analyzer/ # Static JavaScript analysis
│ ├── encryption/ # AES-256 encryption service
│ ├── interceptor/ # Sensitive data interceptor
│ ├── sanitizer/ # Log sanitizer
│ └── security_middleware.go # Unified middleware
├── syncsession/ # Optimized session management
├── plugins/ # System plugins
└── main.go # Server entry point
You can create your own node types:
type MyCustomStep struct{}
func (s *MyCustomStep) Run(
cc *model.Controller,
actor *model.Node,
c echo.Context,
vm *goja.Runtime,
connection_next string,
vars model.Vars,
currentProcess *process.Process,
payload goja.Value,
) (string, goja.Value, error) {
// Your implementation here
return nextNode, payload, nil
}
// Register the step
engine.RegisterStep("my-custom-step", &MyCustomStep{})
GET /health - System health statusGET /metrics - All metrics in Prometheus formatnflow_requests_total: Total HTTP requestsnflow_workflows_total: Total workflows executednflow_processes_active: Active processesnflow_db_connections_*: Database connection metricsnflow_go_memory_*: Memory usagenflow_cache_hits/misses: Cache statistics/debug/info: System information/debug/config: Current configuration/debug/processes: Active process list/debug/cache/stats: Cache statistics/debug/database/stats: Database metricsSee DEBUG_MONITORING.md for complete documentation.
nFlow Runtime includes IP-based rate limiting to protect against abuse:
See RATE_LIMITING.md for complete documentation.
nFlow Runtime has been optimized to handle heavy JavaScript workloads:
Errors are handled consistently:
See STATUS.md for more details.
See DEUDA.md for the complete technical debt list.
git checkout -b feature/AmazingFeature)git commit -m 'Add some AmazingFeature')git push origin feature/AmazingFeature)MIT - see LICENSE file for details.
FAQs
Unknown package
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Four npm packages disguised as cryptographic tools steal developer credentials and send them to attacker-controlled Telegram infrastructure.
Security News
Ruby maintainers from Bundler and rbenv teams are building rv to bring Python uv's speed and unified tooling approach to Ruby development.
Security News
Following last week’s supply chain attack, Nx published findings on the GitHub Actions exploit and moved npm publishing to Trusted Publishers.