src.techknowlogick.com/nomad-driver-tart

Nomad Tart Driver Plugin

A task driver plugin for Nomad that allows scheduling and running Tart virtual machines on macOS hosts.

This driver enables Nomad to manage macOS virtual machines using Tart, providing native virtualization capabilities for Apple Silicon Macs.

Requirements

• Go v1.24 or later (to compile the plugin)
• Nomad v1.9 or later to run the plugin
• Tart v2 or later installed on the Nomad client nodes
• macOS host (Apple Silicon) v13 or later to run Tart

Building the Tart Driver Plugin

# Clone repository
$ git clone [e-mail]:techknowlogick/nomad-driver-tart.git
$ cd nomad-driver-tart

# Build the plugin
$ make build

# For testing or development, you can run Nomad with the plugin directory pointing to your build
$ nomad agent -dev -config=./example/agent.hcl -plugin-dir=$(pwd)

Deploying the Tart Driver Plugin in Nomad

To use the Tart driver plugin with Nomad, you need to:

• Place the compiled plugin binary in Nomad's plugin directory
• Configure Nomad to recognize the plugin
• Create and run a job that uses the Tart driver

Client Configuration

In your Nomad client configuration, you can specify the Tart driver plugin with optional settings:

plugin "tart" {
  config {
    tart_binary = "/path/to/tart" # Optional: specify the path to the tart binary
  }
}

Note about tart_binary: This configuration option allows you to specify a custom path to the Tart binary. This is useful if:

• You have multiple versions of Tart installed on your system
• Tart is installed in a non-standard location
• You're running multiple Nomad clients with different Tart configurations

If not specified, the plugin will use tart and rely on it being available in the system PATH.

Running a Sample VM

In another terminal, run the example job:

$ nomad run ./example/tart.nomad
$ nomad logs <ALLOCATION ID>

Job Configuration

Here's a simple example of how to configure a Nomad job to use the Tart driver:

job "tart-demo" {
  datacenters = ["dc1"]
  type        = "service"

  group "demo" {
    task "vm" {
      driver = "tart"

      config {
        image     = "ghcr.io/cirruslabs/macos-sequoia-base:latest"  # Name of the Tart VM image
        cpu       = 2                 # Number of virtual CPUs
        memory    = 4096              # Memory allocation in MB
        disk_size = 50                # Disk size (in GBs)
        headless  = true              # Run VM without UI
        command   = "echo 'Hello from Tart VM'"  # Command to run inside the VM on first start
        suspendable = false            # Enable VM suspension support

        # Attach additional disks
        disk_mount {
          path    = "/path/to/disk.img"
          options = "ro"              # Read-only mode
        }

        # Share directories with the VM
        dir_share {
          name    = "source"          # Optional name for the mount point
          path    = "/path/to/source"
          options = "ro,tag=src"      # Read-only mode with custom mount tag
        }
      }
    }
  }
}

Task Configuration Parameters

The Tart driver supports the following configuration parameters:

Parameter	Required	Description
image	Yes	Tart VM image name or path
cpu	No	Number of virtual CPUs (default: 2)
memory	No	Memory allocation in MB (default: 4096)
disk_size	No	Disk size in GB (default: 50)
command	No	Command to run inside the VM on first start
headless	No	Run VM without UI (default: true)
suspendable	No	Disables audio and entropy devices and switches to only Mac-specific input devices. Useful for running a VM that can be suspended via "tart suspend" (default: false)
nested	No	Enable nested virtualization if possible (default: false)

SSH Configuration

Parameter	Required	Description
ssh_user	No	SSH username for task communication (default: "admin"). Must match the username configured in the VM
ssh_password	No	SSH password for authentication (default: "admin"). Must match the password configured in the VM
ssh_port	No	SSH port (default: 22)

Security Note: The default images from CirrusLabs (like ghcr.io/cirruslabs/macos-sequoia-base:latest) use insecure default credentials (admin/admin). For production use, you should create custom images with secure credentials or use SSH keys for authentication.

Additional Disk Options

disk_mount {
  path    = "/path/to/disk.img"    # Path to disk image or block device
  options = "ro,sync=none"         # Options for the disk attachment
}

The options parameter supports:

• ro - Mount the disk in read-only mode
• sync=none - Disable data synchronization with storage
• sync=fsync - Enable data synchronization without ensuring it was written
• sync=full - Enable full data synchronization with storage

Directory Sharing Options

dir_share {
  name    = "sharename"            # Optional name for the mount point
  path    = "/path/to/directory"   # Host path to share
  options = "ro,tag=customtag"     # Options for directory sharing
}

The options parameter supports:

• ro - Mount the directory in read-only mode
• tag=<TAG> - Custom mount tag (default: "com.apple.virtio-fs.automount")

Root Disk Options

Parameter	Required	Description
root_disk_options	No	Options for the root disk (e.g., "ro", "caching=cached,sync=none")

Root disk options support the same parameters as disk attachments, plus:

• caching=automatic|cached|uncached - Control disk caching behavior

Networking Options

Parameter	Required	Description
net_bridged	No	Interface name for bridged networking instead of shared (NAT) networking
net_softnet	No	Use software networking provided by Softnet instead of shared (NAT) networking (default: false)
net_softnet_allow	No	Comma-separated list of CIDRs to allow traffic to when using Softnet isolation
net_softnet_expose	No	Comma-separated list of TCP ports to expose (e.g., "2222:22,8080:80")
net_host	No	Restrict network access to the host-only network (default: false)

Acknowledgements

A big thank you to the following existing driver codebases:

• The Nomad Driver Skeleton for providing the layout, and placeholder code of this driver (MPL licensed code)
• Roblox's nomad-driver-containerd for the e2e tests (ASL2 licensed code)

Testing

The Tart driver plugin includes several types of tests:

Unit Tests

Run unit tests with:

$ go test ./tart -short

These tests do not require Tart to be installed and cover basic driver functionality.

Integration Tests

Integration tests require Tart to be installed and test actual VM operations. Run them with:

$ NOMAD_TEST_TART_INTEGRATION=1 go test ./tart -v

You can specify a custom Tart VM image to use for testing:

$ NOMAD_TEST_TART_IMAGE="ghcr.io/cirruslabs/macos-sonoma-base:latest" NOMAD_TEST_TART_INTEGRATION=1 go test ./tart -v

Note When running tests for the first time or pulling fresh images, the download process can take a significant amount of time (potentially longer than 10 minutes depending on your connection). Tests may timeout and fail if the pull takes too long. It's recommended to pre-pull images in a separate process on your servers before running tests to avoid these timeouts:

# Pre-pull recommended images
$ tart pull ghcr.io/cirruslabs/macos-sequoia-base:latest
$ tart pull ghcr.io/cirruslabs/macos-sonoma-base:latest

End-to-End Tests

The e2e directory contains scripts for testing the driver with a live Nomad agent:

$ ./e2e/run_tests.sh

These tests require both Nomad and Tart to be installed, and be run on macOS.

Licensing and Legal

Important Disclaimer: Tart requires a license for use in commercial environments. Please review their licensing information for those details. Additionally, running macOS VMs may have specific End User License Agreement (EULA) requirements from Apple. Users are responsible for ensuring compliance with all applicable licenses and agreements.

As for the license of this codebase, I'm unsure of which license it should be, since the codebase includes a mixture of MPL, ASL2, and the specific code for interacting with tart. I'm sure there is probably a compatible license that I could apply to my changes, but I'm not a lawyer. If you are one, and want to give me advice, I'd be happy to listen. Until then, since the "default" license applies to my code, it's probably not able to be used in any manner, or to even to use the tart specific code elsewhere.

If you want orchestration of macOS VMs, I'd suggest using Orchard, an orchestration tool built by the same developers behind Tart.