capacitor-apple-intelligence

capacitor-apple-intelligence

A production-ready Capacitor v8 plugin that exposes Apple Intelligence with schema-constrained JSON generation for structured AI responses.

Features

• On-device AI - Uses Apple's Foundation Models framework for local, private inference
• Schema-constrained output - Guarantees JSON output matching your provided schema
• Automatic retry - Retries generation once if validation fails
• Runtime validation - Validates output against JSON schema before returning
• Privacy-first - No network calls, all processing happens on-device
• iOS only - Designed specifically for Apple Intelligence on iOS 26+

Requirements

• iOS 26+ (Apple Intelligence requires iOS 26 or later)
• Capacitor 8.0+
• Xcode 26+
• Apple Silicon device (iPhone 15 Pro or later, or M-series iPads)

Note: Apple Intelligence must be enabled on the device for this plugin to work.

Example App

A fully functional example app is included in the example/ directory. It demonstrates all plugin methods with a clean UI:

• JSON generation with schema validation
• Plain text generation
• Language-specific text generation
• Availability checking

See the example README for setup instructions.

Installation

npm install capacitor-apple-intelligence
npx cap sync

Usage

Basic Example

First, check if Apple Intelligence is available:

const availability = await AppleIntelligence.checkAvailability();
if (!availability.available) {
  console.error("Apple Intelligence not available");
  return;
}

Then generate JSON:

import { AppleIntelligence } from 'capacitor-apple-intelligence';

const result = await AppleIntelligence.generate({
  messages: [
    { role: "user", content: "List 3 popular programming languages" }
  ],
  response_format: {
    type: "json_schema",
    schema: {
      type: "array",
      items: {
        type: "object",
        properties: {
          name: { type: "string" },
          paradigm: { type: "string" },
          yearCreated: { type: "number" }
        },
        required: ["name", "paradigm", "yearCreated"]
      }
    }
  }
});

if (result.success) {
  console.log(result.data);
  // [
  //   { name: "Python", paradigm: "Multi-paradigm", yearCreated: 1991 },
  //   { name: "JavaScript", paradigm: "Event-driven", yearCreated: 1995 },
  //   { name: "Rust", paradigm: "Systems", yearCreated: 2010 }
  // ]
} else {
  console.error(result.error?.code, result.error?.message);
}

With System Prompt

const result = await AppleIntelligence.generate({
  messages: [
    { 
      role: "system", 
      content: "You are a helpful assistant that organizes tasks." 
    },
    { 
      role: "user", 
      content: "Create a list of tasks for planning a project" 
    }
  ],
  response_format: {
    type: "json_schema",
    schema: {
      type: "array",
      items: {
        type: "object",
        properties: {
          title: { type: "string" },
          description: { type: "string" },
          priority: { type: "string" },
          estimatedHours: { type: "number" }
        },
        required: ["title", "priority"]
      }
    }
  }
});

Nested Object Schema

const result = await AppleIntelligence.generate({
  messages: [
    { role: "user", content: "Create a sample user profile" }
  ],
  response_format: {
    type: "json_schema",
    schema: {
      type: "object",
      properties: {
        user: {
          type: "object",
          properties: {
            name: { type: "string" },
            email: { type: "string" },
            age: { type: "number" }
          },
          required: ["name", "email"]
        },
        preferences: {
          type: "object",
          properties: {
            theme: { type: "string" },
            notifications: { type: "boolean" }
          }
        }
      },
      required: ["user"]
    }
  }
});

Plain Text Generation

Generate plain text responses without JSON schema constraints:

const result = await AppleIntelligence.generateText({
  messages: [
    { role: "system", content: "You are a creative writing assistant." },
    { role: "user", content: "Write a short poem about the ocean" }
  ]
});

if (result.success) {
  console.log(result.content);
  // "Waves crash upon the shore,
  //  Endless depths forevermore..."
}

Text Generation with Language

Generate text in a specific language using either language codes or full names:

const result = await AppleIntelligence.generateTextWithLanguage({
  messages: [
    { role: "user", content: "Describe a beautiful sunset" }
  ],
  language: "es"  // or "Spanish"
});

if (result.success) {
  console.log(result.content);
  // "El atardecer pinta el cielo con tonos dorados y rosados..."
}
if (result.success) {
  console.log(result.content);
  // "El atardecer pinta el cielo con tonos dorados y rosados..."
}

Image Generation

Generate images from text prompts:

const result = await AppleIntelligence.generateImage({
  prompt: "A futuristic city with flying cars",
  style: "animation",       // Optional: "animation" (default), "illustration", "sketch"
  count: 1,                 // Optional: Defaults to 1
  sourceImage: base64String // Optional: Required for face-based generation
});

if (result.success && result.images) {
  result.images.forEach(base64Image => {
    // Display image
    const img = document.createElement('img');
    img.src = `data:image/png;base64,${base64Image}`;
    document.body.appendChild(img);
  });
}

Note: When your prompt involves generating images of people, you must provide a sourceImage containing a clearly visible face. This is a requirement of Apple's Image Playground API. The source image should be passed as a base64-encoded string.

Supported language codes: en, es, fr, de, ja, zh, it, pt, ru, ar, ko

API

checkAvailability()

Check if Apple Intelligence is available on the current device. Call this first before using other methods.

Response

Property	Type	Description
available	boolean	Whether Apple Intelligence is available
error	GenerateError	Error details (if unavailable)

Example

const result = await AppleIntelligence.checkAvailability();

if (result.available) {
  console.log('Apple Intelligence is ready!');
} else {
  console.log('Not available:', result.error?.message);
  // "Apple Intelligence requires iOS 26 or later..."
}

generate(request)

Generate structured JSON output using Apple Intelligence.

Request

Property	Type	Description
messages	Message[]	Array of conversation messages
response_format	ResponseFormat	Schema specification for the output

Message

Property	Type	Description
role	"system" | "user"	The role of the message sender
content	string	The text content of the message

ResponseFormat

Property	Type	Description
type	"json_schema"	Must be "json_schema"
schema	object	JSON Schema specification

Response

Property	Type	Description
success	boolean	Whether generation succeeded
data	any	The parsed JSON data (on success)
error	GenerateError	Error details (on failure)

Error Codes

Code	Description
UNAVAILABLE	iOS < 26 or Apple Intelligence not available
INVALID_JSON	Model output was not valid JSON
SCHEMA_MISMATCH	JSON valid but doesn't match schema
NATIVE_ERROR	Other Swift/Foundation Models errors

generateText(request)

Generate plain text output using Apple Intelligence.

Request

Property	Type	Description
messages	Message[]	Array of conversation messages

Response

Property	Type	Description
success	boolean	Whether generation succeeded
content	string	The generated text (on success)
error	GenerateError	Error details (on failure)

generateTextWithLanguage(request)

Generate plain text output in a specific language using Apple Intelligence.

Request

Property	Type	Description
messages	Message[]	Array of conversation messages
language	string	Target language - supports codes ("en", "es", "de") or full names ("English", "Spanish", "German")

Response

Property	Type	Description
success	boolean	Whether generation succeeded
content	string	The generated text (on success)
error	GenerateError	Error details (on failure)

How It Works

• Schema Injection: The plugin injects your JSON schema into the system prompt with strict instructions
• Generation: Uses Apple's Foundation Models framework to generate a response
• Parsing: Parses the raw text output as JSON
• Validation: Validates the JSON against your provided schema
• Retry: If validation fails, retries once with a corrective prompt
• Return: Returns structured success/error response

Supported Schema Types

• object - With properties and required fields
• array - With items schema and optional minItems/maxItems
• string
• number / integer
• boolean
• null

Error Handling

The plugin always returns a structured response, never throws:

const result = await AppleIntelligence.generate({...});

if (!result.success) {
  switch (result.error?.code) {
    case 'UNAVAILABLE':
      // Show fallback UI or use alternative
      break;
    case 'INVALID_JSON':
    case 'SCHEMA_MISMATCH':
      // Retry with different prompt or handle gracefully
      break;
    case 'NATIVE_ERROR':
      // Log error for debugging
      console.error(result.error.message);
      break;
  }
}

Web Support

This plugin is iOS-only. On web platforms, it returns:

{
  success: false,
  error: {
    code: 'UNAVAILABLE',
    message: 'Apple Intelligence is only available on iOS 26+ devices...'
  }
}

License

MIT

Contributing

Contributions are welcome! Please read our contributing guidelines before submitting PRs.