
Security News
PodRocket Podcast: Inside the Recent npm Supply Chain Attacks
Socket CEO Feross Aboukhadijeh discusses the recent npm supply chain attacks on PodRocket, covering novel attack vectors and how developers can protect themselves.
@imgly/plugin-ai-audio-generation-web
Advanced tools
AI audio generation plugin for the CE.SDK editor
A plugin for integrating AI audio generation capabilities into CreativeEditor SDK.
The @imgly/plugin-ai-audio-generation-web
package enables users to generate audio content using AI directly within CreativeEditor SDK. This shipped provider leverages the ElevenLabs platform to provide high-quality text-to-speech and sound effect generation.
Features include:
npm install @imgly/plugin-ai-audio-generation-web
To use the plugin, import it and configure it with your preferred providers:
import CreativeEditorSDK from '@cesdk/cesdk-js';
import AudioGeneration from '@imgly/plugin-ai-audio-generation-web';
import Elevenlabs from '@imgly/plugin-ai-audio-generation-web/elevenlabs';
// Initialize CreativeEditor SDK
CreativeEditorSDK.create(domElement, {
license: 'your-license-key'
// Other configuration options...
}).then(async (cesdk) => {
// Add the audio generation plugin
cesdk.addPlugin(
AudioGeneration({
// Text-to-speech provider
text2speech: Elevenlabs.ElevenMultilingualV2({
proxyUrl: 'http://your-proxy-server.com/api/proxy',
headers: {
'x-custom-header': 'value',
'x-client-version': '1.0.0'
},
// Optional: Configure default property values
properties: {
voice_id: 'pNInz6obpgDQGcFmaJgB', // Default voice (Adam)
voice_settings_stability: 0.5,
voice_settings_similarity_boost: 0.75
}
}),
// Sound effects provider (optional)
text2sound: Elevenlabs.ElevenSoundEffects({
proxyUrl: 'http://your-proxy-server.com/api/proxy',
headers: {
'x-custom-header': 'value',
'x-client-version': '1.0.0'
}
}),
// Optional configuration
debug: false,
dryRun: false
})
);
});
You can configure multiple providers for each generation type, and users will see a selection box to choose between them:
import CreativeEditorSDK from '@cesdk/cesdk-js';
import AudioGeneration from '@imgly/plugin-ai-audio-generation-web';
import Elevenlabs from '@imgly/plugin-ai-audio-generation-web/elevenlabs';
// Initialize CreativeEditor SDK
CreativeEditorSDK.create(domElement, {
license: 'your-license-key'
// Other configuration options...
}).then(async (cesdk) => {
// Add the audio generation plugin with multiple providers
cesdk.addPlugin(
AudioGeneration({
// Multiple text-to-speech providers
text2speech: [
Elevenlabs.ElevenMultilingualV2({
proxyUrl: 'http://your-proxy-server.com/api/proxy',
headers: {
'x-custom-header': 'value',
'x-client-version': '1.0.0'
}
}),
// Add more providers here as they become available
// OtherProvider.SomeModel({
// proxyUrl: 'http://your-proxy-server.com/api/proxy',
// headers: {
// 'x-api-key': 'your-key',
// 'x-source': 'cesdk'
// }
// })
],
// Sound effects provider (optional)
text2sound: Elevenlabs.ElevenSoundEffects({
proxyUrl: 'http://your-proxy-server.com/api/proxy',
headers: {
'x-custom-header': 'value',
'x-client-version': '1.0.0'
}
}),
// Optional configuration
debug: false,
dryRun: false
})
);
});
The plugin comes with two pre-configured providers for ElevenLabs:
A versatile text-to-speech engine that supports multiple languages and voices:
text2speech: Elevenlabs.ElevenMultilingualV2({
proxyUrl: 'http://your-proxy-server.com/api/proxy',
headers: {
'x-custom-header': 'value',
'x-client-version': '1.0.0'
},
// Optional: Configure default property values
properties: {
voice_id: 'pNInz6obpgDQGcFmaJgB', // Default voice (Adam)
voice_settings_stability: 0.5, // Voice stability (0.0-1.0)
voice_settings_similarity_boost: 0.75 // Voice similarity (0.0-1.0)
}
});
Key features:
Custom Translations:
cesdk.i18n.setTranslations({
en: {
'ly.img.plugin-ai-audio-generation-web.elevenlabs/monolingual/v1.property.prompt': 'Enter text to convert to speech',
'ly.img.plugin-ai-audio-generation-web.elevenlabs/monolingual/v1.property.voice_id': 'Select Voice',
'ly.img.plugin-ai-audio-generation-web.elevenlabs/monolingual/v1.property.speed': 'Playback Speed'
}
});
A sound effect generator that creates audio from text descriptions:
text2sound: Elevenlabs.ElevenSoundEffects({
proxyUrl: 'http://your-proxy-server.com/api/proxy',
headers: {
'x-custom-header': 'value',
'x-client-version': '1.0.0'
},
// Optional: Configure default property values
properties: {
duration_seconds: 10, // Duration of sound effect
prompt_influence: 0.3 // How much the prompt influences generation
}
});
Key features:
Custom Translations:
cesdk.i18n.setTranslations({
en: {
'ly.img.plugin-ai-audio-generation-web.elevenlabs/sound-generation.property.prompt': 'Describe the sound you want to create',
'ly.img.plugin-ai-audio-generation-web.elevenlabs/sound-generation.property.duration': 'Audio Length'
}
});
You can control various aspects of the audio generation plugin using the Feature API:
// Disable provider selection for speech
cesdk.feature.enable('ly.img.plugin-ai-audio-generation-web.speech.providerSelect', false);
// Disable provider selection for sound effects
cesdk.feature.enable('ly.img.plugin-ai-audio-generation-web.sound.providerSelect', false);
// Control individual provider visibility
cesdk.feature.enable('ly.img.plugin-ai-audio-generation-web.providerSelect', false);
For more information about Feature API and available feature flags, see the @imgly/plugin-ai-generation-web documentation.
You can customize all labels and text in the AI audio generation interface using the translation system. This allows you to provide better labels for your users in any language.
The system checks for translations in this order (highest to lowest priority):
ly.img.plugin-ai-audio-generation-web.${provider}.property.${field}
- Override labels for a specific AI providerly.img.plugin-ai-generation-web.property.${field}
- Override labels for all AI plugins// Customize labels for your AI audio generation interface
cesdk.i18n.setTranslations({
en: {
// Generic labels (applies to ALL AI plugins)
'ly.img.plugin-ai-generation-web.property.prompt': 'Describe what you want to create',
'ly.img.plugin-ai-generation-web.property.voice_id': 'Voice Selection',
'ly.img.plugin-ai-generation-web.property.speed': 'Speaking Speed',
// Provider-specific for ElevenMultilingualV2
'ly.img.plugin-ai-audio-generation-web.elevenlabs/monolingual/v1.property.prompt': 'Enter text to speak',
'ly.img.plugin-ai-audio-generation-web.elevenlabs/monolingual/v1.property.voice_id': 'Choose Voice',
'ly.img.plugin-ai-audio-generation-web.elevenlabs/monolingual/v1.property.speed': 'Speech Speed',
// Provider-specific for ElevenSoundEffects
'ly.img.plugin-ai-audio-generation-web.elevenlabs/sound-generation.property.prompt': 'Describe the sound effect',
'ly.img.plugin-ai-audio-generation-web.elevenlabs/sound-generation.property.duration': 'Sound Duration'
}
});
The plugin accepts the following configuration options:
Option | Type | Description | Default |
---|---|---|---|
text2speech | Provider | Provider[] | Provider(s) for text-to-speech generation. When multiple providers are provided, users can select between them | undefined |
text2sound | Provider | Provider[] | Provider(s) for sound effect generation. When multiple providers are provided, users can select between them | undefined |
debug | boolean | Enable debug logging | false |
dryRun | boolean | Simulate generation without API calls | false |
middleware | Function[] | Array of middleware functions for the generation | undefined |
The middleware
option allows you to add pre-processing and post-processing capabilities to the generation process:
import AudioGeneration from '@imgly/plugin-ai-audio-generation-web';
import Elevenlabs from '@imgly/plugin-ai-audio-generation-web/elevenlabs';
import { loggingMiddleware, rateLimitMiddleware } from '@imgly/plugin-ai-generation-web';
// Create middleware functions
const logging = loggingMiddleware();
const rateLimit = rateLimitMiddleware({
maxRequests: 15,
timeWindowMs: 60000, // 1 minute
onRateLimitExceeded: (input, options, info) => {
console.log(`Audio generation rate limit exceeded: ${info.currentCount}/${info.maxRequests}`);
return false; // Reject request
}
});
// Create custom middleware
const customMiddleware = async (input, options, next) => {
console.log('Before generation:', input);
// Add custom fields or modify the input
const modifiedInput = {
...input,
customField: 'custom value'
};
// Call the next middleware or generation function
const result = await next(modifiedInput, options);
console.log('After generation:', result);
// You can also modify the result before returning it
return result;
};
// Apply middleware to plugin
cesdk.addPlugin(
AudioGeneration({
text2speech: Elevenlabs.ElevenMultilingualV2({
proxyUrl: 'http://your-proxy-server.com/api/proxy'
}),
middleware: [logging, rateLimit, customMiddleware] // Apply middleware in order
})
);
Built-in middleware options:
You can also create custom middleware functions to meet your specific needs.
For security reasons, it's recommended to use a proxy server to handle API requests to ElevenLabs. The proxy URL is required when configuring providers:
text2speech: Elevenlabs.ElevenMultilingualV2({
proxyUrl: 'http://your-proxy-server.com/api/proxy',
headers: {
'x-custom-header': 'value',
'x-client-version': '1.0.0'
}
});
The headers
option allows you to include custom HTTP headers in all API requests. This is useful for:
You'll need to implement a proxy server that forwards requests to ElevenLabs and handles authentication.
AudioGeneration(options: PluginConfiguration): EditorPlugin
Creates and returns a plugin that can be added to CreativeEditor SDK.
interface PluginConfiguration {
// Provider(s) for text-to-speech generation
text2speech?: AiAudioProvider | AiAudioProvider[];
// Provider(s) for sound effect generation
text2sound?: AiAudioProvider | AiAudioProvider[];
// Enable debug logging
debug?: boolean;
// Skip actual API calls for testing
dryRun?: boolean;
// Extend the generation process
middleware?: GenerationMiddleware;
}
Elevenlabs.ElevenMultilingualV2(config: {
proxyUrl: string;
headers?: Record<string, string>;
debug?: boolean;
}): AiAudioProvider
Elevenlabs.ElevenSoundEffects(config: {
proxyUrl: string;
headers?: Record<string, string>;
debug?: boolean;
}): AiAudioProvider
The plugin automatically registers the following UI components:
ly.img.ai.elevenlabs/monolingual/v1
ly.img.ai.elevenlabs/sound-generation
ly.img.ai.audio-generation/speech/elevenlabs.voiceSelection
Generated audio files are automatically stored in asset sources with the following IDs:
elevenlabs/monolingual/v1.history
elevenlabs/sound-generation.history
For customization and localization, see the translations.json file which contains provider-specific translation keys for audio generation interfaces.
This plugin is part of the IMG.LY plugin ecosystem for CreativeEditor SDK. Please refer to the license terms in the package.
FAQs
AI audio generation plugin for the CE.SDK editor
The npm package @imgly/plugin-ai-audio-generation-web receives a total of 535 weekly downloads. As such, @imgly/plugin-ai-audio-generation-web popularity was classified as not popular.
We found that @imgly/plugin-ai-audio-generation-web demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 12 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Socket CEO Feross Aboukhadijeh discusses the recent npm supply chain attacks on PodRocket, covering novel attack vectors and how developers can protect themselves.
Security News
Maintainers back GitHub’s npm security overhaul but raise concerns about CI/CD workflows, enterprise support, and token management.
Product
Socket Firewall is a free tool that blocks malicious packages at install time, giving developers proactive protection against rising supply chain attacks.