OpenAI Audio

Text-to-speech, transcription (including speaker-diarized), and audio translation with OpenAI.

This Module converts text to natural-sounding speech, turns audio into text in the original language, produces per-speaker transcripts with timestamps, and translates spoken audio to English in one call. It pairs naturally with voice Channels: transcribe what the caller said, think, then speak the reply back.

How it works

Audio inputs accept an https://… URL, a data:audio/*;base64,… data URL, or a {variable} containing one of those. Text-to-speech output is saved as a base64 data URL — store it externally or hand it to a voice Channel. The File Name fields matter: the extension tells OpenAI what audio format it is receiving.

Configuration

Each OpenAI Module is configured separately, per Environment — this Module has its own API key, so you can reuse the key you set on the core OpenAI Module or use a dedicated one (for example to track this capability's spend on its own).

Field	Type	Required	Example	Description
OpenAI API Key	string	Yes	sk-...	Your OpenAI API key for this Module. Stored securely. Find it in OpenAI Console → API keys → Create new secret key.
OpenAI Organization	string	No	org-abc123	Your OpenAI organization ID, if your account belongs to one.

Tasks

All Tasks in this Module are Generic (all Channels) — they run the same regardless of how the Session was started.

Output fields: type a variable name without braces (for example transcript) and pick its type (string, number, true/false, object, date). The Task saves its result there, and any later Task can use it by writing {transcript}.

OpenAITextToSpeechTask

Synthesizes speech audio from text. The result is written as a data: URL — store it externally or hand it to a voice Channel.

Property	Type	Required	Example	Description
Model	string	Yes	tts-1	tts-1, tts-1-hd, gpt-4o-mini-tts.
Voice	string	Yes	alloy	One of alloy, echo, fable, onyx, nova, shimmer, ash, ballad, coral, sage, verse.
Format	string	No	mp3	mp3 (default), opus, aac, flac, wav, pcm.
Input Text	string	Yes	{reply}	Text to synthesize. Supports {variable} placeholders.
Output Variable	string	Yes	audio	Variable name (no braces). Receives a base64 data URL.

Example: Set Model = tts-1, Voice = alloy, Format = mp3, Input Text = {reply}, Output Variable = audio. With {reply} = "Your appointment is confirmed for Friday.", the Task saves data:audio/mpeg;base64,… to audio, ready to play back.

OpenAITranscribeTask

Transcribes audio to text in the original language.

Property	Type	Required	Example	Description
Model	string	Yes	whisper-1	whisper-1, gpt-4o-transcribe, or gpt-4o-mini-transcribe.
Language Hint	string	No	en	ISO-639-1 hint, e.g. en, it. Improves accuracy when known.
Audio Source	string	Yes	{recordingUrl}	https://… URL, data:audio/*;base64,… data URL, or a {variable} containing one of those.
File Name	string	No	audio.mp3	Filename + extension. Drives the audio format detection. Default audio.mp3.
Output Variable	string	Yes	transcript	Variable name (no braces). Receives the transcript text.
Segments Output Variable	object	No	segments	Optional. Receives the detailed segment list with timestamps.

Example: Set Model = whisper-1, Language Hint = en, Audio Source = {recordingUrl}, Output Variable = transcript. The Task saves the spoken words — e.g. "Hi, I'd like to change my delivery address." — to transcript.

OpenAITranscribeDiarizedTask

Transcribes audio with speaker diarization — per-speaker timestamps and labels. Saves the merged text plus a per-speaker segment list.

Property	Type	Required	Example	Description
Model	string	Yes	gpt-4o-transcribe-diarize	gpt-4o-transcribe-diarize.
Language Hint	string	No	en	Optional ISO-639-1 language hint.
Audio Source	string	Yes	{recordingUrl}	URL, data URL, or {variable} containing one of those.
File Name	string	No	audio.wav	Filename + extension. Default audio.wav.
Text Output Variable	string	Yes	transcript	Variable name (no braces). Receives the merged transcript text.
Segments Output Variable	object	No	segments	Optional. Receives the list of {speaker, start, end, text} segments.

Example: Set Model = gpt-4o-transcribe-diarize, Audio Source = {recordingUrl}, Text Output Variable = transcript, Segments Output Variable = segments. For a two-person call, segments receives entries like {speaker: "A", start: 0.0, end: 4.2, text: "Thanks for calling…"}.

OpenAITranslateAudioTask

Same as transcribe but translates to English in one call. Only whisper-1 supports this path.

Property	Type	Required	Example	Description
Model	string	Yes	whisper-1	whisper-1.
Audio Source	string	Yes	{recordingUrl}	URL or data URL. Supports {variable} placeholders.
File Name	string	No	audio.mp3	Default audio.mp3.
Output Variable	string	Yes	english	Variable name (no braces). Receives the English translation.

Example: Set Model = whisper-1, Audio Source = {recordingUrl}, Output Variable = english. An Italian voicemail saying "Vorrei spostare il mio appuntamento" is saved to english as "I would like to move my appointment."

Common recipes

Summarize a call recording

Transcribe a recording, summarize it with a chat completion from the core OpenAI Module, and send the summary. The Output Task comes from the Standard Module.

1. OpenAITranscribeTask — Model = whisper-1, Audio Source = {recordingUrl}, Output Variable = transcript.
2. OpenAIChatCompletionTask — Model = gpt-4o-mini, System Prompt = Summarize the call in three bullet points., User Message = {transcript}, Output Variable = summary.
3. OutputTask — Message = {summary}.

Best practices

1. Always set the right File Name extension — the extension drives format detection; a .mp3 name on WAV bytes can fail or degrade transcription.
2. Give a Language Hint when you know it — a correct ISO-639-1 hint measurably improves transcription accuracy.
3. Don't keep large audio in the Session — multi-megabyte data URLs bloat the execution; store audio externally and pass only the link between Tasks.
4. Use diarization only when speakers matter — gpt-4o-transcribe-diarize costs more than plain transcription; pick whisper-1 or gpt-4o-mini-transcribe for single-speaker audio.

Error codes

Code	Name	Description
3005	TaskInvalidData	A required property is missing or an output field is not a valid variable name — details
4001	ModuleException	OpenAI rejected the request or the audio could not be processed — details
4008	ModuleMissingConfiguration	The OpenAI API key is not set in the OpenAI Module's Environment configuration — details
4009	ModuleInputEmpty	The input text or audio source resolved to an empty value — details
4012	ModuleInvalidData	The Session data could not be read or the response could not be processed — details

See the Error Code Reference for the full list.

Related pages

• OpenAI Module — the family core; the API key is configured there.
• OpenAI Realtime — for live, low-latency speech conversations instead of request/response audio.