Live Captions and Live Universal Translator

Introduction

Live Captions and Live Universal Translator are part of the Brightcove AI Suite. They add real-time AI captioning, caption translation, and voice dubbing to Brightcove Live channels.

You configure these options when creating a Live channel, or later from the channel settings. Once the channel is live, generated captions and translated tracks are available in the player track menus.

Creating a NextGen Live Event channel with AI enhancements

Live AI Enhancements are available for NextGen Live channels with the Event Channel type.

Open the Live module.
Click New Channel.
When prompted to choose the channel type, select Event Channel. Live AI Enhancements are not available for Linear Channel.
Complete the channel configuration step, including the channel name, rendition profile, frame rate, region, and input transport protocol.
Click Next.
In step 2, turn on Live AI Enhancements.
Review the frame rate change, then click Proceed. By proceeding, you accept that the channel frame rate will automatically switch to 29.97 fps.

Current limitation: Live AI Enhancements only work with a 29.97 fps frame rate. Brightcove is working on removing this limitation.

AI-enhanced Live limitations

Live AI Enhancements have the following current limitations:

Multiple audio tracks are not supported. If Live AI Enhancements are enabled, the Audio Tracks section displays a message and the Add Audio Track action is unavailable.
Low Latency (LL-HLS) is not supported.

Audio Tracks section showing that Live AI captions do not support multiple audio tracks

Configuring auto-captions and translation

Use the Auto-Captions & Translation section to define the source language and any caption translation languages for the live stream.

Expand Auto-Captions & Translation.
Select the Source Language for the event's primary language. Live AI Enhancements do not support auto-detect, so you must select the actual source language for the live stream. You can also select Use custom Dictionary and Glossary to help improve recognition for custom terms.

The Use custom Dictionary and Glossary option uses the same dictionary and glossary configured for VOD captioning.
Click Add Languages next to Captions Translations Languages.
Select one or more target languages, then click Save.

Caption translation adds approximately 30 seconds of latency to allow real-time processing. This latency stays at approximately 30 seconds even when you add multiple languages, because caption translations are processed in parallel.

Configuring real-time voice dubbing

Use Real-Time Voice Dubbing to generate real-time dubbed audio tracks from captions.

Expand Real-Time Voice Dubbing.
Click Add Languages next to Dubbing Languages.
Select one or more dubbing languages, then click Save.

Dubbing is generated from caption translations. Any languages added for dubbing are also added to caption translation.

Real-time voice dubbing adds approximately 60 seconds of latency to allow real-time processing. This latency stays at approximately 60 seconds even when you add multiple dubbing languages, because dubbing languages are processed in parallel.

Editing Live AI settings

You can review and update Live AI settings from the channel settings.

You can edit the following settings:

Remove Live AI Enhancements as a whole.
Add or remove caption translation languages.
Add or remove dubbing languages.

Open the Live channel.
Click Channel Settings.
Select Live AI Enhancements.
Update the Live AI settings as needed.
Click Save.

Channel Settings showing Live AI settings and warning that settings cannot be edited while live

Viewing generated captions and audio tracks

Connect your source to the channel the same way you would for a standard NextGen Live stream.

Open the Live channel.
Click Start in the Control Room header and wait for the channel to be ready for the encoder connection.
In the Input Source panel, copy the Input URL and Input Stream Name.
Configure your encoder with the Input URL as the stream endpoint and the Input Stream Name as the stream name/key.
Start streaming from your encoder.

Generated caption tracks are available from the player captions menu.

Live channel player showing generated caption options

If you configured real-time voice dubbing, generated audio tracks are available from the player audio track menu.

Live channel player showing generated audio track options

Supported languages

Live Autocaption

Live Autocaption supported languages and codes (Part 1)
Language	Code
Afrikaans	`af`
Arabic	`ar`
Azerbaijani	`az`
Bashkir	`ba`
Belarusian	`be`
Bulgarian	`bg`
Bangla	`bn`
Bosnian	`bs`
Catalan	`ca`
Czech	`cs`
Welsh	`cy`
Danish	`da`
German	`de`
Greek	`el`
English	`en`
English (Australia)	`en-AU`
English (United Kingdom)	`en-GB`
English (United States)	`en-US`
Esperanto	`eo`
Spanish	`es`
Estonian	`et`
Basque	`eu`
Persian	`fa`
Finnish	`fi`
French	`fr`
Irish	`ga`
Galician	`gl`
Gujarati	`gu`
Hebrew	`he`
Hindi	`hi`
Croatian	`hr`
Hungarian	`hu`
Indonesian	`id`
Italian	`it`
Japanese	`ja`
Kazakh	`kk`

Live Autocaption supported languages and codes (Part 2)
Language	Code
Kannada	`kn`
Korean	`ko`
Lithuanian	`lt`
Latvian	`lv`
Macedonian	`mk`
Malayalam	`ml`
Mongolian	`mn`
Marathi	`mr`
Malay	`ms`
Maltese	`mt`
Multiple languages	`mul`
Dutch	`nl`
Norwegian	`no`
Punjabi	`pa`
Polish	`pl`
Portuguese	`pt`
Romanian	`ro`
Russian	`ru`
Slovak	`sk`
Slovenian	`sl`
Albanian	`sq`
Serbian	`sr`
Swedish	`sv`
Swahili	`sw`
Tamil	`ta`
Telugu	`te`
Thai	`th`
Filipino	`tl`
Turkish	`tr`
Uyghur	`ug`
Ukrainian	`uk`
Urdu	`ur`
Vietnamese	`vi`
Chinese (Simplified)	`zh-CN`
Chinese (Traditional)	`zh-TW`
Chinese (Cantonese)	`zh-yue`

Live Translation

Live Translation supported languages and codes (Part 1)
Language	Code
Afrikaans	`af`
Akan	`ak`
Amharic	`am`
Arabic	`ar`
Assamese	`as`
Aymara	`ay`
Azerbaijani	`az`
Bashkir	`ba`
Belarusian	`be`
Bulgarian	`bg`
Bhojpuri	`bho`
Bambara	`bm`
Bangla	`bn`
Tibetan	`bo`
Bodo	`brx`
Bosnian	`bs`
Catalan	`ca`
Cebuano	`ceb`
Central Kurdish	`ckb`
Corsican	`co`
Czech	`cs`
Welsh	`cy`
Danish	`da`
German	`de`
Dogri	`doi`
Lower Sorbian	`dsb`
Divehi	`dv`
Ewe	`ee`
Greek	`el`
English	`en`
Esperanto	`eo`
Spanish	`es`
Spanish (Mexico)	`es-MX`
Estonian	`et`
Basque	`eu`
Persian	`fa`
Dari	`fa-AF`
Finnish	`fi`
Fijian	`fj`
Faroese	`fo`
French	`fr`
French (Canada)	`fr-CA`
Western Frisian	`fy`
Irish	`ga`
Scottish Gaelic	`gd`
Galician	`gl`
Guarani	`gn`
Konkani	`gom`
Gujarati	`gu`
Hausa	`ha`
Hawaiian	`haw`
Hebrew	`he`
Hindi	`hi`
Hmong	`hmn`
Croatian	`hr`
Upper Sorbian	`hsb`
Haitian Creole	`ht`
Hungarian	`hu`
Armenian	`hy`
Indonesian	`id`
Igbo	`ig`
Western Canadian Inuktitut	`ikt`
Iloko	`ilo`
Icelandic	`is`
Italian	`it`
Inuktitut	`iu`
Inuktitut (Latin)	`iu-Latn`
Japanese	`ja`
Javanese	`jv`
Georgian	`ka`
Kazakh	`kk`
Khmer	`km`
Kurdish (Northern)	`kmr`
Kannada	`kn`
Korean	`ko`
Krio	`kri`
Kashmiri	`ks`
Kurdish	`ku`
Kyrgyz	`ky`
Latin	`la`
Luxembourgish	`lb`

Live Translation supported languages and codes (Part 2)
Language	Code
Ganda	`lg`
Lingala	`ln`
Lao	`lo`
Lithuanian	`lt`
Mizo	`lus`
Latvian	`lv`
Literary Chinese	`lzh`
Maithili	`mai`
Malagasy	`mg`
Māori	`mi`
Macedonian	`mk`
Malayalam	`ml`
Mongolian	`mn`
Mongolian (Traditional)	`mn-Mong`
Manipuri (Meitei Mayek)	`mni-Mtei`
Marathi	`mr`
Malay	`ms`
Maltese	`mt`
Burmese	`my`
Nepali	`ne`
Dutch	`nl`
Norwegian	`no`
Northern Sotho	`nso`
Nyanja	`ny`
Oromo	`om`
Odia	`or`
Querétaro Otomi	`otq`
Punjabi	`pa`
Polish	`pl`
Pashto	`ps`
Portuguese	`pt`
Portuguese (Brazil)	`pt-BR`
Portuguese (Portugal)	`pt-PT`
Quechua	`qu`
Romanian	`ro`
Russian	`ru`
Rundi	`run`
Kinyarwanda	`rw`
Sanskrit	`sa`
Sindhi	`sd`
Sinhala	`si`
Slovak	`sk`
Slovenian	`sl`
Samoan	`sm`
Shona	`sn`
Somali	`so`
Albanian	`sq`
Serbian	`sr`
Serbian (Latin)	`sr-Latn`
Southern Sotho	`st`
Sundanese	`su`
Swedish	`sv`
Swahili	`sw`
Tamil	`ta`
Telugu	`te`
Tajik	`tg`
Thai	`th`
Tigrinya	`ti`
Turkmen	`tk`
Filipino	`tl`
Klingon	`tlh`
Tswana	`tn`
Tongan	`to`
Turkish	`tr`
Tsonga	`ts`
Tatar	`tt`
Tahitian	`ty`
Uyghur	`ug`
Ukrainian	`uk`
Urdu	`ur`
Uzbek	`uz`
Vietnamese	`vi`
Xhosa	`xh`
Yiddish	`yi`
Yoruba	`yo`
Yucatec Maya	`yua`
Chinese (Simplified)	`zh-CN`
Chinese (Traditional)	`zh-TW`
Chinese (Cantonese)	`zh-yue`
Zulu	`zu`

Live Dubbing

Live Dubbing supported languages and codes (Part 1)
Language	Code
Arabic	`ar`
Czech	`cs`
Danish	`da`
German	`de`
Greek	`el`
English (Australia)	`en-AU`
English (Canada)	`en-CA`
English (United Kingdom)	`en-GB`
English (Ireland)	`en-IE`
English (United States)	`en-US`
Spanish	`es`
Spanish (Mexico)	`es-MX`
Finnish	`fi`
French	`fr`
French (Canada)	`fr-CA`
Hebrew	`he`

Live Dubbing supported languages and codes (Part 2)
Language	Code
Hindi	`hi`
Hungarian	`hu`
Italian	`it`
Japanese	`ja`
Korean	`ko`
Dutch	`nl`
Norwegian	`no`
Polish	`pl`
Portuguese	`pt`
Portuguese (Brazil)	`pt-BR`
Portuguese (Portugal)	`pt-PT`
Romanian	`ro`
Russian	`ru`
Swedish	`sv`
Thai	`th`
Turkish	`tr`

FAQs

Who can use Live Captions and Live Universal Translator?
This feature will be available as part of the AI Suite and is currently part of a selected beta test. For more information, contact your account manager.
Does Live Universal Translator usage consume AI Suite credits?
Yes. Live Universal Translator consumes AI Suite credits. For more information, contact your account manager.
Are generated captions and dubbed audio available on the VOD asset after the live event?
Yes. If you select the option to save the live event as a VOD asset, the caption and dubbing files are included as part of that VOD asset.
Can generated live captions or dubbed audio be edited after the event?
Yes. If you save the live event as a VOD asset, the captions become standard Brightcove caption files that you can edit in the caption editor as usual.