TRY NOW

Multilingual closed captions in livestreams (automatically generated)

Clevercast allows multilingual closed captions for a live stream to be generated in real time through Artificial Intelligence (AI). In the Clevercast backend, simply select the languages for which to provide closed captions. Any number of languages is possible.

When your broadcast starts, Clevercast automatically converts the speech into closed captions and translates them into different languages. No human intervention is needed.

Automatically generated closed captions can be combined with audio translations (RSI) and manually transcribed captions via Translate@Home. Closed captions that are the result of real time transcription by an interpreter can also be automatically translated into different languages.

Clevercast delivers a high-quality, global live stream using adaptive streaming. The embedded player’s closed caption menu allows your viewers to choose their preferred language at any point in the live stream. Clevercast player provides the best possible experience on any device and platform. All closed captions are recorded by Clevercast and can be used for Video on-Demand.

If you require a fully hosted solution, you can also use Clevercast Webinar with multilingual closed captions. If your content is pre-recorded you may want to use pseudo-live streaming, with support for closed captions and audio translations.

How does it work?

» Create a T@H Event in Clevercast. Choose the language spoken in the live stream as default language.

» Add the speech-to-text and/or manual transcription language. For speech-to-text, create a ‘speech context’ with names and terminology that are likely to be used.

» Add automatic translations based on the speech-to-text and/or transcription languages.

» Set the event to ‘preview’ and send an RTMP or SRT broadcast to Clevercast: auto-generated captions will become visible in the preview player. When using manual transcription, interpreters can see the stream and transcribe it.

» Press the ‘Start Event’ button when the action is about to start. Go to your site, to which you copied the embed code. Use the player’s CC menu to turn on closed captions.

Fragment from Sintel by the Blender Foundation. Press the headset menu to change the language.

How does it work?

» Create a T@H Event in Clevercast. Choose the language spoken in the live stream as default language.

» Add the speech-to-text and/or manual transcription language. For speech-to-text, create a ‘speech context’ with names and terminology that are likely to be used.

» Add automatic translations based on the speech-to-text and/or transcription languages.

» Set the event to ‘preview’ and send an RTMP or SRT broadcast to Clevercast. When using manual transcription, interpreters can see the stream and transcribe it.

» Press the ‘Start Event’ button when the action is about to start. Go to your site, to which you copied the embed code. Use the player’s CC menu to turn on closed captions.

Brands and organizations using Clevercast for multilingual live streaming

Features

A multilingual closed captions account has the same features as other multilingual Clevercast accounts.

Features specific to closed captions accounts are:
» The ability to configure live events with closed captions
» A language room for real time transcription by interpreters
» An application that automatically generates closed caption through speech-to-text conversion and machine translation
» The display of multilingual closed captions in Clevercast player
» Server-side recording of the video with closed captions and (optionally) conversion to and hosting as Video on-Demand

Accuracy of closed captions

If closed captions are the result of manual transcription, the quality depends on the human interpreter. If she does a good job, the automatic translations will also be pretty good.

The accuracy of captions generated through (translated) speech-to-text mainly depends on:
» The clarity of the audio and of the speaker (eg articulation, speed, accent, dialect)
» The speaker’s language: speech-to-text conversion usually works better for more common languages (eg English, Spanish)
» Word usage: if many technical or infrequent words are used, this often has a negative effect. Names and abbreviations are also often not recognized.

The accuracy can be improved by defining a speech context, which is a set of words and phrases that Clevercast can pass on to the speech-to-text engine. This can be updated during the live stream.

We expect the accuracy to improve considerably in the future. The language modules used by Clevercast are evolving at a fast pace. We expect that new possibilities will allow us to generate closed captions in a more flexible and accurate way. We may also add new modules for specific languages or features.

Frequently Asked Questions

General questions about platform, player, streaming, plans …?
See our multilingual streaming and FAQ pages.

How many closed caption languages are possible?
Unlimited. In practice, it depends on your plan.

Is it possible to combine closed captions with audio translations in the same live stream?
Yes, if you are using Translate@Home. In that case, viewers can select both an audio language and closed captions in the player.

Can an audio translation be automatically converted to closed captions?
Not yet, but this will become possible in a future version.

Can speech-to-text be used if different languages are spoken in the floor audio?
In this case, speech-to-text is currently not ideal since Clevercast expects the source language and dialect to be set in advance. In a future version, it will become possible to adjust this during a live stream.

What are the costs? How can I order?
Use our price calculator to get a quote for a monthly plan. To order, send us the quote number and we’ll send back an invoice. For more info, see our pricing FAQ.

What are ‘automated caption-generating hours’? How are they calculated?
Unless you only use manual transcription, your plan will include a number of ‘automated caption-generating hours’. They are the sum of the minutes of speech-to-text conversion and translation used. You can calculate this by taking the minutes you broadcast to Clevercast, multiplied by the number of automated caption languages.

When broadcasting simultaneously to both our main and backup server, the number of hours will double.

For example, if you broadcast during 1 hour to a single streaming server and have 3 caption languages that are automatically generated, you’ll use 3 hours.

Getting started

Setting up a live stream with automatically generated live captions isn’t difficult; see the ‘How does it work‘ section above. For further instructions, please refer to our multilingual closed captions manual. Interpreters who want to do real-time transcription to create closed captions can consult our transcription manual.

To get started, request a trial account. Keep in mind that your trial account will contain a limited number of automated caption-generating minutes. If you exceed this, your account will automatically be disabled.

Want to try it yourself?

Try Now