Google's AI Translator prototype translates your tone as well as your words

We all know that communication depends on more than what you say. How do you say that it is often just as important. That's why Google's latest prototype AI translator not only translates the words that come out of your mouth, but also the tone and cadence of your voice.

The system is called Translatotron and Google's researchers detail how it works in a recent blog post. They do not say that Translatotron will turn to commercial products any time soon, but it will probably happen in time. As explained by Google's translation manager at The edge Earlier this year, the company's current goal is to add more nuance to its translation tools to create a more realistic discourse.

You can hear what this looks like in the audio examples below. The first clip is the entrance; the second is the basic translation; and the third attempts to capture the voice of the original speaker.

As you can hear, the translation is not transparent, but it is impressive. You can listen to many other audio extracts of Translatotron here.

Although this is most impressive for the laypersons, the attraction of Translatotron for artificial intelligence engineers lies in the fact that it translates the lyrics directly from the audio input to the audio output. without by translating it into the usual intermediate text.

This type of artificial intelligence model is called end-to-end system because there is no stop for tasks or subsidiary actions. Google says that end-to-end translation produces results faster while avoiding the risk of introducing errors during multiple translation steps.

Even more interesting perhaps, the data processed by the model is not raw audio data. Instead, it uses spectrogram data, or detailed visualizations of the sound. Basically, it means that we translate speech from one language to another using images, which is staggering.

As always with Google's translation efforts, there is reason to be skeptical about how such systems will work in the wild. The company often unveils new translation and auditing tools that are ambitious and often less fluid than expected. Still, the future is getting ready and the translation of AI is constantly improving.