Pro Interpreters vs. AI Challenge: Who Translates Faster and Better?

[Narrator] Artificial intelligence

has been threatening everyone's jobs,

including interpreters.

Inaccurate interpretation can be the difference

between a good or bad nuclear deal.

So Wired tested a speech translator called KUDO

versus human interpreters to see which better transmits all

of the necessary elements

for people to understand what they're hearing,

including emotion, intonation, completeness,

spontaneity and speed.

[KUDO] Liars jobs.

[Narrator] To test out if AI is really better

than a human.

Let's begin.

Test one.

Here we go.

[Narrator] We gave Barry and KUDO a 2020 speech

from King Felipe of Spain.

The speech is about the COVID crisis.

The subjects will focus on emotion.

Because of the pandemic.

I parachuted into this not knowing anything

about the speech, when it was given.

In this speech, what matters more

is the emotion and the ability to connect

with the audience because he's speaking to the citizens

of Spain at a very difficult time

when many have suffered loss,

are going through a number of challenges.

This reminds us of very deep feelings.

At this time, many of you are fighting against the illness

and what it has done in your homes.

And the words are just the vehicle to get that across.

[Narrator] Now let's evaluate the speech translator.

[KUDO] Good evening, I address you on this Christmas Eve

when we are living truly exceptional circumstances

due to the pandemic.

In terms of content and completeness,

the AI did very well.

[KUDO] The virus has broken into our lives,

bringing suffering, sadness or fear,

altered our way of life.

It struggled with word choice in a number of areas.

It referred to an overflow in some hospitals.

[KUDO] And also overflow in some of our hospitals.

Now, what that was referring to

is at that time when hospitals were overwhelmed,

when the king was recognizing the healthcare workers

for the work that they had done

and he recognized the tremendous physical

and emotional burden that they were carrying

and the AI translated that

as the great emotional charge and physics on their backs.

If you know Spanish, you can go back and figure out

how it is that the AI determined to use those words.

But obviously, that is borderline nonsensical.

If you look

at what the speech-to-speech translation produced,

it was by and large, quite accurate.

However, it's very difficult to get the sense

of what the king is seeking to portray

through the AI in its current state.

[Narrator] Test two, spontaneity.

We gave Walter and KUDO a 2021 speech from former senator

and now current Colombian president, Gustavo Petro.

The most powerful and nonproductive sectors

of this country, 20 billion pesos a year.

So I took the hour beforehand to look him up,

look at some of his videos.

Just listen to the level of language he employs.

Listen to his rhythms.

Listen to, you know, watch his body language.

[Narrator] Now let's see how the speech translator did.

[KUDO] You are, Mr. President-in-Office.

[Gustavo speaking in Spanish]

[KUDO] In the year of 2019, this Congress decided to give

to the most powerful and unproductive sectors

of the country, 20 billion pesos a year.

The particular strength of artificial intelligence

was in capturing those terms of art.

Artificial intelligence captured president-in-office

or sitting president,

[Walter speaking in Spanish]

minister of finance,

[Walter speaking in Spanish]

special drawing rights.

That's a consecrated term.

It's a term that's enshrined

that you can find in any number of glossaries.

That's the official translation, and I missed that.

Official caucuses as well

that the artificial translation rendered that very clearly

and it pulls those in and did so very well.

That's what I felt it was most effective at.

And where I felt it had the most shortcomings

was because of the pauses.

[KUDO] When El Pais has a stagnant economy

and an impoverished middle class

what this tax reform brings.

[Gustavo speaking in Spanish]

[KUDO] It is to increase taxes

on the productive company,

small and medium fundamentally.

[Gustavo speaking in Spanish]

[KUDO] An absolute contradiction and absurdity.

The artificial intelligence creates these little islands,

these little bubbles that read like complete thoughts

but are not because there's a flow.

There's a flow that's coming after the pause.

And thanks to an absolutely aggressive social policy,

making a gift to the bankers.

[KUDO] And thanks to an absolutely

aggressive social policy,

how to issue money to give to bankers.

So the pauses in this particular speech seem

to throw the artificial intelligence off.

You have the vocal inflection.

Increasing taxes on those who govern Colombia.

That you know allows you to understand

there's something else coming.

I'm not finished yet.

Unproductive capital, those who don't work

but prey on the state, on the people and on nature.

Thank you, sir.

[Narrator] Test three.

We gave Barry and KUDO a 2021 speech

from El Salvadorian President Nayib Bukele.

The speech is highly crafted, prepared

and read from a teleprompter at a high rate of speed.

We are going to make use of BANDESAL.

I will tell you that it was a very challenging experience

for me as a professional interpreter

to interpret that speech accurately and completely.

What you have to do is process that information

and extract the ideas to be able

to give your listener something that's usable

for them to be able to follow from that huge amount

or that torrent of words that's coming at you.

[Narrator] Now let's evaluate the speech translator.

Salvadoran people, good morning.

As you all know,

the world has suffered an increase in prices.

When I was able to go back and listen

to the AI interpretation, what I noticed

throughout the entire speech

is that it did not miss any of the content.

[KUDO] The cost of these goods increases

to match the supply of new dollars,

new euros, new yen, and therefore,

that affects world prices.

[Nayib speaking in Spanish]

[KUDO] That is why we are experiencing a global inflation

that hits the pockets of everything, of all the people

in the world.

And El Salvador is no exception.

I was taken aback

by the artificial intelligence's ability

to get all of the content and hold onto it.

So in terms of completeness, it was very good and better

than what I was able to do at that rate of speed.

However, it often had mistakes and challenges

with making the syntax, the word choice,

and sometimes the grammar in English,

and sometimes it would choose words

that weren't really the most accurate way,

the way a native English speaker would say something.

It talked about new currency issues.

[KUDO] This inflation generated by the incorporation

of new currency issues, usually dollars, euros, yen.

Which was really ambiguous

because it was talking about new infusions

of cash or currency into the monetary system.

And people could confuse issues

with problems and think, okay,

what's the problem with these new currencies?

What is happening is that the process

that the professional interpreter has been doing

for as long as professional interpreters have been working

is to process that information

and then serve it up to the listener already processed

to know this is what's being talked about.

So rather than saying new currency issues,

I would've said infusions or injections

of more cash into the monetary system.

[Narrator] So for a profession that started

with the Nuremberg trials and continues to be used

for important meetings and negotiations today,

when is it appropriate to use an AI solution?

I would say in informal situations, situations

where there is not a high level of consequence for mistakes.

I would not trust this in courtrooms.

I would not trust this in the legal system.

If there is a decision that has to be made,

a medical interaction, asylum cases.

I don't know that I would trust it

for situations that require metaphorical language,

that don't catch the figurative meaning.

Do I want to have a high-level diplomat

or a president of my country speaking

to another president through AI to make important decisions

about the future of literally millions of lives?

No. We need to look at it.

We need to examine it.

We need to find out how we can use this constructively

and how we can use it carefully to aid us and not be afraid

of it, because that's no use if we're not flexible.

We're eminently in a world where we need to be flexible

and we need to be open

to new ideas and new ways of doing things.

It is not a replacement, and you have to understand that.

And if we're able to understand the power of the tool,

we can respect the tool and use it to everyone's benefit.