“Il pleut des cordes,” say the French when it’s raining hard. “It’s raining ropes,” is the direct translation. It’s the kind of idiom which can trip up an AI translator. “You can imagine the dilemma this throws AI into,” says François Yvon, a research director specialised in automatic translation with the French National Centre for Scientific Research CNRS.
He says a well-trained system will know that ropes are not literally falling from the sky and use the English phrase “it’s raining cats and dogs”. “But is it going to find the equivalent when it’s translating into Indonesian?” asks Yvon. A comment on the weather is pretty low-risk, but more and more translation is being done by machines, and on topics with higher stakes than the weather. In April, Google said people are now translating, external one trillion words a month with Google Translate.
That doesn’t count the others: DeepL, Microsoft, Claude, ChatCGT and Gemini, not to mention made-to-measure corporate translation systems. Direct sales of machine translation, external (i.e. outside general AI subscriptions) were worth an estimated $1.4bn (£1bn) last year. “For plenty of use cases, raw AI translation is now the right call: multiplying your costs by 100 to catch the last few percent of errors isn’t rational if you’re translating internal documents or product listings,” says John Zimmerman, founder of OKA (Open Knowledge Association), a Swiss non-profit that translates Wikipedia articles into other languages. “For an encyclopedia it’s different.
Wikipedia runs on trust. A wrong fact can sit there for years and get cited by journalists and AI systems,” he says. With more than 10,000 published pieces, mostly into Spanish, Portuguese and English, OKA is, Zimmerman says, the single largest contributor of translated articles to the collaborative online encyclopedia. Which is something it absolutely could not have become without AI.
“Translating 10,000 articles the traditional way would cost millions. In our process, the AI produces the draft, then a human translator checks every sentence,” he explains. But that human input is required less and less. “The machine output is now good enough that skipping the human entirely has become a serious option, and a real temptation across the industry.
Two or three years ago, with tools like DeepL, our translators still rewrote a good chunk of every draft. “Today the prose often doesn’t need fixing at all. So, the question is no longer whether AI translation is good, it clearly is, but whether it’s good enough to remove the human,” says Zimmerman. Ariella Baltié is a translator and legal expert specialised in translating court documents from English into French.
She was contacted recently by someone who’d already translated his documents with AI, asking her to check them. “There were two major errors right at the beginning,” she said. The AI had translated “Le demandeur” (the plaintiff) as “the requester”. “Totally awkward for legal papers,” says Baltié.
When the AI saw “Madame X, épouse Y” (Ms X, wife of Y’), it came up with “Ms X marries Y”. In other words, it mistook a noun for a verb, completely losing the meaning of the phrase. The consequences of mistranslation in the legal sphere can be serious. In the US, for example, errors in machine-translated asylum applications have led to delays in the release of detainees.
Accuracy is a matter of life and death in the medical field. “The units for dosages and blood tests often include Greek letters,” says Carol Bereuter, who is a specialist in medical translations. She came across one automatically-translated text that had translated the greek letter μ as milli (one thousandth) instead of micro (one millionth). “That’s a serious error.
If you give someone a milligram when it’s meant to be a microgram, you are giving them a thousand times the dose,” points out Bereuter. She also makes the point that, as AI translators have got better, catching errors has got more difficult. “Whereas in the past, with things like Google Translate, the sentence would jump out at you because there’d be a grammatical error or it would sound clunky, today it sounds perfectly legitimate. It sounds fluid.
It sounds natural. So the errors are a lot harder to spot.” “If it’s something that will bear your signature, there’s a risk for your reputation. And if decisions are going to be taken on the basis of a translation concerning a medical procedure, a contract you might sign or a major purchase you might make, the risk is even greater,” warns François Yvon. Even sentences that seem quite clear to a human can trip up an AI service, he says.
“When you have words with different meanings, it’s usually very clear for a human which one applies because they understand the context, but when a machine translator comes across the sentence ‘translate into Polish’, for example, it is quite capable of deciding it must be all about shoe cream, says Yvon. Context is often crucial. If an AI translator hasn’t grasped that we’re talking about a football match it is liable to mis-translate “taking a corner”. Last year OKA ran a study, funded by Wikimedia.
It studied 119 articles, 10 language pairs and more than 1,000 hours of translator work. “27% of the AI-generated text still got corrected before publication. Three quarters of those corrections were errors the AI introduced, and 5 to 6% of errors changed the meaning of the original text,” says Zimmerman. Eradicating those errors is likely to be challenging.
“As things stand, I think the fundamental way these systems function makes it quite hard to imagine how they could become 100% reliable,” says François Yvon. This is because probability is a crucial part of the way AI translates each word rather than another and what is probably correct is not the same as what is correct. Especially in the realm of something as subtle and surprising as language. He thinks a new generation of tech will be an improvement.
“We will see translation systems that are increasingly capable of evaluating what they do and correcting themselves using different AI tools simultaneously: one to do the translation, a second to criticize it and a third to produce a new version, for example.” But perhaps the big moment will be when AI can show doubt. “The real breakthrough won’t come until these systems are not merely capable of translating but also of telling us when they’re not sure,” Yvon says. The critical tech staying safe by going underground Tokenomics: Why making AI pay is tricky Why airlines are warning over lithium-ion batteries
