Can AI chatbots take over scholarly translation? I used ChatGPT to help me copyedit text that it had translated. Conclusion: it’s probably more efficient to hire a professional human translator from the start.
I had the interesting opportunity to work with a European economics magazine as it experimented with translating articles into English with ChatGPT. I was one of the human copyeditors who were tasked to edit the ChatGPT output.
I believe in the responsible and ethical use of AI and usually do not use generative AI for editing work. Only in this case, the client was using ChatGPT so it gave me the chance to also experiment with it. (This was ChatGPT 4.0.) In this article, I reflect on what I found.
(The magazine stopped the experiment after two issues and told me they were going to look into getting in-house translators. I think that was a wise decision!)
The conditions of the experiment
The conditions were thus:
- I don’t understand the original language.
- For the first issue, I only received the English translation. For the second, I also got the original text.
- I presumed the client was using the paid version of ChatGPT.
The findings
So here are a couple of things I tried and what I found.
The ChatGPT output was pretty good but did not adjust the structure of the piece.
The first thing that struck me was that the ChatGPT output was pretty good. Some of the drafts sounded completely convincing as an English article.
Others were garbled and made little sense, which seemed to reflect the long sentences of the original. They also had structural issues: English writing generally puts the main message at the beginning of the article or paragraph, but that did not seem to be the typical flow for scholarly writing in that European language.
ChatGPT was helpful as a creative partner.
For the first batch of articles, I didn’t have the original text. So when I was mystified by the English, I asked ChatGPT to translate the English into the original language and then retranslate that “original” into “clear, plain English.” Sometimes, this worked.
ChatGPT was also good in suggesting alternative wordings (even if I didn’t like most of them!) and what might be problematic in the use of a particular word. I didn’t take those suggestions at face value, but ChatGPT’s responses pointed to ways I might do follow-up research.
Where I suspected a false translation (e.g., “false friends“), I could prompt ChatGPT to suggest alternative words based on the possible different meanings of the original word/phrase.
ChatGPT wasn’t so helpful suggesting revisions.
When I prompted ChatGPT to reword certain sentences using “plain language,” its output wasn’t very convincing. It still used big, flashy words and the sentences were not much clearer or better than the originals.
Tone seemed to also be a difficulty for ChatGPT, which either went wordy and jargony if I asked for “professional” tone or completely colloquial if I said “conversational.”
(Granted, my prompt may not have been good enough. I didn’t have the patience to refine the prompt further.)
ChatGPT is not (yet) good enough as a translator.
ChatGPT translation is very helpful for getting the gist, the broad sense, of a text. (I use Google Translate for Thai and, while it’s not at all accurate, it’s enough to give me an idea what the writer might be saying.)
But for creating polished scholarly text that makes sense and reads well in that language, in my view, it’s not ready yet.
Texts are more than words strung together. Different languages organize thoughts in different ways and a clear, well-written text in one language will be organized differently in another. Placing words and phrases in the same order as the original text does not make the translation strong; it still takes a skillful human to translate a text (or take the AI output and check it as a translator) so it flows and sounds natural in the target language.
I suspect that was the conclusion my client came to as the experiment ended. I am full of compassion for my commissioning editor who had to check all my queries against the originals and fix them. It’s much more cost-effective to give the text to an experienced human translator from the start.
Summary
- ChatGPT is not good enough to replace human translators.
- It can be useful for trying out ideas and untangling complex text.
We are however living in an age where our digital content will likely be machine/AI-translated. Make it easier for your global readers by using plain language principles.
Cover image by Mohamed Hassan from Pixabay.
