ChatGPT, self-defined as a “large language model developed by OpenAI based on the GPT (Generative Pre-trained Transformer) architecture”, has taken the world by storm.
Our world, which is already quick to shortcut any job possible, has turned to ChatGPT for countless tasks ranging from writing your essay for you to giving you recipes for dinner using only the ingredients you have in your pantry.
It truly is an amazing tool that has changed the face of technology, but today we put it to the test in translation efficacy and pit it against what some might deem a giant when it comes to Machine Translation, Google Translate.
Key Takeaways:
- ChatGPT vs Google Translate: a comparison for multilingual translation finds different uses for each platform
- Preliminary evaluation of ChatGPT and Google Translate for translation robustness finds Google Translate to be better at translating long-form content
- ChatGPT translation is better at cultural idioms and expressions but struggles with longer-form text
- Google Translate tests better with longer-form text and translating into more target languages
- Launch of ChatGPT and GPT -4 engine: promising AI chatbot for translations and for not only translating content but creating content in-language
- Both tools use interesting strategies for improving translation accuracy over time and still require human translation for validation and cultural adaptation
- Source language and natural language processing challenges in both translation platforms
- Tone adjustment, creative content, and detailed responses are still ongoing challenges for machine translation platforms
- Social media posts and biomedical abstracts have varied success in translation tasks across both platforms
- Short-term vs. long-term translation goals, for example, balancing performance and utility
- Different families of languages and training data impacts translation outputs
- Chat GPT and Google Translate vs. human translators leaves room for collaboration and growth in the translation industry
Machine Translation has become an important tool in breaking down language barriers. With the rise of global commerce, many businesses are crossing over into other countries and have to conduct business in multiple languages. Content creation in one language only is no longer the standard with many content creators having to turn to translation professionals to help them diversify their content writing.
Google Translate is one of the more popular commercial translation products that most small businesses and language professionals turn to, but a worthy competitor might be joining the ranks.
ChatGPT, as an AI tool, has many functionalities. When asked about its potential uses, ChatGPT itself actually lists Language Translation; more specifically, it states, “ChatGPT can be used to provide real-time language translation, making it easier for people who speak different languages to communicate with each other.”
As a regular user of Google Translate, which I used regularly in my travels to help me oscillate between languages, the possibility of another machine translation tool that could potentially offer a more accurate translation piqued my interest. I had to test it out for myself, so I took to the World Wide Web and tested both platforms on accuracy, functionality and user experience. Keep on reading to find out which of these takes the cake when it comes to machine translation.
Testing on Accuracy
Accuracy is one of the most critical factors to consider when using a machine translation tool.
Google Translate has been in the game a long time, using Neural Machine Translation (GNMT) technology. It’s a technology that constantly improves the quality of translations by learning from millions of examples. With a large end-to-end framework, the system learns over time to create better, more natural translations, which, funnily enough, I got to see in action in my tests (more to come below). On top of that, Google Translate supports over 100 different languages.
ChatGPT is a large language model (LLM) that responds to prompts by processing large amounts of data. This input becomes the basis for the tool to predict an output consistent with the user’s instructions. ChatGPT knows at least 95 natural languages, so it is less robust than Google Translate in terms of language pairs, but we’re testing on the languages that it knows to keep the experiment controlled.
I conducted my tests in Arabic, which both platforms are familiar with. To start, I decided to go for the more difficult idiomatic expressions, translating them between English and Arabic.
I started with an Arabic expression that literally translates to “Tile (or Pave) the Sea”, a cultural idiom to express a challenge or provocation in a moment of anger. Google Translate did not pick up on the expression at all, offering the translation of “Sea Tiles”. ChatGPT actually understood the phrase and translated it correctly into “Pave the Sea.”
I decided to turn the tables and translate an English expression into Arabic. This time we went with “I’ll take a rain check and see you next week.”
Google Translate still wasn’t able to pick up on the expression and translated rain check literally into the term “check the rain” in Arabic.
ChatGPT, on the other hand, was able to pick up on the expression and offered a more accurate translation, even using the correct term in Arabic for someone to excuse themself from an event or outing formally.
So far, ChatGPT appears to be superior in translating cultural expressions and idioms.
I tried to translate another Arabic expression that literally translates to “on my head”, but is usually a term used to express a willingness to do something for someone, similar to “Anything for you” in English.
While Google Translate did translate the phrase correctly, ChatGPT took it a step further, offering insight into what the phrase actually means.
The next step was to test the translation of more general text. I pulled quotes from Twitter, one in English and one in Arabic, to try to translate into both languages. Let’s start with the Arabic social media post, which was about the results of a charity campaign in the United Arab Emirates.
Google Translate offered an adequate description, but the translation had a lot of missing pieces. It was more like it provided a summary in English (with a poor choice of words) rather than translating every single phrase.
ChatGPT offered better translation accuracy, translating every single phrase rather than summarising the key points across one sentence. Not a perfect translation, but it definitely fit more with the intended message of the original post.
When it came to translating an English social media post into Arabic though, Google Translate, which seems to have learned from all of the terms I kept feeding it, provided a much more accurate translation than ChatGPT.
The phrase spoke of peace and positivity. Google Translate was able to pick up on all the statements correctly, understanding the placement of each term in the sentence and translating it accurately into Arabic.
ChatGPT here mistranslated some of the terms; for example, it translated “peace of mind” into “inner peace” instead. A slight difference, but this instantly puts into question the accuracy of the platform when it comes to wordy sentences.
The short expressions were a walk in the park for ChatGPT, but when it comes to longer sentences, I think it has more to learn, mainly when translating from English to Arabic. It’s got the translation into English down though, and seems to have a better understanding of the English language over Google Translate.
That being said, it appears as though Google Translate has learned from all of my inputs of idioms. I decided to try out one more sentence with an English expression to be translated into Arabic, and it seems like it finally figured out that it’s not a literal phrase.
It translated “takes the cake” into “more useful”, which isn’t exactly what I was trying to say, but it understood that two things were being assessed, and one was to be found more beneficial or “useful” to the reader by the end of the article.
All in all, there are minor performance differences, but there is room for improvement on both platforms when it comes to accuracy.
Google Translate learns very fast and adjusts quickly. It has a good understanding of the Arabic language and is good at translating longer-form phrases into Arabic but doesn’t read Arabic as well to translate into English.
ChatGPT, while really good at translating idioms and expressions, could perform better with translating longer-form, wordy phrases into English. ChatGPT also reads Arabic better to translate into English.
Platform Functionality
Next, we’re testing the functionality of both platforms. Functionality plays a critical role in choosing a translation tool. You want to be sure that the tool works for the purposes that you need it.
Both tools support a variety of languages, with Google Translate offering more language translations than ChatGPT. On that basis alone, Google Translate could win. However, let’s look a little deeper into how both tools learn and how a user can use the different functions and features.
Google Translate is a web-based tool that can also be used as a mobile app. It uses a neural network to translate text and has been trained on a large number of benchmark test sets. The mobile app can also translate text in images, allowing travellers to quickly and easily translate signs in foreign languages, a functionality that ChatGPT doesn’t have yet as it is a text-based tool.
ChatGPT is an artificial intelligence-based chatbot that uses a deep learning algorithm to generate human-like text. If you take that into consideration, maybe the slight errors it has been displaying make sense and will be perfected as more and more people use it for translation. However, ChatGPT is a desktop application and can only be used in limited settings. In an office setting, ChatGPT is ideal, but for those on the go, Google Translate is easier to use.
User Experience
User experience is an important consideration when choosing a machine translation tool as it can affect adoption and usage rates.
On Google Translate, the user can quickly go in, select their languages, type in a phrase and go.
In ChatGPT, the machine needs to be prompted. A user will need to explain to ChatGPT that they want a translation. For a more precise translation, you could also prompt ChatGPT to assume the role of a machine translation tool, which is what I did that might have led me to a more accurate translation in my trial. However, in terms of steps to take to translate, Google Translate is faster to use.
As for the user experience over time, this is linked to the machine learning process. Both Google Translate and ChatGPT have learning strategies to provide a better translation experience to their users.
Google Translate, through the help of GNMT, is able to remove bias from their translations and provide relevant and detailed answers. ChatGPT, on the other hand, uses a high-resource pivot language to improve its translation ability across different languages. This is useful in providing more human-like translations, which some people may prefer.
Translation Performance
The translation performance of both platforms has been evaluated in many ways. In a preliminary study, ChatGPT performed better than Google Translate in translating biomedical abstracts. However, in a Google Translate test involving a social media post written in Spanish, Google Translate performed better.
In my experiment, I found that Google Translate provides more accurate translations for longer-form posts and performs better in translating things into other languages rather than English.
ChatGPT performed much better when it came to the cultural idioms and expressions but was hit-or-miss for longer-form text. There seems to be a benefit to each platform; however, the true test lies in the learning of each platform over time.
ChatGPT performed much better when it came to the cultural idioms and expressions but was hit-or-miss for longer-form text. There is a benefit to each platform; however, the true test lies in the learning of each platform over time.
Final Thoughts
ChatGPT and Google Translate are both useful tools for breaking down language barriers.
While ChatGPT is a promising general-purpose language model that has shown good results in preliminary evaluations, Google Translate remains the most popular machine translation tool in the market. However, with the launch of ChatGPT and the GPT -4 engine, we can expect better translations, improved translation outputs, and more award-winning stars in the different families of generative models.
Both tools have their strengths and weaknesses, and the choice of which to use depends on the translation task, language pair, and intended meaning. However, for the translation industry, both tools may require further linguistic validation and cultural adaptation.
While they could be helpful in small translation jobs, in most cases, a human translator might be required to confirm the translation as, at this point, they can’t be blindly used. That being said, if you’re ever looking for translation services, you know where to find us!
P.S. Shameless plug, I know. I couldn’t help myself.
P.P.S. You would do the same…
