To get data about the quality of free machine translation, TechScribe produced an article in which most sentences were optimised for machine translation. Free machine translation software was used to translate the article into Spanish and into Norwegian. Professional translators evaluated the fluency and the accuracy of the translations.
For the 660 evaluations of fluency, 84% of the translations were ‘excellent’ or ‘satisfactory’ and 16% of the translations were ‘bad’ or ‘incomprehensible’. For the 660 evaluations of accuracy, 87% of the translations were ‘correct’ or ‘partly correct’ and 13% of the translations were ‘incorrect’ or ‘nonsense’.
“The English source contains examples of what not to write. Therefore, if we ignore the low-quality results for these sentences, the quality is better than the evaluations show,” says Mike Unwalla, the principal of TechScribe. The translators gave comments such as, “Overall the meaning of the text is easy to understand.”
“Free machine translation has some problems, and is not as good as commercial machine translation. However, the quality will become better. TechScribe commissioned the evaluations because I did not find data on the Internet,” says Mike. “The evaluations give business people new data that can help them to decide whether free machine translation is a suitable business strategy.”
For the full set of results, see the following web pages:
• English to Spanish, www.international-english.co.uk/mt-evaluation-en-es.html
• English to Norwegian, www.international-english.co.uk/mt-evaluation-en-no.html