Leonid Glazychev, CEO
We see the word “quality” often. It is intended to evoke a sense of trust, but this trust is based on a very vague metric. So, what exactly is quality? We can easily outline three major facets of quality relevant to almost any service industry:
However, state-of-the-art processes, technology and copious industry certificates can only get one so far without a team of well-educated, competent, attentive employees.
Even with few or no formal processes, a company can theoretically produce high-quality output materials. This situation is not unheard of with experienced freelancers. (Whether this quality is consistent enough is a separate question.)
Even a combination of a perfect production process and exceptional resources cannot guarantee consistently high quality of output materials without implementing a comprehensive quality measurement and control system.
All of these quality facets are addressed in the much-loved holistic approach to translation and localization.
Both the quality of the production process and the quality of resources are universal concepts, and there are recognized and established ways of assessing them. At the same time, quality of output services and materials is unique for each industry. Within the language industry, it is primarily perceived as translation quality. This is exactly where common agreement both starts and ends, because when it comes to defining and measuring translation quality, we see a multitude of interpretations with little in common.
Some companies still insist that good quality lies in having two sets of eyes (a translator and editor/proofreader) on each text. Others point to ISO or other certifications, or to the experience and training of their employees or to their automated tools. While the above are all important, they refer to processes or resources (other quality coordinates) and don’t tell us anything about quality measurement.
Before we can talk about quality, we must do two things:
Another striking fact about the language industry is that while both proprietary and publicly available quality metrics exist, most (if not all) have the following problems:
In more technical terms, none of the existing metrics are based on a robust methodology.
Most metrics, including the venerable LISA quality model, concentrate on formal or technical issues that are more objective and easier to prove or demonstrate (i.e. country standards, typos, syntax errors, tag corruption, etc.) but ignore the crucial fact that translation quality is not entirely objective by nature, and human perception differs from that of a machine.
Quality factors that play the biggest role in real life are general translation adequacy and readability, not minor linguistic or technical problems. Regrettably, none of the existing metrics even touches upon the semi-objective nature of these primary factors affecting our perception of translated text, or upon ways of measuring these factors given lack of complete objectivity.
In short, metrics for measuring translation quality do exist, but they are not well justified, contain major gaps and oversights, and are generally unreliable.
So we ask ourselves, “If everything is that bad, what can we do?” The good news is that the gaps and deficiencies listed above served as the stimulus for the Logrus IT team to take the language quality methodology and metrics issue head-on.
The result is our original Quality Triangle approach to language quality. This methodology is universal, concentrates on factors that are most important to human perception, and has a hybrid nature that combines both holistic and atomistic quality factors. It also addresses the issue of objectivity in quality evaluation and provides concrete ways of measuring quality.
The Quality Triangle methodology also allows to create universal, flexible and adjustable quality metrics covering a whole spectrum of contexts and expectations, including MT applications.
The Quality Triangle methodology and metrics based on it comprise the foundation of the WK54884 ASTM work item.
If you want to learn more about the Quality Triangle language quality measurement methodology, as well as things like semi-objective quality factors, holistic and atomistic quality levels, etc., you’re more than welcome to read about it here: Logrus IT Universal Quality Triangle Approach.