What Should Be Included in LQAs

Three Types of Quality Applicable to Translation

At first glance, it might seem logical that the only area where quality control is required in the field of translation is language quality – but this only scratches the surface. Any production process involves three separate quality aspects: quality of the process, quality of the project, and quality of the product or service. In order to establish what should or should not be included in LQAs, let me first provide a brief, descriptive guide to all translation quality aspects.

Quality of the process. This is the highest-level aspect related to general translation work organization. Quality of the process is presumably one of the key areas affecting the choice of translation vendor. It deals with availability of well-designed and documented production and technological processes in place, and many other high-level aspects. More information on this subject can be found on the GALA website under the Supply Chain Assessment Project (SCAP), in which I actively participated.

In reality, we look primarily at actual or imaginary process quality before purchasing goods or services. It constitutes a part of the brand’s reputation, which, in turn, strongly affects our expectations and choices in almost all areas – from food to services to cars. Quality of the process frequently goes under scrutiny during post-mortems when there is a sense that some problems were caused by improperly designed or combined processes.

Quality of the project. This mid-level quality aspect deals with the particular project that produced translated materials in question. Statistically, the quality of the project strongly depends on the quality of the process. The better the process, the higher the probability that a project runs smoothly and completes successfully. But there is no guarantee here; any particular project can go wrong, even in a perfect environment, due to human error or objective factors. A poorly executed project could cause production delays or quality compromised by procedural violations or unjustified shortcuts.

One of the best real-life illustrations of project quality is your airport experience. The process generally runs well, everything is organized, and flights take off more or less on time. But, there are also numerous exceptions, and many of them are caused not by poor process design, but rather by factors like the weather.

Quality of the product (service). This is the only area that the ultimate consumer looks at and deals with. While well-organized processes and smoothly running projects increase the likelihood of producing a better product or service, this is not a given. The reverse is also true: Even with basic, far-from-perfect processes and projects, one can still obtain good results, because a lot depends on the individuals working on this task. For instance, a very experienced translator can still produce perfect results by applying his/her skills and knowledge to compensate for organizational and process flaws.

What Should Be Included in LQAs (and What Should Not)

Now that we have briefly discussed all three types of quality, let me proceed with some considerations and examples.

Can Process, Project and Product (Service) Quality Factors Be Combined?

The overall goal of Quality Assurance (QA) in the translation and localization industry is to measure the suitability of translated content for its intended purpose. This, by definition, eliminates any process- or project-related factors from both methodology and quality metric.

One should remember that assessing the quality of translated content is by no means equivalent to translation vendor or project evaluation. A dramatic delivery delay or some other project-level problem might have cost you some silver in your hair, but did not directly affect the quality of translated content posted on your website.

As for process quality, there is no better example than drinking water. Using a trusted bottled water brand or knowing how authorities are caring about your health investing in sophisticated water purification systems is good, but seeing the proof of actual water quality in the quarterly report sent by the township is reassuring nonetheless.

Personally, I not only use a reverse osmosis water purification system at home, but also check water quality with a simple and inexpensive TDS meter once a month. This unsophisticated procedure only takes a few minutes and gives me priceless peace of mind. It also results in many surprises when I test various samples of water for fun. For instance, friends of ours were absolutely confident that a natural spring by their house was a source of perfectly pure water “with healing properties.” Measurement revealed a TDS varying between 300 and 600 ppm – between good and fair on the tap water quality scale, i.e. heavily mineralized. This emphasizes the need to assess product quality objectively, without regard to processes, reputation or other organizational-level factors.

Of course, some correlation undeniably exists. We do not need to measure quality in 100% of all cases, and can limit checks to randomly selected samples when we are confident in the process as such. However, I would suggest to check tap water prior to drinking if located in a remote area with a reputation for poor quality. But, unlike the topic at hand, this scenario is related to the frequency or extent of checks rather than their process- and project-neutral nature.

To summarize: A logical and consistent quality assurance model should only deal with product or service quality, disregarding process and project quality factors. (Any model combining two or more of these factors is nonviable and flawed by design. Experience of the CLIENT who ordered or initiated translation or localization is by no means equivalent to the experience of the USER.)

Dividing Issues by Origin

As already mentioned in my first article, any viable quality assurance methodology needs to be production process-oriented, which includes not only making procedural and financial sense, but addressing problems discovered during quality assurance directly to the responsible party. Otherwise, unnecessary delays and mistreatment become commonplace, and problems are very likely to evolve into chronic ones.

Typically, at the higher level, there are only two independent parties involved in the translation process: the client (or document owner) and the vendor (or translation team) doing the job. While we all know the vendor is always the culprit if anything goes wrong, it is essential to emphasize that the client plays an important role in defining translation quality through processes and project management.

Most client-end processes cannot be altered or fixed by the vendor, and client-end oversights at the process-level are hard to offset downstream. Terminology process provides the quintessential example: Unless the client provides the vendor with a well-prepared, complete terminology glossary and approach to treat exceptions, new terms not present in the glossary, etc., or outsources this task to a vendor, ensuring correct and consistent terminology becomes a problem. When a proper terminology process is replaced by simple guidelines like “use the TM” or “look at the current translated website” (a widespread practice, regrettably), we can forget about terminology quality, however hard the vendor works. It is common knowledge within the industry that both existing TMs and translated websites contain numerous inconsistencies, lack newer terms, do not provide a clue to which version is correct or current, etc.

At least, in the case described above, the vendor understands the problem, can ask questions, and attempt to resolve problems as they arise. Under other circumstances, all additional knowledge resides on the client side, and the vendor is clueless. This includes cases listed below:

  • Copyright and/or similar pieces cannot be translated as is, but rather need to be substituted by specially prepared and approved pieces relevant to the particular market.
  • Some content present in the source text might be irrelevant for the target market, because certain features are not supported.
  • The target market sometimes calls for customized additions to product functionality and/or documentation that are irrelevant to the source market. The UnionPay payment system, for example, is ubiquitous in China, but hardly known in Europe or the US.
  • Local subsidiaries often expect source materials to be “trans-created” rather than simply translated. We learn this the hard way when they are extremely unhappy with translation quality, start to edit everything themselves, to produce a target text that hardly resembles the source.

All these examples fall under market specifics and have nothing to do with translation itself. They need to be singled out and addressed on the client end through process modification (by providing instructions, guidelines and/or approved localized copyright pieces). Moreover, there is almost no way for translators or reviewers to figure this out independently.

This creates a strong case for separating translation quality assurance into two separate areas by problem origin and process:

  • Language Quality Assurance (LQA) addresses problems introduced by the translation vendor (or team) and related to product or service quality.

LQA can be conducted by any client-designated third party or an independent team on the vendor or client end. All feedback goes to the vendor (translation team) for review. Actual errors (excluding reviewer mistakes and false positives) are fixed upon reconciliation between the reviewer and the translation team. Agreed fixes can be implemented by any party and must be reflected in files, glossaries and TMs.

  • Market Compliance Audit (I suggest referring to this as “MCA”) addresses problems originating on the client (or document owner) end due to process and/or project imperfections. These problems are not directly related to translation itself, and can be neither discovered nor fixed by the translation vendor (team) or reviewer alone.

Market Compliance Audit (MCA) can only be conducted by an authorized client representative and addresses problems similar to those described above. It is required for all documents without exception. All feedback goes to the client (document owner) for review and assists with fixing processes, instructions, and documents on the client end.

Market Compliance Audit (MCA) does not need to be synchronized with LQA, because the two are completely unrelated. On the contrary, the earlier MCA starts, the better, preferably prior to translation start. Consider it the client’s homework assignment; this way, changes reflecting local market requirements are introduced in a timely manner, without extra effort.

Important!  As far as the Market Compliance Audit (MCA) was conducted, all changes to the target text going beyond translation (such as adding, removing, changing or replacing pieces based on local market specifics) need to be documented as special client requirements for each project. This is the only way to ensure that LQA will reveal such problems.

How to Figure out Where a Particular Atomistic Issue Belongs: LQA or Market Compliance Audit

To classify any particular atomistic issue type or subtype one needs to answer a relatively simple question:

Can an independent, competent third-party reviewer find this issue in the target text given an ideal situation, i.e. when all required documents, including source and target texts, Terminology Glossary and Style Guide are available, but no special text addition, deletion or substitution instructions are provided?

In other words, is the issue type in question directly related to process or project organization on the client (document owner) end?

If the answer to the first question is “Yes” (or the answer to the second one is “No”), the issue type or subtype belongs to LQA. Otherwise, it falls within the area covered by the Market Compliance Audit.

Important!  Issue types and subtypes falling under Market Compliance Audit (MCA) should not be included in LQA metrics, because they do not belong there. These issues cannot be explicitly revealed during an LQA. An LQA can only reveal them indirectly as deviations from special client requirements for a particular project.

Don’t get me wrong, all quality factors are important. However, some of them shouldn’t be included in LQAs, but addressed elsewhere. Not everything that applies to quality should be included in the quality metric; LQA needs to follow the production process and can’t cover factors that are outside of its scope.

Why Verity-Related Quality Issues Should Not Be Included in LQA Metrics

Verity, along with its subtypes (Completeness, Legal requirements, Locale-specific content), is described within the MQM core. Scanning through the definitions, one can easily conclude that issues covered by Verity and its subtypes relate to source text imperfections, copyright and legal errors, and deviations caused by the target market specifics, all of which fall under Market Compliance Audit (MCA), where they belong. Quality metrics can only include issue types that are individually discoverable during LQAs, which is not the case here. As explained earlier, Verity-related issues do not originate during translation as such; their root causes lie in process and project organization on the client end. At the LQA stage, they can only be classified by the reviewer as deviations from project-specific instructions.

The Proper Scenario: Market Compliance Audit Separated from and Followed by LQA

To illustrate, let me provide an example: Let’s assume that the original software product provides multiple options to pay for a subscription, and its Chinese version includes an additional one (UnionPay), which is not supported in other markets. The product also has an add-on targeted at the US market, and this add-on is not localized into any other languages.

If the client did due diligence, including a market compliance audit (MCA), that MCA would reveal that an additional payment option needs to be described somewhere in the user assistance materials, but only for the Chinese market. (On the software end, that would be addressed simply by enabling an optional UnionPay payment method). It would also reveal the need to eliminate all references to the add-on for non-US markets. In an ideal world, the MCA would also result in particular instructions for translation vendors that would emphasize these two points, including an additional piece to be translated into Chinese. Alternatively, a certain piece could be instructed to be removed from all languages but Chinese.

Client representatives would immediately notice if something was wrong with the localized version, including the presence of UnionPay in non-Chinese translations or references to the add-on in any translated versions. They would also easily classify these problems by type or subtype.

Translators or reviewers have no way to predict whether UnionPay is an acceptable payment option somewhere in Europe or South America (which, theoretically, it might be). They also have no idea that the add-on in question is not translated into other languages, unless explicitly instructed. A translator would never recognize these as mistakes looking at both source and translated texts alone, and they can only avoid making them by following project-specific instructions provided by the client and generated during MCA. During the LQA stage all such issues fall under a single category, i.e. deviation from these instructions.

Finer type/subtype distinctions are simply artificial from the reviewer’s viewpoint. The closest equivalent is the Check Engine light in your car: when it lights up, it might signal a whole variety of actual issues, but this finer distinction is only important for the mechanic (and your wallet). Generally, at a consumer level, the light is an indicator that something requires attention. Most drivers would find an array of LEDs on the dashboard providing detailed trouble info useless, and would definitely prefer to leave it to a professional.

The Nightmare Scenario: LQA Is Mixed with Market Compliance Audit (MCA)

We have discussed a proper software localization scenario earlier. Now let’s imagine that everything is mixed together, the MCA was not carried out in a timely manner, and no project- or market-specific instructions were generated.

As already mentioned above, translators are typically almost clueless regarding particular market-related changes. If materials are handed off without any special notices, they would automatically translate everything as is following general guidelines and glossaries.

Then the fun starts: Localization managers on the client end are typically not the ones making decisions about market specifics. As far as there was no MCA conducted, they are also clueless (as well as third-party reviewers, if any), and the product is released without changes required for the market in question. That is, with multiple references to the US-specific add-on and UnionPay.

Problems are only revealed accidentally by local subsidiaries who start complaining. Sometimes, they would also prefer to see certain pieces trans-created rather than translated, unaware of the process details, and come up with piles of criticism targeted at localizers. All these complaints are addressed to the translation team without any further investigation.

At this moment, the translation team is already under severe pressure. It has to waste a lot of time going through the list of complaints only to find out that most of them have nothing to do with translation, but were rather caused by missing instructions. An intense exchange between the translation team, client localization managers, local subsidiaries and managers responsible for global product release follows. It includes discussion about the need to trans-create parts of material rather than translate it (which is an absolutely different task), list of languages affected, and multiple other things. Finally, a tedious, posthumous market compliance audit (MCA) is carried out and all changes are introduced with a significant delay, mutual dissatisfaction, a healthy amount of rework and inevitable unnecessary spend.

It’s up to us to draw the proper line between MCA and LQA.


Getting back to the original question of what should or should not be included in LQA metrics:

  • Language Quality Assurance needs to be process- and project-agnostic. It should only concentrate on the quality of the product or service and address issues introduced by the translation vendor (or team).
  • Issues related to the quality of the process or project should not be included in LQA metrics. Any model combining those three factors is nonviable and flawed by design.

Problems originating on the client (or document owner) end that are not directly related to translation and cannot be either discovered or fixed by the translation vendor (team) or reviewer alone need to be covered by the Market Compliance Audit (MCA). The MCA puts the emphasis on process and project organization on the client end. It needs to be carried out prior to project start and provides project-specific instructions for translation teams.