The Quality
Assurance
Series

Reliably measuring something that’s not completely objective: The Quality Triangle in Language Quality Assurance (LQA)


Leonid Glazychev, Ph.D. | CEO, Logrus IT

The article introduces the first hybrid language quality assurance (LQA) methodology, the Quality Triangle. This methodology combines holistic and analytical approaches and provides the most flexible solution for a wide variety of applications and subject areas. Special attention is paid to the proposed approach detailing how holistic quality factors can be taken into account given the challenge they represent: these factors need to be assessed, but are at the same time not completely objective by definition.
The resulting Quality Triangle methodology provides a reliable model for incorporating semi-objective, holistic factors into quality assurance metrics and can serve as the foundation for building models applicable in real production processes and serving a multitude of purposes.
Keywords: language quality assurance, holistic approach, analytical approach, atomistic criteria, acceptance threshold, objective factors, semi-objective factors

Read article

Building Language Quality Assurance (LQA) Models. Part II. A Simplified LQA Model for Crowdsourcing Environment


Leonid Glazychev, Ph.D. | CEO, Logrus IT

In Part I of this paper, I have presented the universal Quality Triangle (Quality Square) methodology for building LQA metrics. In Part II, my goal is to demonstrate that this methodology is applicable not only to professional LQAs for which it was originally developed, but also to LQAs based on crowdsourcing, which are often the only way to evaluate translation quality for public or government-funded projects.

In this second scenario, the general approach and process used are no less important than the metric itself; both are described in full detail below. The paper also discusses the results of an actual project carried out using the developed process and metric, wherein the goal consisted in reviewing a translated version of an important public portal.

Read article

Language Quality Assurance (LQA) – Part III. What Should Be Included in LQAs (and What Should Not)


Leonid Glazychev, Ph.D. | CEO, Logrus IT

My first article describes a general approach to building language quality assurance (LQA) models on three primary pillars: two holistic factors – readability and adequacy – that apply to the text as a whole, and atomistic quality, which is assessed based on the number and severity of local issues discovered in particular units (sentences, strings).

After it was published, I received some interesting questions and suggestions that were mostly concentrated around the following areas:
  • Some atomistic quality frameworks, such as the MQM framework, are quite extensive and often comprise a number of various quality issues.
    • Should all issue types and subtypes be included in real-life quality metrics?
    • Are all these issues relevant for language quality assurance (LQA)?
    • Are there issues that are important for translation quality in general, but need not be included in LQA metrics?
  • The most popular request was to provide more detail on building bridges between my suggested quality approach and particular quality frameworks. Many people noted that high-level factors like adequacy and readability often seem to have “relatives” within various existing atomistic quality frameworks. For instance, the MQM framework uses terms like “fluency” and “accuracy” that appear similar, but are not identical. Naturally, the following questions arise:
    • Can similar and/or similarly named quality factors coexist on holistic and atomistic levels, and what are the ramifications?
    • Should we attempt to integrate these names or concepts?
    • As I frequently referred to the MQM framework as a good source for illustrating atomistic-level issues, people wondered why I omitted “verity” – the third high-level issue category.
  • Other questions dealt with an absolutely different, but fascinating area: which modifications or extensions do we need to add to the approach to cover cases where translated text needs to not only be readable and equivalent to the source, but accounts for specific requirements like correctly conveyed tone and/or voice, use of language specific to the target audience (teenagers, retirees, etc.), or similar factors? Typical examples include computer games and various content with artistic value.


I am very grateful for all this feedback, especially because it prompted me to address these questions in depth. While I tried to outline a universal approach to building quality metrics, my goal was not to present something all-encompassing, overcomplicated, and cumbersome, but rather to provide a universal and flexible foundation that can be easily developed, customized and extended as necessary.

This article is dedicated to the first topic: what should and what should not be included in LQA metrics. It provides a brief overview of various types of quality (process, project, and product/service (translation)) and explains why we need to avoid mixing these types while creating real-life quality models, why it is necessary to distinguish between LQA and Market Compliance Audit, and why “verity” shouldn’t be used in LQA metrics.

Language Quality Assurance – Part IV will be a natural continuation of this article and is targeted at people interested in more technical details. It provides an insight into the relationship between holistic and atomistic quality factors and discusses ways to minimize confusion while combining suggested methodology with particular quality frameworks.

Part V will address more exciting topics, like finding the right place and approach to accommodate factors like tone and voice (essential for areas such as computer games) within the Quality Triangle (or Quality Square) model without creating controversy or inconsistency.

Read article

Language Quality Assurance (LQA) – Part IV. Relationship between Holistic and Atomistic Quality Issues


Leonid Glazychev, Ph.D. | CEO, Logrus IT

This paper discusses ways to bridge my suggested quality approach and particular quality frameworks.  It is intended for those interested in more technical details and provides further insight into the relationship between holistic and atomistic quality factors. I will also talk about ways to minimize confusion while combining suggested methodology with particular quality frameworks.

Many people noticed that high-level factors like adequacy and readability often seem to have “relatives” in various existing atomistic quality frameworks. For instance, the MQM framework uses concepts like “fluency” and “accuracy”, which sound more alike than they really are.

The questions I will try to address are as follows:
  • Can similar and/or similarly named quality factors coexist on holistic and atomistic levels, and what are the ramifications?
  • Should we try to align these names or concepts?
  • Why the third high-level category of issues included in the MQM framework (the so-called “verity”) is left out from the model I am suggesting.


Read article

Language Quality Assurance (LQA) – Part V. Applying the Quality Triangle Model for Artistic and Special Content


Leonid Glazychev, Ph.D. | CEO, Logrus IT

The original Quality Triangle (Quality Square) model described in my first article was primarily targeted at one of the most popular market segments – technical translation and software localization. But there are other very important and interesting areas that I would like to address.

Unlike Part III and Part IV, the following article does not deal with in-depth technicalities, but focuses on applying the model to content with artistic value. The primary task in this case is finding the right place and approach to accommodate factors like tone and voice within the Quality Triangle model without creating controversy or inconsistency. This is essential for assessing the quality of a computer game or other content with special or additional requirements. Here, I discuss natural limitations related to these factors (primarily the high level of subjectivity) and ways to strike a balance between taking these things into account while preserving objectivity, predictability, and common sense.

Read article