Plug-ins for XBench

A couple of weeks ago our engineers found that DataBridge, a Russian translation agency, developed a plug-in for XBench to check Russian translations. I certainly wanted to try it (and by the way, it is also free).

XBench again...

This tool may seem to be my favoite, but that's not true. Unfortunately my favorite tool is not developed yet. But XBench is free and really very helpful in some aspects, so if I find something interesting in it I just cannot help sharing this.

I didn't use it for quite a long time, so ApSIC released build 396 back in February, and I only found this out last week. This build is claimed to load big files much faster and to have more enhancements.

Vison of a QA tool

It's for about five years I have a vision of an ideal formal translation QA tool - at least for our needs. A tool that would spot errors correctly, determine different languages (which is so easy!) in a multilingual project and not report inconsistent translations when the same phrase is translated into different languages.

It would intelligently detect where a capital letter is required and where it is a false positive to report it etc. It would easily and correctly handle flexing languages when checking terminology. It would automatically not report untranslated segments like "Windows Vista". It would easily open any of numerous bilingual file formats, allow to make changes on the fly and once changed, automatically recheck if the error is in fact corrected. It would provide you with clean and intuitive reports where a click of a button allows to correct all the errors of the same type, make a note for the future that this error is highly possible again and advice on adding it to a checklist. It would accumulate statistics and provide recommendations to translators and QA specialists based on different criteria. It would automatically "pre-correct" errors in new projects based on its experience with previous ones.

To be continued or never published :)

QA functionality in XBench 2.7

XBench 2.7 (build 0.183)

Supported checks. This tool officially does not support Unicode, and this is probably the main drawback of the application that eventually resulted in a rather high error level. For this reason, it does not support checking Arabic, Chinese, Farsi as well as Czech and Polish TTX files.
It does not have punctuation checks enabled by default; however, they are easy to enable via XBench rules which may significantly improve its error reporting in real life.
False positives. XBench reported corrupt characters for all non-Latin and non-Cyrillic characters.
Multilingual project support. Like many other QA tools, this one finds inconsistencies between translations into different languages and reports terminology errors in untranslated segments.
Conclusion. This tool is very new to the market, but probably on of the most promising ones to date. Its extensive file format support, additional functionality and extension capabilities together with the fact that the tool is currently free allow to suppose many companies, particularly small ones, may want to select it as their QA solution.

QA functionality in QA Distiller 6.0.0

QA Distiller™ 6.0.0, build 188

Supported checks. This tool supports the widest number of possible checks which still may be extended using regular expressions. However, a serious drawback is that it does not check tags identity. The test file included a hyperlink which was intentionally changed in all translations; however, QA Distiller ignored it.
Additionally, we couldn’t find a way to check untranslatables in Distiller. There are two places where you can set a list of untranslatable items, and our idea was that the tool should make sure they are identical in source and target text. However, Distiller did not report missing untranslatable which existed in the test file.
Another weak point is number formatting check. Distiller only makes sure the number includes separators specified in parameters of the target language, but does not check the order of the separators. So, for example, it will consider 1,222.33 and 1.222,33 to be the same numbers with regard to number formatting.
Multilingual project support. In addition to reporting inconsistencies between different languages, Distiller also handles multilingual batches together with multilingual dictionaries in a strange way. For the first file it encounters, it tries to match all the glossary files in spite of the language indicated in the translated file and the glossary, which results in numerous “ignored terminology” errors. For the second target language, it tries to match it to all the glossaries until it finds the correct one. Then it perfectly matches the rest of the translated files with correct glossaries and doesn’t generate error messages.
Right-to-left language support. While this is the most comprehensive QA tool so far, it definitely lacks right-to-left languages support. Sentences in those languages are still aligned left-to-right, and if a segment ends with non-Arabic/Farsi/Hebrew words or digits, QA Distiller often handles the end of the segment incorrectly which results in a false error message. Additionally, it reports terminology errors in almost every segment because of incorrect RTL text handling. If you open the same file in MS Word and do a simple search for the glossary term you will be able to locate it easily while Distiller insists the term translation is missing.
It does not support Farsi by default, so we had to define a new language which resulted in reporting too many corrupt characters (480 occurrences).
Additional observations. Inability to change error severity may also be considered as a disadvantage (at least it makes the software less flexible and customisable).
Conclusion. At the moment, this is the most comprehensive, yet rather expensive standalone solution on the market.

QA functionality in ErrorSpy 4.0

ErrorSpy 4.0, build 001

Supported checks. The total set of supported checks is quite extensive with some specifics listed below. ErrorSpy includes presets for some languages, but they are sometimes incorrect (e.g. incorrect quotation marks for French). It does not support Chinese Traditional as well as right-to-left languages without additional customisation and does not support specifying more than one set of quotation marks in case of nesting. Although it allows to specify decimal and thousand separators to check number formatting, we failed to make it check it. In fact it only reported unmatched figures.
File import. The tool cannot check for skipped segments because it does not import skipped segments at all. This is not convenient if you need to locate any segments that were left untranslated.
False positives. ErrorSpy reported “space required after punctuation mark” errors even if the corresponding checkbox is deselected.
Right-to-left language support. No support by default; however, the tool allows to create new languages. For Arabic, it reports Latin characters to be punctuation marks although they were listed as valid characters in language configuration.
Additional observations. English user interface contains translation errors (for example, one of the checkboxes reads: “spaces that require a space before”).
Another drawback is that ErrorSpy sometimes corrupts the first letter of language names.
This tool does not remember the directory it recently worked with. It is quite inconvenient when you work in a non-default directory.
For some reason, more and more interface elements switch to German with each test run. The interface gets back to English after restart.
The tool crashed on each attempt to check Russian. This may be a problem of this particular installation, but may also be a problem of the whole release. It must be noted, however, that version 3 ran smoothly on the same computer.
Conclusion. Although the tool significantly improved compared to its version 3, the first build seems to be quite unstable. However, with such rapid progress, this tool is rather promising.

QA functionality in Wordfast

Wordfast version 5.51t3

Supported checks. The amount of checks supported is rather limited, but may be extended using custom macros.
File import. Wordfast determines HTML files by <html> tag at the beginning, but not by the real content, whereas in real life this tag may often be omitted.
Multilingual project support. We failed to make it check terminology against the correct glossary. After checking the Arabic test file, Wordfast continued to apply Arabic glossary to the rest of the languages despite of numerous setup changes, glossary recreation, deletion etc. Even when there was no Arabic glossary existing on the computer, Wordfast still reported Arabic terminology errors. It might have got much better scores if it used correct glossaries.
Right-to-left language support. It doesn’t properly handle right-to-left languages and just like all other tools reports terminology errors in untranslated segments.
Conclusion. As many other plug-in tools, WordFast provides quite a good solution for those who select it as a TM tool and do not want to implement a standalone QA tool.

QA functionality in SDL Trados QA Checker 2.0

SDL Trados QA Checker 2.0, plug-in to SDL Trados 2007

Supported checks. Basic set of checks performed by SDL Trados QA Checker is rather extensive compared to other plug-in tools and may be extended using regular expressions. This tool does not allow to specify Chinese full stops as valid punctuation marks. Moreover, Arabic and even Easter European characters cannot be included into forbidden characters list. It also does not check quotation marks and number formatting.
False positives. Unlike all other tools, it generates false positives by counting skipped and empty segments as incomplete ones.
Conclusion. In general, the tool is good enough for translators who work in Trados TagEditor, but may be hard to employ in dedicated quality assurance departments where batch processing of mono- and multilingual projects is normally required.

QA functionality in Star Transit XV Professional

Star Transit XV Professional, version 3.1 SP 21 Build 617

Supported checks. Star Transit employs the most limited number of checks without any further customisation. Available customisation is provided via fixed value lists and does not allow to add e.g. custom delimiters which in our case was necessary for Farsi .
False positives. Due to the limited number of checks supported by Star Transit it generates one of the lowest numbers of false positives.
File import. Import of TTX files is not correct enough; tags are represented in an unusual manner which hinders work with files.
Right-to-left language support. This tool proved to be surprisingly good at checking terminology in right-to-left languages. It also showed probably the best handling of right-to-left languages in general.
Reportability. The tool does not provide any reports; all errors need to be corrected “on the fly”.
Conclusion. In general, the real functionality of the tool is closest to the claimed one; however, it is too limited.

QA functionality in SDLX 2007 QA Check

SDLX 2007 QA Check, build 7014
Supported checks. Basic set of checks performed is also rather limited. A user can extend and customise it using regular expressions; however, regular expressions are often beyond the qualification of a QA manager.
This tool does not check number values and does not check number formatting, double punctuation marks and brackets unless you set up a regular expression. It also does not check tags, and though it is hard enough to change tags in SDLX, TTX files converted to SDLX format may contain corrupt tags which won’t be detected.
Skipped translations are not converted from TTX files and therefore are also not found.
SDLX does not allow specifying Chinese full stops as a valid punctuation mark.
False positives. QA Check generates false positives for forgotten translations (counted as partial translations as well). Also many false positives are generated for partial/incomplete translations (they are not differentiated in SDLX) because incompleteness is determined only by translation length, not taking into account the number of sequential source words found in target segments.
Multilingual project support. As many other tools, QA Check Checks translation consistency between different languages.
Right-to-left language support. QA Check displays such texts left-to-right which hinders work with files and leads to reporting non-existing terminology errors.
Conclusion. With additional customisation, this tool is quite a good solution for SDLX users that do not want to involve additional standalone tools into their work processes.

QA functionality in Déjà Vu X Workgroup version 7.5.302

It's been almost two years since I researched translation quality assurance tools available on the market. The full research was presented at Translating and the Computer 29 conference in November 2008 and is published at my company Web site. Needless to say, I would like to update the results and find something really new in this area.

So far, just to start with some facts, I'll re-publish short benchmark results from the research. And I'll start alphabetically, so Deja Vu is the first.

Déjà Vu
Supported checks. The number of default checks is rather limited; however, the application supports custom SQL queries which most probably allows for extending the amount of possible checks and further customisation.
Multilingual project support. While checking several files translated into different languages, Déjà Vu applies the TM and glossary for the first language to all files no matter what their target language is.
Right-to-left language support. For Arabic and Farsi it reported terminology errors even where the translated term could be easily found using Find feature.
Reportability. The style of error indication probably fits translators who want to check the translation “on the fly”, but is rather inconvenient for a dedicated quality assurance department.
Conclusion. In general, this is one of few tools declared capabilities of which are close to real ones. The tool is only suitable for checking its native files. Although it supports other most common formats including Trados, SDLX and Star Transit, conversion is quite time-consuming and is not in general worth it.