Note 1: This is a revised version of a text previously published at the eMpTy Pages blog under the heading “Post-editing” – what does it really mean?”. This version is more up-to-date (and slightly enlarged), but the blog post is followed by several interesting comments not included here.
Note 2: This new version, published on 2 April, is a very much revised version of the one previously published here.
You might wonder in what way editing of hits in a CAT tool’s TM is different from editing of ”hits” in an MT engine. Because if there was not a clear difference there would not be any reason to invent a particular term for the latter.
But the term ”post-editing” is established since the 1980s, so there should be something to it.[1] And the way I see the difference is this: Certainly for many years, Déjà Vu – in particular – but also memoQ and perhaps also other CAT tools have cleverly put together TM fragments into more or less complete target segment translations, but they can almost never pre-translate whole documents, something which MT can do. It is true that the MT-translated target text might be almost totally useless, but the point is that a client might come to you and say: Here is a machine translated document; could you go through it and produce a useful result? Or: Here is a French document; could you run it in this MT motor and edit the target segments into good German? The expectation, of course, being that the use of MT will make the translation cost less.
The difference lies not so much in the job itself – even if many people say that post-editing of MT-translated texts is rather much different from the ”ususal” translation in a CAT tool with one or more TMs, or without one – as the fact that an MT produces a complete translation – usable or not – of (in theory at least) every piece of source text.
Post-editing also means that the client requests use of an MT motor (or has already used it). If I, in a normal job, use MT and even produce the whole translation by editing MT-translated segments, that’s a different case (and one which does not concern anyone but me, provided that confidentiality is not compromised in any way). Another factor to consider is whether the translation task concerns a pre-translated document or the translator translates the text segment by segment in the usual way but using the suggestions from an assigned MT. I’ll discuss that later in this article.
The matter of quality
In a post-editing job, a level of quality is also specified – the client wants a translation which is good enough for its purposes but does not want to pay for one that is “unnecessarily” good. Therefore the following quality levels have been defined, in the ISO standard 18587:2017, Translation services – Post-editing of machine translation output – Requirements and other documents:
- Light post-editing (also called “gisting”): The final text is understandable and correct as to content, but the editor need not – and should not – strive for a text much better than that; s/he should use as much as possible of the unedited MT version.
- Full post-editing: The result, according to some definitions, should be ”indistinguishable from human output” (in the words of ISO 18587), or ”publishable”. But there are are conflicting views on this: some sources say that stylistic perfection is not expected and that clients actually do not expect the result to be comparable to “human” translation. ”Do not worry too much about style, standards of textuality” and ”Quality expectations: medium”.[2] And: ”Texts that are post-edited should not strive for linguistic perfection; instead, the goal should be linguistic adequacy.”[3]
Looking past the definitions in the standard and instead using definitions of what I perceive as practice, I would say that there are in fact three levels of post-editing MT output: At the top there is the result ”indistinguishable from human output”; i.e. it is impossible to tell whether MT has been used or not. Slightly below that, there is the ”full” post-editing: correct in all regards but perhaps not top-level stylistically. And then there is the ”ligth” level: useable for understanding but not more (not much fun to read).
Of course these categories are only points on a continuous scale; it is difficult to objectively test that a PEMT text fulfils the criteria of one or the other. (Is the light version really not above the target level? Is the full version really up to the requirements? Has the client specified what type of full version is required?).
There are some interesting research results as to the efforts involved, insights which may be of help to the would-be editor. Thus it seems that editing medium quality MT (at all levels) takes more effort than editing poor ones – it is cognitively more demanding than discarding and rewriting the text. Also the effort needed to detect an error and decide how to correct it may be greater than the rewriting itself; and reordering words and correcting mistranslated words takes the longest time of all.
There is also interesting research which shows that a translation’s “fluency” – in the eyes of the post-editor – trumps “correctness”[4], and that a translation which contains a preferred wording but is in fact incorrect will pass, while a correct translation will often be edited to include a preferred wording.[5]
An additional aspect is that all jobs involving “light” quality is likely to be avoided by most translators since it goes against the grain of everything a translator finds joy in doing, i.e. the best job possible. Experience also shows that all the many decisions that have to be made regarding which changes need to be made and which not often take so much time that the total effort with “light” quality editing is not much less than that with “full” (or even ”best”) quality.
Pre-translation or interactive editing?
Then there is the question of whether the job involves a pre-translated document or segment by segment “interactive” translation in a CAT tool with the aid of an MT motor. I have read many articles and presentations and even dissertations on post-editing, and strangely, very few of them have made this distinction. In fact, in many of them it seems as if the authors are primarily thinking of the latter (and the vast majority of the research is done on interactive work). Furthermore, most descriptions or definitions of “post-editing” do not seem to take into account any such distinction. All the more reason, then, to welcome the following definition in ISO 17100:2015, Translation services – Requirements for translation services:
post-edit
edit and correct machine translation output
Note: This definition means that the post-editor will edit output automatically generated by a machine translation engine. It does not refer to a situation where a translator sees and uses a suggestion from a machine translation engine within a CAT (computer-aided translation) tool.
And yet… in ISO 18587, Translation services – Post-editing of machine translation output – Requirements, we are back in the uncertain state: the above note has been removed, and there are no clues as to whether the standard makes any difference between the two ways of producing the target text to be edited.
This may be reasonable in view of the fact that the requirements on the “post-editor” arguably are the same in both cases (and it seems that was the rationale for the decision to delete the note). And it may not matter to the quality of the work performed or the results achieved. But it matters a great deal to the translator doing the work. Basically, there are three possible job scenarios:
- The job consists of editing (“post-editing”) a complete document which has been machine-translated; the source document is attached, and the client defines the desired level of quality. The editor (usually an experienced translator) can reasonably well assess the quality of the translation and based on that make an offer. The offer should take into account any necessary adaptation of the source and target texts for handling in a CAT tool.
- The job is very much like a normal translation in a CAT tool except that instead of, or in addition to, an accompanying TM the translator is assigned an MT engine by the client (usually a translation agency). Here, too, a level of quality is defined. The agency may have a template (similar to the “Trados grid”) for payment, or simply a standard level related to the payment for “normal” translation – normally 60%. But usually it is not possible for the translator to assess in advance the time required (partly because there is still no method for judging in advance the quality of an MT engine).
- The same as B, but the payment is based on a post-analysis[6] of the edited file and depends on how much use has been made of the MT (and, as the case may be, the TM) suggestions. As in B, it is not possible to assess the time required, nor in this scenario the final payment. Also, s/he may not know how the post-analysis is made, in which case the final compensation will be based on trust. (And, of course, if this method of basing payment on a post-assessment of the job done becomes accepted, one can easily foresee it being applied as well to traditional jobs using CAT tools in combination with TMs, without machine translation.)
In addition to this, there are differences between scenarios A and B/C in how the work is done. For instance, in A you can use Find & replace to make changes in all target segments; not so in B/C (unless you start by pre-translating the whole text using MT) – but there you may have some assistance by various other functions offered by the CAT tool and also by using Regular expressions (regex). And if it’s a big job, it might be worthwile, in scenario A, to create a TM based on the texts and then redo the translation using that TM plus any suitable CAT tool features (and regex). And so on.
What about the future?
I was given an interesting view of the development of translation work is given by Arle Lommel, senior analyst at CSA Research and an expert in the field. It goes like this:
A major shift right now is that post-editing is being replaced by “augmented translation.” In this view, language professionals don’t correct MT, but instead use it as a resource alongside TM and terminology. This means that buyers will increasingly just look for translation, rather than distinguishing between machine and human translation. They will just buy “translation” and the expectation will be that MT will be used if it makes sense. The MT component of this approach is already visible in tools from Lilt, SDL, and others, but we’re still in the early days of this change.
Here, “light” post-editing is not even in the picure; I am not aware of today’s demand but I can imagine that this type of work in future will be of interest mainly for larger companies, which then probably can handle it internally.
If mr Lommel is correct, it probably means that we can stop using the “post-editing” misnomer as we do today – editing is editing, regardless of whether the suggestion presented in the CAT tool interface comes from a TM or an MT engine. (This erasing of boundaries is well described by Sharon O’Brien.[7]) It should be reserved only for the very specific case of scenario A. This view is taken in, among others, the contributions by a post-editor educator and an experienced post-editor in the recently published Machine Translation – What Language Professionals Need to Know[8],[9].
My own view of the future is as follows (without any figures at all to back it up):
As neural MT – and “adaptive” statistical MT – produces better and better results, and this becomes known (as it will be) by clients such as big companies and translation agencies, the situation described by mr Lommel will also mean that prices will be forced further down as productivity rises. But this may not be all doom and gloom, or as this quotation states:
The translators of tomorrow will have more in common with skilled engineers than with today’s linguists, who operate in a craft-driven model. The will wiels an array of technologies that amplify their ability and the will be able to focus on those aspects that require human intelligence and understanding, while leaving routine tasks to MT.[10]
(Or, as someone put it, “machine translation will only replace those who translate like machines”.)
Thus it seems that “post-editing” as a service is likely to disappear. But the task of editing MT output will remain, although looking more and more like the task of editing/reviewing the output of a fellow translator. This probably also means that the need for special “post-editors”, as well as a corresponding special training, will disappear – although it will always remain a fact that some translators will avoid the job of editing while others enjoy it. And certainly editing/reviewing merits a place in the education of translators – many of the shortcomings of post-editors found in the research are obviously not primarily caused by the fact that the text-to-be-edited comes from MT.
And while we await that time, very far away I believe, when the only need for human translators will be for translating fiction, we translators of today should strive to make the best possible use of the situation where MT, even if not required by the client, is a resource to be used, or not, as any (other) TM. It is here and will not go away even if some people would wish it to. Or put another way:
[NMT] represents a major breakthrough that [Langugage Service Providers] and their clients should actively investigate. Those that wait will find themselves at a disadvantage.[11]
References:
[1] Maybe a clue is to be found in this statement: “Post-editing should not be confused with pre-editing.” Although how such a confusion might arise I don’t understand. (From Trusted Translations, https://www.trustedtranslations.com/translation-services/post-editing.asp.)
[2] O’Brien, Sharon, Roturier, Johann, and de Almeida, Giselle (2009): Post-Editing MT Output. CNGL. http://www.mt-archive.info/MTS-2009-OBrien-ppt.pdf
[3] Hansen-Schirra, Silvia, Schaeffer, Moritz, and Nitke, Jean (2017). Post-editing: strategies, quality, efficiency. In Porsiel (ed.): Machine Translation. Berlin: BDÜ Fachverlag.
[4] Martindale, Marianna J. & Carpuat, Marine (2018): Fluency Over Adequacy: A Pilot Study in Measuring User Trust in Imperfect MT. https://arxiv.org/pdf/1802.06041.pdf
[5] Koponen, Maarit (2013): This translation is not too bad: An analysis of post-editor choices in a machine translation post-editing tas. In Proceedings of MT Summit XIV Workshop on Post-editing Technology and Practice. https://pdfs.semanticscholar.org/b659/ec47ebf3d05fe38ada7ed45d3afd54434d74.pdf
[6] See for instance Memsource’s ”Post-editing Analysis”, https://help.memsource.com/hc/en-us/articles/115003942912-Post-editing-Analysis
[7] O’Brien, Sharon (2016): Post-Editing and CAT. In: 2016 48 EST Newsletter. https://issuu.com/est.newsletter/docs/2016_48-est
[8] Hansen-Schirra, Schaeffer, Nitke, ibid.
[9] Grizzo, Sara (2017): Working as a post-editor: a field report. In Porsiel (ed.): Machine Translation. Berlin: BDÜ Fachverlag.
[10] Lommel, Burchardt and Macketanz (2018): Will neural technology drive MT into the mainstream? In MultiLingual, January 2018. http://dig.multilingual.com/2018-01/index.html?page=0
[11] Lommel, Burchardt and Macketanz, ibid.
Posted in 