NEUROTRAD

Neural machine translation and human-machine parity: evaluation and post-editing aspects

Ref. no. B1-2020_07
2021-2022


The aim of NEUROTRAD project is defining human-machine parity in neural machine translation and studying its impact on the evaluation and post-editing of both human and machine translation. This international and inter-university project where five Universities are participating (four Spanish Universities, i.e., UMA, UVA, UAH and UCO and one American University, i.e., Kent State University) aims to compare the workflow of neural machine translation + postediting to the workflow of human translation + editing from an empirical and holistic approach. The six specific goals of the project are the following:

1. Define the concept of human-machine parity in neural machine translation.

2. Establish a threshold for the most frequent machine translation metrics (BLEU, METEOR, NIST and RIBES) indicating that the machine translation text has reached human-machine parity.

3. Determine patterns for neural machine translation post-editing that has reached human-machine parity as well as patterns for the correction of human translation and check for differences and commonalities.

4. Compare productivity, post-editing effort and translator's behaviour (with parameters such as pause-word ration, post-editing time for word, etc.) in human translation editing and neural machine translation that has reached human-machine parity.

5. Evaluate human translation and neural machine translation that has reached human-machine parity in order to check the types of errors found.

6. Write post-editing guidelines for neural machine translation that has reached human-machine parity.