Machine translation testing

Recently, I read a research paper called TransRepair: Automatic Testing and Improvement of Machine Translation. The paper describes a methodology called TransRepair for automated testing of machine translation models under the software testing domain. Below, I will summarize the paper in several ways and discuss the key points. Introduction to TransRepair TransRepair is a method for automatically detecting and fixing conformance problems in machine translation software. It provides both black-box and gray-box approaches to address machine translation software conformance issues.The main steps of TransRepair include generating test cases, creating test guidelines, and automating the repair process. The method provides clear, rigorous and detailed algorithms for test case generation and uses four methods for quantifying sentence differences for comparison. In addition, TransRepair utilizes the principle of structural consistency as … Read more

I have previously read the paper Structure-Invariant Testing for Machine Translation, which proposes a method for detecting the robustness problem of machine translation software systems. Below I will detail my understanding of its contents from several aspects. thrust SIT is a method for detecting robustness problems in machine translation software systems. This method utilizes a metamorphosis relation in a metamorphosis test, i.e., “structural invariance”. By selecting original sentences, generating similar sentences, obtaining results from translation software, performing constituent parsing and quantifying sentence differences, and filtering and detecting problems according to a set threshold, SIT can efficiently detect robustness problems in machine translation software systems. According to the experimental results, SIT can process 2k+ sentences in 19 seconds and achieves 70% accuracy for Google/Bing Translate. However, there is still … Read more