A low-resource language may have useful text but few translated sentence pairs. Feeding that text into continued pretraining is an obvious response. It is not necessarily the most effective one.
An October 13 study revisits back-translation: translate monolingual sentences, then train in the reverse direction while interleaving genuine parallel examples. In its evaluated low-resource settings, the approach beats continued pretraining and can outperform using substantially more parallel data alone. Read the research paper.
The text needs the right learning objective
Monolingual data can teach a model how a language looks. A synthetic translation pair teaches a relationship between languages. The same raw material therefore supplies different supervision depending on the pipeline.
The quality of intermediate translations remains consequential. Errors can become targets, so decoding, domain matching and anchoring with genuine pairs deserve explicit measurement.
The authors include human evaluation for two languages, which helps interpret automatic scores. Results across their selected languages and small backbones should not be extended to every language pair.
For a team with scarce labelled data, the lesson is practical: do not equate more pretraining with better use of available text. An established method can become newly competitive when a stronger starting model produces more useful synthetic supervision.
Questions, news tips or corrections? Contact [email protected].
