Ablation Analysis of Seq2Seq Models and Vanilla Transformers for Spanish to English Translation

Abstract:

Machine translation, a widely known field in computer linguistics, is a challenging problem which deals with translating between different languages. We undertake the task of translating from Spanish to English in this paper. We provide a comparative analysis of various sequential and non-sequential models such as Seq2Seq, Seq2Seq with teacher forcing, Seq2Seq with attention, and transformers with encoder–decoder architectures for the proposed task. Our experiments show that transformer architecture along with label smoothing attains the best BLEU score on the UN dataset, and it is the most efficient choice. Furthermore, we compare various decoding methods like nucleus sampling, top-k sampling, and greedy approach and also conduct an ablation analysis of each of these until we find the best performing method. Our proposed architecture combination is easily extensible and could be used for other translation tasks as well, with minor changes in the pre-processing steps depending upon the languages under consideration. This method is also computationally economic and might help under constrained environments.