Enhanced E-Branchformer Speech Recognition Model Based on Relative Positional Attention and Bidirectional Decoding

In the field of end-to-end speech recognition, the E-Branchformer model has achieved competitive performance across a variety of tasks. To evaluate and further improve its effectiveness in low-resource language scenarios, this paper proposes an enhanced E-Branchformer speech recognition model. The proposed model optimizes the self‑attention module of the conventional E‑Branchformer encoder by employing a scaled dot‑product attention mechanism with relative position encoding, thereby strengthening the ability to capture sequential positional information as well as local and global dependencies. In addition, mixed‑precision training is adopted to improve training efficiency and reduce GPU memory footprint. Furthermore, a bidirectional decoder together with a joint CTC/attention decoding strategy is utilized to learn richer contextual information. Experimental results demonstrate that the proposed enhanced E-Branchformer model achieves average Character Error Rate (CER) reductions of 0.76% on the low-resource Tibetan speech recognition dataset, 4.22% on a noisy Tibetan speech dataset, and 0.08% on the open-source Aishell-1 Mandarin benchmark, compared to the conventional E-Branchformer baseline. These findings validate the effectiveness of the enhanced model in improving both Tibetan and Mandarin speech recognition performance, as well as its robustness under noisy acoustic conditions. Furthermore, through architectural refinements and the integration of mixed-precision training strategies, the enhanced E-Branchformer attains a favorable trade-off among recognition accuracy, computational efficiency, and memory footprint.