Result Gap between CTC and RNN-T Multi-Task Training

[teowenshen]

Between the non-streaming results of vanilla and CTC, it is clear that the model with CTC performed better for test-other. However, is there any idea or intuition about why the models trained with a secondary CTC loss performed slightly worse for test-clean?

	test-clean	test-other	comment
greedy search	2.17	5.23	--epoch 30 --avg 9 --max-duration 600
greedy search (with ctc)	2.23	5.19	--epoch 30 --avg 8
modified beam search	2.15	5.20	--epoch 30 --avg 9 --max-duration 600
modified beam search (with ctc)	2.21	5.12	--epoch 30 --avg 8
fast beam search	2.15	5.22	--epoch 30 --avg 9 --max-duration 600
fast beam search (with ctc)	2.23	5.18	--epoch 30 --avg 8

[yfyeung]

These gaps are so slight, I prefer to believe they are caused by noise.

[teowenshen]

Understood. Thanks for your reply!