Confidence scores with Zipformer models

[asterixvn]

Hi all,

I am decoding a Zipformer model with sherpa-onnx (and K2/icefall) and I am wondering if there is any way to get confidence scores for the hypothesized tokens with sherpa-onnx-offlne or other tools.

If this is not possible, can you please share any hints on where I can get this information from the source-code or how I can generate the confidence scores? Information from K2/icefall scripts and programs will be helpful also.

Many thanks.
Bac

[csukuangfj]

Yes, it is possible.

Taking the greedy search as an example,

sherpa-onnx/sherpa-onnx/csrc/offline-transducer-greedy-search-decoder.cc

Lines 52 to 55 in 03ff9db

	auto y = static_cast<int32_t>(std::distance(
	static_cast<const float *>(p_logit),
	std::max_element(static_cast<const float *>(p_logit),
	static_cast<const float *>(p_logit) + vocab_size)));

You can get the log_prob of a token from the above code.

Note that you can compute log_softmax from logit and then get the log_prob of the max token at time t.

[asterixvn]

Thanks a lot, Fangjun. It helps!

[KarelVesely84]

#571