fail k2.compose(H, LG, inner_labels="tokens")

[liveroomand]

when i run local/test.py, fail in line 136 --->HLG = k2.compose(H, LG, inner_labels="tokens")

error:

/var/www/k2/csrc/array_ops.cu:279:void k2::RowSplitsToRowIds(const k2::Array1&, k2::Array1*) Check failed: num_elems == row_splits[num_rows] (275402852 vs. 30203481)

[ Stack-Trace: ]
/opt/conda/lib/python3.10/site-packages/k2/lib64/libk2_log.so(k2::internal::GetStackTrace()+0x34) [0x7f0e202a19b4]
/opt/conda/lib/python3.10/site-packages/k2/lib64/libk2context.so(k2::internal::Logger::~Logger()+0x2a) [0x7f0e2083cd4a]
/opt/conda/lib/python3.10/site-packages/k2/lib64/libk2context.so(k2::RowSplitsToRowIds(k2::Array1 const&, k2::Array1)+0x202) [0x7f0e20856952]
/opt/conda/lib/python3.10/site-packages/k2/lib64/libk2context.so(k2::IndexAxis0(k2::RaggedShape&, k2::Array1 const&, k2::Array1)+0x405) [0x7f0e20a55fc5]
/opt/conda/lib/python3.10/site-packages/k2/lib64/libk2context.so(k2::Index(k2::RaggedShape&, int, k2::Array1 const&, k2::Array1*)+0x36b) [0x7f0e20a56efb]
/opt/conda/lib/python3.10/site-packages/_k2.cpython-310-x86_64-linux-gnu.so(+0x1b2502) [0x7f0e263ad502]
/opt/conda/lib/python3.10/site-packages/_k2.cpython-310-x86_64-linux-gnu.so(+0x17b6de) [0x7f0e263766de]
/opt/conda/lib/python3.10/site-packages/_k2.cpython-310-x86_64-linux-gnu.so(+0x158284) [0x7f0e26353284]
/opt/conda/lib/python3.10/site-packages/_k2.cpython-310-x86_64-linux-gnu.so(+0x3dda7) [0x7f0e26238da7]
python3() [0x4fd907]
python3(_PyObject_MakeTpCall+0x25b) [0x4f705b]
python3() [0x5098bf]
python3(_PyEval_EvalFrameDefault+0x13b3) [0x4ef0e3]
python3(_PyFunction_Vectorcall+0x6f) [0x4fdd4f]
python3(_PyEval_EvalFrameDefault+0x4b26) [0x4f2856]
python3(_PyFunction_Vectorcall+0x6f) [0x4fdd4f]
python3(_PyEval_EvalFrameDefault+0x13b3) [0x4ef0e3]
python3(_PyFunction_Vectorcall+0x6f) [0x4fdd4f]
python3(_PyEval_EvalFrameDefault+0x13b3) [0x4ef0e3]
python3(_PyFunction_Vectorcall+0x6f) [0x4fdd4f]
python3(_PyEval_EvalFrameDefault+0x31f) [0x4ee04f]
python3(_PyFunction_Vectorcall+0x6f) [0x4fdd4f]
python3(_PyEval_EvalFrameDefault+0x31f) [0x4ee04f]
python3() [0x5951c2]
python3(PyEval_EvalCode+0x87) [0x595107]
python3() [0x5c5ef7]
python3() [0x5c1030]
python3() [0x459781]
python3(_PyRun_SimpleFileObject+0x19f) [0x5bb5bf]
python3(_PyRun_AnyFileObject+0x43) [0x5bb323]
python3(Py_RunMain+0x38d) [0x5b80dd]
python3(Py_BytesMain+0x39) [0x5883f9]
/usr/lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f0ed94cdd90]
/usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x7f0ed94cde40]
python3() [0x5882ae]

Traceback (most recent call last):
File "/home/liweiyong/project/icefall/egs/commonvoice/ASR/./local/compile_hlg.py", line 168, in
logging.basicConfig(format=formatter, level=logging.INFO)
File "/home/liweiyong/project/icefall/egs/commonvoice/ASR/./local/compile_hlg.py", line 158, in main
logging.info(f"Processing {lang_dir}")
File "/home/liweiyong/project/icefall/egs/commonvoice/ASR/./local/compile_hlg.py", line 136, in compile_HLG
# also change other places in icefall that are using
File "/opt/conda/lib/python3.10/site-packages/k2/fsa_algo.py", line 486, in compose
ans = intersect(a_fsa_inv,
File "/opt/conda/lib/python3.10/site-packages/k2/fsa_algo.py", line 407, in intersect
out_fsa = k2.utils.fsa_from_binary_function_tensor(a_fsa, b_fsa,
File "/opt/conda/lib/python3.10/site-packages/k2/utils.py", line 629, in fsa_from_binary_function_tensor
value, _ = b_value.index(b_arc_map,
RuntimeError:
Some bad things happened. Please read the above error messages and stack
trace. If you are using Python, the following command may be helpful:

  gdb --args python /path/to/your/code.py

(You can use `gdb` to debug the code. Please consider compiling
a debug version of k2.).

If you are unable to fix it, please open an issue at:

  https://github.com/k2-fsa/k2/issues/new

[pkufool]

Can you clarify the size of your H and LG? I suspect it is an overflow of int32.

[ALIVE321]

Similar to #1010. Any way to solve this?

[liveroomand]

2024-05-10 03:23:26,208 INFO [compile_hlg.py:133] H shape before Composing HLG: (501, None)
2024-05-10 03:23:26,208 INFO [compile_hlg.py:134] LG shape before Composing HLG: (11745505, None)

[liveroomand]

Similar to #1010. Any way to solve this?

That's true, Smaller LG Compose succeeded （H.shape: 501, LG.shape: 11745505）, the larger LG Compose fail （H.shape: 501, LG.shape: 51395348）. the larger LG there is no other good way to Compose. how much is the upper limit of Compose

[danpovey]

If this is a graph-building exercise of some kind it may be better to use the kaldifst project https://k2-fsa.github.io/kaldifst/
to do the graph-building, then if needed it can be converted to the k2 format.

[ALIVE321]

If this is a graph-building exercise of some kind it may be better to use the kaldifst project https://k2-fsa.github.io/kaldifst/ to do the graph-building, then if needed it can be converted to the k2 format.

I've tried KaldiFST following the PR here, but the result just did not change.

[pkufool]

What do you mean "just did not change"?

[ALIVE321]

What do you mean "just did not change"?

I got the same error as above, after the optional OpenFST procedures.

[pkufool]

You mean you can not compile HLG even with openfst?

By using openfst, you should convert H and LG from k2 to openfst, then compile HLG using openfst, convert the HLG back to k2 at last.

[csukuangfj]

What do you mean "just did not change"?

I got the same error as above, after the optional OpenFST procedures.

Could you show the error logs after switching to kaldifst?

[ALIVE321]

What do you mean "just did not change"?

I got the same error as above, after the optional OpenFST procedures.

Could you show the error logs after switching to kaldifst?

Sorry that I didn't save the logs. I followed the LibriSpeech script in egs/librispeech/ASR/local/compile_hlg_using_openfst.py.

It started running slowly with openfst for several hours, and the same error as mentioned in this issue happened again when composing HLG at Line-156.

My lm.arpa has 80 million grams with order 1 to 3.

[liveroomand]

openfst

k2 to openfst use "from kaldifst.utils import k2_to_openfst ---> H_fst = k2_to_openfst(H)". compose use "HLG_fst = kaldifst.compose(H_fst, LG_fst)"， it success. how to convert openfst to k2

[ALIVE321]

openfst

k2 to openfst use "from kaldifst.utils import k2_to_openfst ---> H_fst = k2_to_openfst(H)". compose use "HLG_fst = kaldifst.compose(H_fst, LG_fst)"， it success. how to convert openfst to k2

Check files of k2-fsa/icefall#606. The conversion between k2 and fst is successful, but I guess the problem here is that k2.compose is still used after converting fst to k2.