Suspicious unnecessary reshape procedure in network.py

[Franklalalala]

I was reading deepmd-kit\deepmd\utils\network.py, puzzled by this part. ↓

deepmd-kit/deepmd/utils/network.py

Lines 61 to 76 in 159e45d

	if activation_fn != None:
	if useBN:
	None
	# hidden_bn = self._batch_norm(hidden, name=name+'_normalization', reuse=reuse)
	# return activation_fn(hidden_bn)
	else:
	if use_timestep :
	return tf.reshape(activation_fn(hidden), [-1, outputs_size]) * idt
	else :
	return tf.reshape(activation_fn(hidden), [-1, outputs_size])
	else:
	if useBN:
	None
	# return self._batch_norm(hidden, name=name+'_normalization', reuse=reuse)
	else:
	return hidden

Since activation funcs do not change the dimension of variable hidden, it seems unnecessary to reshape it.
I tested “relu”, “relu6”, “softplus”, “sigmoid”, “tanh”. None of them change the dimension. (“gelu” was not supported by tensorflow 1.x.)
Or there is an exception I haven't found.

[Franklalalala]

I tested gelu in tensorflow 2.x, it didn't change the dimension either.