The reason is all here.
So a few modders like @Philozoraptor managed to achieve this behavior by increasing the damage of spears. However, this is not ideal as this means that both infantry and cavalry will always use their spears against any kind of enemy and will refuse to switch to their secondary weapons when enemies are too close for spear range. Thus, even if we make spearmen use spears, I will have to code additional behavior to make them switch to their sidearms, such as making them drop their spears, or removing their spears, or decreasing the damage of their spears.
I understand that currently the main reason why spearmen don't use their spears against enemy infantry is because of the poor damage and performance of spears. However, there are already plenty of mods that solve this. The final step is for TW to change the AI melee weapon priority in the C++ engine.
So a few modders like @Philozoraptor managed to achieve this behavior by increasing the damage of spears. However, this is not ideal as this means that both infantry and cavalry will always use their spears against any kind of enemy and will refuse to switch to their secondary weapons when enemies are too close for spear range. Thus, even if we make spearmen use spears, I will have to code additional behavior to make them switch to their sidearms, such as making them drop their spears, or removing their spears, or decreasing the damage of their spears.
I understand that currently the main reason why spearmen don't use their spears against enemy infantry is because of the poor damage and performance of spears. However, there are already plenty of mods that solve this. The final step is for TW to change the AI melee weapon priority in the C++ engine.