I'm running into similar issues at Nevyansk Castle today.
I've just joined the Sturgians and I need to fix things quick. I've just rushed Varcheg to take it back and ran on to Nevyansk Castle to do the same. I get swarmed just as I finish the first trebuchet so attack I must.
Now, I must say, I really like the layout of Nevyansk. That kill-corridor to the gate is brutal, it's amazing. In previous campaigns this was always one of my last sieges so I was knocking down walls. This would obviously prove to be a lot more painful.
Unfortunately, it looks like the position of the siege ladders needs to be fixed because my men were just dancing in front of the ladders until, after about 5 laps, they decide to take one of the ladders. Even with just 8 defenders on the wall the Vlandians were having a picknick. I went up the ladder on the right and tried to hold out for my guys to come up but the defences are pretty impressive, you can't 1-man this on realistic damage at all.
Left wall:
Right wall:
What about the gate, you ask? So after standing in that kill-corridor, getting shot from both sides, they still manage to break down both gates. And then decide that it's more fun to take the ladders up...
Leaving a rather disappointed Vlandian defense all ready in their conga line:
My men, miraculously, actually overcame the defenses on the right wall eventually. (God bless a good Ulfhednar) Only to just pose there waiting for the arrows to hit. Like a cat that's finally caught a mouse and lost its hunter's instinct.
This should have been a close battle. One that I could easily lose because of the excellent archer positions here. But I'm losing decisively because of the ladders and AI pathfinding, unfortunately.