if you will fight against the Raverstern then it should be enough but if against someone else then you need more men if you want to keep the castle you take. I just find out that sieging against the Raverstern is a pain in a place where sun dont shine! I just cant beleve how much stronger they are in sieges then in normal skirmishes or large scale battles.