High leadership will also boost your morale, which allows you to maintain more troops. Either that or stockpile a ton of food in your inventory.
I'd be at least level 20 with at least strong pathfinding, first aid, surgery on your heroes. You will want to take a castle quickly, so crank your army to the max and have a good force before starting. Huscarls are great for storming a castle door, but not super strong against Swadian Knights. If you have Vaegir troops to upgrade, make some Knights to counter the Swadians. Mind you, 50 top tier troops will roll basically any army that the computer fields except for a King's, though. But they have to be top tier, and you will take casualties, so you need to replace them.