A lot of people on this board keep comparing the high end units of each faction, which really is the wrong way to look at it.
When you look at the Rhodok standard army, you'll find a lot of high tier units. This makes siege very difficult, but more importantly than just having tough sharpshooters and sargents is the fact that all the lower tier troops use piercing weapons as well. Even a spearman can hit you hard through lordly plate, and there are tons of spears in all those battles.
The Khergits might suck on foot, but they also have loads and loads of the highest tier troops, almost all of them will have ranged weapons. Their high tier troops have good armour, and generally fast weapons. This can mean in some castles you'll roll over them, but in others you'll get whipped unless you have lots of troops with the thickest armour to endure large volleys of arrows. There are a few very tough Khergit castle designs.
I find the Vaegirs one of the weakest AI defenders. They have very few marksmen in their armies, and their mid tier archers are just ok and do poor in melee. One thing consistent with them is that they use a lot of slashing weapons, since they are the worst against armour you can do quite well against them with the right army, they also wear lighter armour than most, especially in low tiers.
The Nords are a mixed bag, we all know their archers suck and their melee rule, but what decides the win or lose is their army composition. Even the most seasoned Nord army doesn't have more than a couple Huscarls in the pack, the longer you wait the more and more warriors and veterans build up(you don't need to worry about the others really), and they are plenty deadly. This is a very good early attack option, maybe a second option.
The Swadians probably have the weakest units as an overall army, but there is an annoying oddity about them. While Knights might rule they are similar to most other faction high end units, there are a couple in an army, and the man at arms aren't impressive on the ground. Their infantry line is weak, as is their archer line, in fact the bulk of their army always seems to be in the militia phase. If you've tried to train up Swadian militia you'll probably notice how it takes forever for them to switch over too. While they are one hit kills they also carry a crossbow and military pick... either one of those can inflict far more damage than anything at that tier of troop should be able to put out. They are actually the only army I find my heavy armoured heros going down far faster than they should, I'd leave them till round 3...
My order of attack would be Vaegir/Nord (in place 1 or 2) , Swadian, Rhodok/Khergit (in place 4 or 5). This is dependent on what troops you use to attack, how long you wait to attack, and the forces in castles when you start attacking. Vaegir and Nords also have superior physical land layouts for defending, a very major issue when you are not aligned to anyone.