I didn't think there was any difference between the two as far as quests were concerned... Anything one faction will assign should have a mirror from the other faction (Capture Swadian Nobleman/Capture Vaegir Nobleman, Raise 5 Swadian Knights/Raise 5 Vaegir Knights, bring me 8 Coursers/bring me 8 Coursers, etc)
I could be mistaken, though, I've never been on the Swadian side...
The main difference between them is in troops and how they're equipped... For instance, Vaegir Knights almost always ride a Warhorse, and carry cutting weapons (swords, halberds, etc), while Swadian Knights use either Hunters or chargers (perhaps some Warhorses), and will often come with lances or morningstars... Some also say that Swadian archers are better than Vaegir ones (at each respective level on those branches), though I'm unsure if this is really true (I do know that at their highest level, Swadian Sharpshooters almost always equip themselves with crossbows, while Vaegir Marksmen have a good chance of equipping a longbow, which is faster firing)...
So knowing the differences between each faction's troops will tell you which one you'd rather have on your side, and which one you'd rather be fighting against...
Personally, I went with Vaegirs, but not really for the above reasons... I had a bunch of Vaegir Knights in my party, and I noticed that most of them were wearing the same armor and shield as myself... So, I just decided that if I looked like one of them, I might as well join them! lol