I think it's very subjective to say that this or that way of fighting is better. It depends greatly on how you play, what you like to do, what you're good at, and of course, the quality of your troops. Both mounted and unmounted combat can be challenging. I've read a lot about swadian cavalry being overpowered, yet with the underappreciated Rhodok spearmen and crossbowmen I've managed to counter their charges quite effectively. Likewise, the praised nord infantry has yet to prove an actual menace. Now that I've incorporated some cavalry to my army, they don't stand a chance (and no, I don't play on easy difficulty).
I believe the trick to an effective unmounted army is to commandeering them correctly; you have to be more a general than a warrior. Once you enter the fray, you can forget about giving orders around. Here is where being on a horse gives you an edge, since you have a better view of the battlefield, plus the mobility and capacity to effect skirmish moves before enemy groups reach your own and the ability to quickly reach places where your help is needed once the real battle begins. So in the end, I think this is the only actual advantage that being mounted gives: a tactical advantage, which is no small thing, but still just one thing. The rest can be compensated.
Also, know your enemies. If you go into a battle without checking what the rival has, you might have a few surprises. First and foremost, learn the troops of each faction, their strengths and weaknesses (this, I suppose, is rather obvious). Second, check what the enemy lord has in his army before engaging him. Sometimes they might attack you by surprise while travelling on the map and not allow you to do this. Well, you can still get an idea of what they have by scouting their troops once you enter the battlefield (even if you don't recognize their troops precisely by their appearance). Needless to say, a horse will come in handy. Even if you like to fight on foot, you can have a horse just to scout (given that none of your damned lackeys will do it for you).
As far as troop cavalry is concerned, I find spears to work quite nicely. The longer the better. If you're an average archer, when a horseman charges you, you can wait until the very last second and put an arrow in his head, which is always fun. For your troops, just put them together on an elevated position, but beware of their infantry because, as far as I can tell, in this scenario the ones that are in the lower position have an advantage (I suppose it's because they hack away at their enemy's feet beneath the shields). Otherwise, rivers, steep hills and forests are a pain for cavalry, so use them.
As in real life, usually the best army (as far as this subject concerns) is the most balanced one, because it has greater versatility when addressing a battle. Plus, it makes the game more varied. Once you get a castle, you can store troops and change your marching army depending on what you'll face.
Those are my slightly extensive two cents about the issue.
and swamp cavalry charges. The last one is very, very fun. Also, horses cost too much to keep buying more; you're better off saving that money for weapons and armor for your companions. Besides, horses can't be used in sieges.
