I find the Elves very scary in the field - the Elintoran Knights, Moon Knights, and Elites are powerful and I hate fighting Archduke Berlaeduil and Queen Arwen in open field combat, my knights are just weaker.
But oddly enough, I've had extreme success in siege combat of all things. The Tolranian Lords I poached from King Torlian actually field surprisingly good archers, and I also poached Valthauionnon and Merisirduil from the Elintor. I've also still got quite a large collection of my own Elintoran Rangers.
I just move cavalry in front of archers and infantry in front of cavalry, while keeping Alindel (Surgeon 10/Wound Management 10/First Aid 10), Slyter (Engineer 10, Spotting 10, Pathfinding 10) and my Elintoran Master Scout (gives me Pathfinding 10+4) as far away from the action as possible;
[Walls = '+']
[Infantry = 'I']
[Cavalry = 'C']
[Archers = 'A']
[Support = 'S']
+++++++++++++++++++++++++
+ Filthy Elves are here +
+++++++++++++++++++++++++
---------------------------------------------
---------------------------------------------
---------------------------------------------
-------IIIIIIIIIIIIIIIIIIIIIIIIIIIIII--------
---------------------------------------------
-------CCCCCCCCCCCCCCCCC--------
---------------------------------------------
-------AAAAAAAAAAAAAAAAAAA--------
---------------------------------------------
---------------------------------------------
---------------------------------------------
----------------------S---------------------
---------------------------------------------
It also helps massively that I bring a Darkforest Ranger Bow, 100 arrows, I have 430 proficiency, and I poach ammo liberally from my archers.
All companions are equipped with Bladed Crossbows/highest level bows they can wield (Zaira can use the Darkforest Ranger Bow too)/or Leaf Bombs.
I can't conquer the towns/castles in a single go though, because the Elintoran Noble Maidens are a very lethal infantry line and slice my knights/infantry into ribbons.
Instead, I clear all the archers I can from the walls, making sure to spam Battlecry for the sweet 42% group heal every time it's off cooldown, and retreat when my guys run out of ammo.
Trying to make the Big Push(TM) to win the siege in one go is always extremely costly because the AI is much better at defending from inside the walls than ON the walls. What happens, is if I use my Kingsman Poleaxe to knock out a Noble Maiden or two, the gap in the line allows 30 Rangers still at their spawn point to pincushion me with arrows. If I send my infantry/dismounted cavalry up, they just take insane losses as every gap from a dead Noble Maiden is filled with a hailstorm of arrows.
No, I find it's considerably more effective to fight from outside the walls, oddly enough, as that means more of my archers are firing at them than their archers are firing at me. As soon as I scale the walls I lose that advantage and sustain massive losses.
My archer line used to be comprised mainly of my own Elintoran troops, but attrition has meant that more of my archers are now Perisno Darkforest Rangers, rescued Tolranian Archers, and rescued Zann Longbowmen/Kheshigs/Swifthunters (I'm allied with the Zann). They're a bit weaker than the Elves but not THAT much weaker, and they get the job done.
I bring around 370 men, and occasionally 2-5 allied lords, and I'm winning. But only in sieges.
Open field combat is much more dangerous. The Elves excel at destroying my knights in hilly terrain. Oddly, I win easily in VERY long range archery matches or by directly ramming my entire army into them at close range on flat fields, but in mixed chaotic skirmishes on bumpy uneven ground where Elven horse archers can shine (see: most of the crappy terrain north-east of Lille), I take huge losses even when I bring a much larger army. Berlaeduil and his Moon Knights can go eat a gentleman's sausage.