Rhodoks are annoying, because of their spears. They kill me from a distance if im on foot, and they stop my horse if im on one. Also, their crossbows. And their shields. Being an archer is useless against them. Khergits are very annoying because they scatter and you have to spend lots of time to just get one. Also, horse archers. Nords are annoying, because suddenly throwing axe + face. VERY often. Swadians are annoying because if they have more than 40 knights + some men-at-arms, they steamroll you. No matter what. Sarranids are annoying because of their mamelukes, quick firing archers and well armoured infantry.
Vaegirs are annoying because their marksmen kill half your troops before you even get to them. And their knights. Bleh. Bandits are annoying because they have all kinds of weapons and they swarm you. Farmers are too easy.
Did i just say that everyone is annoying?