I don't think archers are OP, but I do dislike archers, although it's just personal preference.
I dislike archers because quite frankly, their presence make the game more challenging without making the game more fun. As an infantry, I enjoy fighting other infantry, even those who are a thousand times better than me, if only because the action is engaging and overwhelming. With all the attacks, blocks, feints, holds, and fancy footwork, the whole play experience is an active one that is constantly fun. Versus cavalry, they have to charge me, and even if I'm caught with only a short sword and no shield, fighting a cavalry involving awareness of when they charge you, dodging, blocking, and finding good opportunities to attack. Again, the play experience is an active one that is constantly fun.
And then, you get an archer who starts shooting at you from halfway across the map. What do you do? You know that from this distance, most of his shots are going to miss, but if he concentrates on you, eventually he'll adjust his aim enough to hit you, so you can't completely ignore him. So you start zigzagging and running in weird patterns, but now you're doing a lot of work when he's just casually staying there, and the advantage is STILL his. You're then faced with a dilemma. Do you go and hunt this archer down or try to fight other people while hoping he won't shoot you in the back? If you choose the latter, every now and then he DOES shoot you in the back, and you regret your decision. Whereas, if you go hunt him down, you have to endure the long boredom of walking slowly (with your shield up) as you close in on him. At this point some annoying archers might turn around and run, and you'll curse the fact that he runs faster than you (or at least the same speed). If you can force the archer into melee, it's more of a fair fight, and I wouldn't mind if the archer is some amazing melee person who totally owns me at my own specialty as an infantry, but the very fact that I had to do so much work just to start the fight, the very fact that he has the advantage for such a long time before I can even attack... that gets frustrating.
It's even worse when an archer is shooting at you from halfway across the map, and you look and realize he's on top of some giant mountain or some house's roof. Now to get to him you'll either have to slowly climb up that mountain or find the tricky paths that lead you on top of that roof. By this point, you might as well just give up, stay out of sight of that archer or make the shield a permanent part of your game from now on.
And as a cavalry versus an archer, it's more of an even match... unless he's camping at some high spot your horse can't get to.
That said, if I do things well, archers are just as easy or hard to take down as any other class, and everything does seem balanced. Plus, it's a medieval warfare game and it just doesn't feel right without arrows flying everywhere. I also likes to play as a ninja archer every now and then, moving around discreetly and shooting people in the back, so I'd be a hypocrite if I complained too much too vocally. Archers just punish laziness a bit more than other classes do, that's all.