They're not invincible but they're definitely over-powered compared to infantry especially in terms of game mode mechanics. I'd argue that archers are more broken and harder to counter than cavalry in my experience, especially on TDM where the spawn system means someone will always eventually spawn behind you, leaving you as a free kill if it's a cavalry or archer.
Cav definitely get spammed too much in that mode too, to a ridiculous level. If you play on East Asia TDM just look at the scoreboard and 90% of the time the top players will be cavalry or archer. On occasion 90% of the team will consist of only archers or cavalry too.
Oh hey I play East Asia too! Yes people spam cavs a lot there. As for the archers, learning to watch out for sniping points really helps. Walk close to cover, watch out for high grounds and dark places (people love
shooting from the shade), etc. It's like how you finally get used to the horse sound and charge+stab tempo. It's certainly something that can be learned, and personally I don't have much problem with them. The spawn shot is indeed hard to deal with tho.
From my experience, infantry players just love two handed weapons too much. Using shields is seen as a noob move or something. Well, not using a shield is how you get shot.