Cavalry are, indeed, very strong in this game. It also doesn't help that AI lords typically will not have a strong army (bigger than yours pretty much always, but the troops are all over the place with usually 10% or less top tier troops). The AI also does not use very good tactics while battling which adds to their problem. They basically will take the brunt of a cavalry charge every time and die miserably doing so.
As the player, you can play around this a bit because the enemy cavalry will charge* at you. If you let the enemy party follow you near a cliff and then battle, you can abuse the terrain to take out their cavalry with your infantry. You can also do so in spots where enemy archers can't really reach you.
* There is one time they won't - when they are outnumbered and have a lot of archers, the AI will sometimes set up their troops on a nearby hill and wait for you. If you really want to play cost effectively, you can stand just inside their range that they shoot with your shield up and wait for them to run out of ammo lol. Then they will charge with all their troops.
You can definitely win wars through brute force. The main thing is you also need some tactics to go along with it (not the tactics skill - though it does help - just battle strategy). It is the primary strategy I have used for a long time because it is hard to get a lot of lords to join you and also keep them all happy with nice distribution of fiefs.
I say you should stick with this a bit longer. I kind of like the times where you are desperately trying to protect your kingdom. It is rare for me to actually be in a position where I am legitimately worried about the outcome. However, if this is not your thing, then restarting is a fine option. You've likely learned a lot from this save file. On top of that, you can always come back to it if you want to

I also like to have a separate save file where I am not planning on going for a kingdom or becoming a vassal. It's nice to be able to run around and do whatever I feel like doing.