You do know that every level reduces you learning speed for all skills permanently right? So trying to powerlevel/exploit dum stuff like charm and athletics actually retards (what word actually means) all future growth in more important skills that you would actually benefit from.
Athletics is fine later when you've got high riding, weapons of choice and other skills you need but grinding it early is a bad idea dude.
well from the builds i have tried. honestly you dont need to max anything in riding. first couple levels are fine. no millitary bonus stats for your men really. you can get a desert horse which is fast and does what you need. as for slowing progress with all the builds i have tried focused or not the progress is about the same by the time you get to that level. so when i leveled pure 2 hand and athletics for instance my 2 hand was still low and athletics still around perk 3 by lvl 11.
with builds taht focus on trading and charm ect i dont require a horse because its all auto resolves. athletics no points about perk 3 at lvl 11. so it doesnt matter so much.
if you want to focus on any weapon stats you either do ranged or 2 hand they are the only ones worth anything to your army.
if you want your units to be stronger in battle, athletics leadership and medicine.
personal defence atheltics medine seige and scouting.
there is really only 3 options you either focus only on your character. only on your army. or your focus on trade.
getting 50 points in charm isnt wasted or slowing your progress by that much and it allows for more relation. more renown from perk 2 which levels your clan which speeds up the games development.
to be frank it seems relitivly useless to use anything but social and intelligence stats for anything. leveling atheltics as you travel.
the only reason to focus on weapons and smithing is for a solo character . since you get massive damage buffs ect but just having movement speed is enough to win any fight which is why you probably like horses so much.
charm is probably one of the top stats in the game either way becasue it opens up more troop recruitment early. easier quest completion early. so more gold. then later it helps you get castles and get allies.
the only reason to level riding and any weapon ability is purely if you want to make a character thats able to turn tides of battle on their own. but you can do that without the buffs either way. you dont need stats in polearm to lance someone or 1 hand to use a shield. and no weapon has any required stats to use.
i would consider atheltics one of the most important abilities in the game so idk why you wouldnt level it first. it single handedly will make you able to win any fight on foot. aswell as buff your troops allowing them to win many more wars. medicine would be a close second as the healing to your men allows for more fights and the hurt soldiers you get back even with a .8% chance is crazy. my losses went from like 40 too 10 with not even a 1% chance . allowing for more experianced troops. who are now buffed with bonus hp and damage