You can get riding 4 from your starting character if you choose the answers right. Let me check. Well, I played with female nomad/apprentice/lady in waiting/forced out of your home. It gives you riding 4 and a spirited courser in your starting inventory. Crappy dagger and a shield is enough to win against a drunk in tavern, if he happens to be around. If you don't plan on investing in archery, which your stats do not suggest, you may consider this or similar start and leave AGI at 7. Not perfect for winning tournaments, but works OK otherwise.
There is a book for Weaponmastery. On the art of fighting with swords? Something like that. Also there are inventory books for Trainer and two medical skills. I forgot which.
Inventory Management scales with INT stat. If you want to increase INT a lot, you are better off putting points into everything else. Inventory Management does not give you anything worth having. More place for thrashy loot which will only slow you down.
Trainer scales with INT too. Really, seriously, if you want to max this stat go for Engineering and medical skills. At 14 you build siege towers in 6 hours and ladders instantly. At 14 First Aid all your downed companions respawn with 80% health for the next round. Only one in five downed units die, and you get them up in no time because of high wound treatment.
Trainer seems nice, but I tried it and early on you are better off investing in other skills. That's what my powerplaying experience says. Anyway, have a quick look at this thread http://forums.taleworlds.com/index.php/topic,289920.0.html. We discuss here some finer points of what would be the most efficient build possible and I test it in practice.