Not to mention i will also pay him to fight with me and it is looking like my army/ faction/ kingdom is more likely to win if he is my prisoner. They should make it so i spend more money converting him.
Well, when 50% of my army starts becoming people from various defeated enemies, there should be some consequence. Currently having prisoners is an effective recruitment path in longer campaigns.
I think how it should work is, ownership of a fief and to a lesser extent faction association, supersedes relationship blocking recruitment. Otherwise, how do starting independent players recruit? That or we need separation.
Why should it be easy for an independent player to recruit people to fight for him? A fief lord can use his levy because of the ideas that bind the village community to the kingdom, but an indipendent player is a nobody, basically. I don't think it was very common for farm hands to join some errant knight and fight for him. It takes a certain sort of man to fight for a living, and not fight for the things he loves only. Independent players could recruit mercenaries like I said in point 5. These mercenaries could be found in cities in greater numbers than currently in the taverns. Perhaps there couls be bands of mercenaries, like looters, which you could hire also. Maybe the current main unit tree could be made into the mercenary tree. I don't think you should be recruiting from villages unless that village has some feodal connection to you.
I liked how mods in WB made it so castles effected party size. Bring that back and give me a reason to want castles for once.
Party size limits are totally arbitrary. Units should be so expensive that their cost would be the limit of having them. This would force the player to raid villages, which is what warring armies did constantly in times of war.