I have never even seen the option to recruit from castles. I have it in towns and villages.
I know I need positive relations, permission, possibly same religion, etc etc etc I know all that and still I have never seen the option not even in my own castles.
As anyone actually recruited troops...