To anyone concerned
I found out that you need lords permission to recruit in towns owned by one. You don’t need to have his (lords) permission to recruit in villages he owns but you still need to have permission of village elder.
So if anyone from Brytenwalda team is reading this post. Try to implement feature that allows to recruit in villages after lord agree for you to do so, no matter village elder agreement.
Apart from it, it is a great game, keep it up!