Companions are idle in castles when giving a fief first

[Basti87]

When you give a companion a fief first, he will stay in a castle / town forever.

What are you and your men doing ?
"We are resting at xy. I need to raise some men before attempting anything else. I must attend to this matter before I worry about the affairs of the realm"

Besides he has 2 castles.

They also refuse any actions options
"I am sorry. I need to attend my own business at the moment"

 

[Basti87]

Well I found a solution.

You have to activate the cheatmenu and use the CHEAT: Let us return back to a friendly town.
It's the only method to get them out of a castle or town. they will fill up their troops and roaming around..... actually that what a lord normally does.

Edit: I Guess it was because of their low renow, They couldn't recruit troops and so they stuck at my castle forever.
When I start my kingdom of my own and accept a lord they don't get this issue.