I have only tried this on family members, including my wife. I go to a castle or town and I choose to leave my wife or my sister or brother there. Right away they leave the settlement I chose to leave them at and go somewhere else. This means you can't leave your party members in a settlement of your choosing. Party members you leave in a settlement should stay there until you pick them up again and not move around to other places.