If the lady was commanding a party, it takes a while for them to make the journey to you while their party disbands. Sometimes they can be reported as being in one of their old towns for a while, but you can't see them there. If you own a city or castle, they'll head there, otherwise they'll head to the city your brothers and sister went to when you completed their quest.
Sometimes they bounce about a bit on the way to you, but they will get there in the end...