That makes sense to have them congregate. Maybe have a low chance that one will get scattered rather than return? (i.e. some may be captured and become separated). And if you didn't want to go find them yourself, maybe have a companion mission to go "Rescue _____"
Just thoughts!