Siege enemy castles first. Start with the closest to your border. If you can't take castles yourself, start a campaign, wait until a few lords join you and try again. After you've taken every enemy castle, start on their towns. And once you've finished those you may have to hunt down the last few lords wandering around the map. If you can't find them use ctrl-t. A window should pop up stating the faction was defeated if they don't have any units left.