I disagree.
You gain a lot of relations by releasing them. More than you get by just fighting them. So if they look like they're going to raid you, fight them, release them. In no time your relations with their family will be way up in the 20s or 30s.
I my current campaign I'm "friends" with dozens of lords just because I've released their family members.