You need the high Trade perk that allows you to barter fiefs - then you can gift (or trade) fiefs with whichever lord you like.
If you don't want to grind up the perk but don't mind a bit of cheating, use the export/import mod give yourself the perk, trade the fiefs you don't want, and then use the mod again to remove the perk.