Hmmm, if there's ever a feature that allows you to disarm, or be disarmed, or your weapon breaks, or get into a drunken duel in the local taverns, including the ones in M&B, it could have relevance. Personally, I only use it to slap the merchant around in one of the quests for fun.
The rest of my two cents - strength, agility, power strike, and maybe athletics should influence the effectiveness. I don't think any of the weapon proficiencies would be very applicable. One-handedness would be an almost-but-not-quite relevance towards it. If it needed a proficiency, it would have to have its own.
*light bulb above head* A disarming skill only usable when unarmed. Surely, you'd eventually need to rough someone up trying to get close enough. No super ninja abilities or anything like that.
Imagine getting an iron gauntlet hay-maker from someone riding by on a horse. I usually end up running the offender over with the horse instead. I dunno, there could still be a niche for it somewhere.
When I mentioned a "slower" punch, I don't mean like pushing-the-dude's-face-to-the-side slow, just a slightly slower wind-up.