Or maybe, like the current system in Guild Wars, where you can choose where you want your cloak to be worn. You can wear it just in battles, just in towns, in both, or in neither. All of us realistic people would set it to "In Town Only" whereas all the people who wanna look kickass as well as kick ass would set it to "Battle Only" or "All."
It would be rather nice, if we could adjust where we want it, for realism/coolness sake.