Well, the faction you're robbing won't be too pleased as you can imagine, if you can handle the backlash it's just fine. Just make sure you don't exaggerate or there will be a personal crusade against you by the other faction, and in that case you'll be on your own (your faction won't help you in battles unless they are hostile to the other party).
There are means to make peace with an affronted faction, but they can be somewhat expensive. It's your game, so do as you please
