Well, the faction you're robbing won't be too pleased as you can imagine, if you can handle the backlash it's just fine. Just make sure you don't exaggerate or there will be a personal crusade against you by the other faction, and in that case you'll be on your own (your faction won't help you in battles unless they are hostile to the other party).
There are means to make peace with an affronted faction, but they can be somewhat expensive. It's your game, so do as you please
The only problem is what the above poster said, and also your king will get mad at you. He will say that you are giving them reason to attack you. But if you don't care about losing a few points with him and the other faction, go for it.
Hiho, I'm sorry but I don't recall where we know each other from - can you tell me your ingame nick (or just send me a personal message please)? I'm better at remembering ingame nicks/situations than forum nicknames
On topic, if you actually do raid the caravan and a personal crusade against you should start (and it's too late to make peace with your enemies by bribing one of the enemy lords), you could attempt to get your faction to go on war against the enemy faction, but it can be somewhat hard to do so. Or you could just destroy the enemy faction yourself (but that's not really an option until pretty late into the game). Gotta love how much freedom this game grants you in this regard