A simple fix could just be to have the faction banner fly over the clan banner. That way you can still tell at a glance what faction a lord is a part of and clan banners can still be whatever color you want it to be.
That's what paintings on shields are for, but then you notice what everybody's complaining about, me included: this banner system is useless, it makes individual lords and clans visually unremarkable.
Look at this map:
Most lords have distinct charges and patterns in their heraldry. From the colour you can see who was loyal to the king in Paris and who wasn't (the red trend was probably started by the Duke of Aquitaine). And this comes from before heraldry was a thing. Before heraldry and inherited sigils and regularised coats of arms there were simply colours marking alliegance.
You can see here that there's a blob of blue supporters of the Capetian line. Another blob of red nobles who traditionally opposed them. Another one, the Duke of Normandy, who also goes his own way, and the Duke of Brittany used to go his own way (ermine, black on white) but the new Duke is now a companion to the King of France, Pierre de Dreux (from the house of Vermandois)
All the fleurs-de-lys signal members of the royal family. Meanwhile the Duke of Champaigne and the Duke of Bougogne, the Lord of Lusignan, the Lords of Valois, Dreux and Vermandois, they all signal allegiance with that.
So, why not implement something similar?
Distinctly Vlandian. Red, gold and white (Gules, gold and argent, if you want), lions, roses, keys and horseshoes. Distinct as well. Why not?