Same boat, it happens at random. Sometimes launching the game, loading a map/mp server, leaving game, and launching it again helps for some time, but sometimes all goes well and suddenly Bannerlord starts eating up RAM until it crashes cause there is no more RAM to use.
It can happen an 5, 10, 15 or 60 minutes after launching a game, but more often the problem shows up within first 15 minutes be it single or multiplayer.