Funnily enough ever since I started the game up again after the last post Bannerlord has not crashed once. If some dev already suspected the culprit issue and fixed it, thank you. If it comes back I will post here again.
Edit: it has now crashed again the same way on low-population tdm server (this happened about 1 minute after this post, lmao)