It comes with a console command to immediately clear all parties inventories but i doubt that this is the issue/solution here.. I had that scouting xp fps drop too after some battles and with big tracks around but fortunately it didnt lead to a crash for me and i could just sit it out.
I would try to enable cheatmode and teleport the hell out of that area (ctrl + left click on map) to prevent the scouting xp check to run. Maybe thats enough to prevent a crash until the tracks are gone if thats the cause of the crash. I guess other parties around there will still cause lag until the damn track is gone though but i hope it reduces the lag enough to prevent the game to crash. No guarantee that it works. Im basically talking out of my ass here
If this doesnt work try to install butterlib so you get a useful crashreport and share it here. Since you already tried using mods taleworlds will only tell you "we cant support modded games.." (in about a month)
Edit: Another idea is to type "campaign.multiply_campaign_speed 0.5" (or even slower; everything above 0 should work) into the console and enable "fast forward" on the campaign map. This causes your game to run at half the speed and should in theory half the cpu load. Then take a shower or grab a beer or something and see if your game survives this time.. again no guarantee just throwing in ideas. Just remember to type "campaign.multiply_campaign_speed 3" (or whatever you prefer as multiplier) afterwards, in case there is an "afterwards", so that "fast forward" isnt slowmotion anymore