I've updated to 1.6.2 today and caught the Ghilman leaving the Northern Empire's service in the problem save file, didn't crash. Let the game ran for 2 hours after that as a test, no crashes. Should've crashed multiple times by that time. I haven't seen anything about this in the patch notes but I guess this is fixed somehow?