In Progress General Worldmap compressed_shader_cache

[NPC99]

Sandbox Main_map's shader cache includes a compressed_shader_cache.sack in addition to terrain_shaders_header_data.bin. Any worldmap saved from the scene editor only saves a terrain_shaders_header_data.bin file. Running a custom worldmap in-game without a compressed_shader_cache.sack fills the rgl log with missing shader warnings (i.e. pbr_terrain, pbr_terrain_gbuffer, pbr_terrain_shadowmap etc.). The rgl log implies that sometimes the missing shaders are compiled on the fly and sometimes not. Running a custom worldmap with a copy of the Sandbox compressed_shader_cache.sack eliminates missing shader warnings from the rgl log. Clearly, we need to copy the sandbox compressed_shader_cache.sack into our custom worldmap's shader cache. However, every time we save a worldmap from the scene editor, it wipes the contents of that scene's shader cache and only writes a terrain_shaders_header_data.bin, requiring a manual copy and paste after every edit.
I understand that Taleworlds only built one worldmap, but modders need scene editor saves to include a compressed_shader_cache.sack, if the scene is marked as a worldmap, or for the scene editor not to wipe everything in the scene's previous shader cache and only overwrite its terrain_shaders_header_data.bin.
This issue may be further complicated when modders can create custom shaders, which will not be included in the Sandbox compressed_shader_cache.sack. I have no idea how to create a custom compressed_shader_cache.sack.

EDIT: On loading Sandbox Main_map into the scene editor merely to inspect its Navmesh, without saving it, the editor appears to have deleted its compressed_shader_cache.sack on exit. If correct, that's even more worrying.

 

[Murat Türe]

The compiled shader cache files are added to the build at the publish stage of the game. It takes time to compile all shade combinations so we do not do it at save time. Also, since the terrain files are in most cases different, the re-save of the scene invalidates them. It is in our task list to add that shader compilation task to the publish button in the editor so that the finished mods will not have spikes or long loading times because of missing terrain shaders. If you have any other questions, I would gladly answer them too.

 

[NPC99]

The compiled shader cache files are added to the build at the publish stage of the game. It takes time to compile all shade combinations so we do not do it at save time. Also, since the terrain files are in most cases different, the re-save of the scene invalidates them. It is in our task list to add that shader compilation task to the publish button in the editor so that the finished mods will not have spikes or long loading times because of missing terrain shaders. If you have any other questions, I would gladly answer them too.

Hi @Murat Türe , thanks for your swift reply. Great news that a fix is already on TW's task list.

I've only worked on worldmaps, but I can see from inspecting khuzait_castle_002 that all scenes need a
compressed_shader_cache.sack. The one for Main_map is 636Kb while that for khuzait_castle_002 is 3,162Kb,
suggesting considerable differences between scenes. Do they hold more than terrain shaders? I'm having
difficulties implementing custom materials in my worldmap without crashes. Clearly, my copying across
Main_map's compressed_shader_cache.sack to my map only helps for things that are common between them,
ignoring anything new I've added. Would custom materials be covered in a compressed_shader_cache.sack
or are they entirely unrelated to it?

 

[Murat Türe]

These files only contains the compiled shaders of the terrain nodes. Thus, the number of unique terrain nodes (unique with respect to the layer combinations it use and some other properties), has a direct influence on the size of the of those files. Absence of these cache files should not incur crashes for the game, only downside would be long load times and spikes throughout the game. If there are crashes, I would need the logs to find out the reason.

 

[NPC99]

These files only contains the compiled shaders of the terrain nodes. Thus, the number of unique terrain nodes (unique with respect to the layer combinations it use and some other properties), has a direct influence on the size of the of those files. Absence of these cache files should not incur crashes for the game, only downside would be long load times and spikes throughout the game. If there are crashes, I would need the logs to find out the reason.

That's good to know. I was trying to create two additional variants of the map_icon_empire_city_01 material, so certain empire settlements would retain their native colour schemes while others looked different. The new materials were identical to map_icon_empire_city_01 merely substitute two alterative diffuse textures. These alternative materials were then assigned in the scene editor to each wall, gatehouse, tower, keep, arena etc. resulting in a few thousand <material> references being added to the scene.xscene file. I'll retest by updating a stable version of my worldmap and upload any crash logs.

 

[NPC99]

@Murat Türe as you can see from the following screen shot, I was unable to upload the crash report.

Spoiler

However, I've uploaded the 32,143Kb crash zip file to this media fire link https://www.mediafire.com/file/1lsu....47_8dc9d29e96d7b033b77c7b0bcddc6b26.zip/file

As previously, crashes occurred after entering an inventory or trade screen. Thanks for looking into this and let me know if you need anything else.

PS I had cheat mode enabled to teleport around the map for testing.

 

[Jance]

Just noting that the issue being described in this post is the same one in this ticket (NPC is developing our map )