SceneObj tools | Scripts to losslessly unpack, merge and repack .sco files; externally edit props, AI meshes and terrain with standard software

Users who are viewing this thread

Swyter

Grandmaster Knight
This handy tool suite allows viewing and editing the inner contents of your Mount&Blade 1.011 and Mount&Blade: Warband scene object files (SceneObj/scn_*.sco) as if they were normal .zip files, and then munging them back!

These files contain mission objects (scene props, flora, items, entry points), AI meshes (for agent pathfinding) and terrain/ground layers (in the case of exteriors). All of them can now be opened with standard software like Blender, GIMP, Photoshop, ImageMagick and a plain text editor.

Note: These tools are still in their infancy and they may easily break or corrupt your scenes, or change them in unwanted ways. It is a very good idea to either work with version control (Git, Mercurial, SVN, Perforce, ...) to easily revert changes, or at least make backup copies of your .sco files before using this. Maybe working in a separate folder.
Please help test it and upload your sample files in case something didn't work as expected. Your feedback and suggestions are appreciated!

  
download.gif
You can download the latest version of the unpacker and repacker below. Scroll down for the date.
https://github.com/Swyter/mab-tools/releases/tag/swy-sco-tools
 
Last edited:
This is something that I've been wanting to do for a while, and now it's here. Unfinished but still useful. It has been beta tested in the Discord for a few days, and there have been some nice screenshots of progress during its development.

Here is a sneak peek of what you can do with it:
unknown.png
unknown.png
unknown.png
unknown.png
unknown.png
unknown.png
imagen.png


It is a true Swiss-army-knife of scene editing:
  • Import random 3D models as your AI mesh via the Wavefront OBJ format;
    • Supporting quads and random face sizes, not only triangles.
  • Merge props in scenes, view and edit the list of mission objects as a JSON file in your text editor of choice;
    • Move or clone props or AI mesh pieces between scenes.
  • Import and losslessly reimport anything terrain-wise.
    • The recreated files will be byte-by-byte identical if not tweaked.
  • Work directly with the 32-bit floating point heightmaps that the game uses.
  • Work directly with the painted RGB values and texture coverage layers that the game uses.
  • Reindex your mission objects and props to work with a different mod.
    • You'll be able to port scenes between two versions of the same mod, or even between two different mods.
    • It allows you to potentially swap or rename props in an automated way.
    • It can automatically delete obsolete props, no longer listed in the mod.
  • Recombine and match random chunks from different SCO files.
    • You can potentially combine the objects + AI mesh + and terrain, with each chunk sourced from a different SCO file, into a new and improved file. i.e: you can grab the props from scn_a, the AI mesh from scn_b and the terrain of scn_c and merge them together.
  • It allows you to potentially resize scenes without losing your data.
    • Even after you change the terrain code and its properties, no more black border corruption.
This exists to complement the shortcomings of the venerable in-game scene editor. Not to replace it. ¯\_(ツ)_/¯
 
Last edited:
Here are some usage notes from our conversations on Discord to get you guys started; keep in mind that these command-line tools have a help page if you pass the -h argument, as shown below:

https://discord.com/channels/411286129317249035/411291053702774784/1026650352642166845
You either need to drag-and-drop your .sco file onto mab_sco_unpack.exe, or launch it with cmd, which then allows you to do more advanced stuff.

Then if you drag-and-drop your extracted folder into mab_sco_repack.exe it should do the reverse stuff, overwriting the original .sco without asking (it will save to folder name + .sco). That's why I'd do a copy or rename it first, that part isn't great right now.

GIMP and Photoshop should open most of these PGM (grayscale), PPM (RGB), PFM (floating-point grayscale) image formats directly for the most part. But you can use ImageMagick if you want to convert them from/to PNGs and more.

https://discord.com/channels/411286129317249035/411291053702774784/1026660918265909269
To import the AI mesh .OBJ file in Blender and other 3D programs you need to set Z as up. Which is what the RGL engine uses.
unknown.png

https://discord.com/channels/411286129317249035/411291372704890921/1022601929001533552
The unpacking/repacking scripts should be in much better shape, for those wanting to give them a shot. All the features have been implemented, there may be some problems, but I think they can already be useful for editing AI meshes, changing props and doing terrain stuff. I have even turned the repacker script into an SCO remixer, with something like a mab_sco_repack.py ./scn_advcamp_dale -o scn_out.sco --terrain scn_a.sco --aimesh scn_b.sco you can directly copy over chunks from other SCOs. You can even clear part of an SCO with something like mab_sco_repack.py . -o scn_existing.sco -mo empty -ai keep -te keep (i.e. remove mission objects/props, leaving AI mesh and terrain alone, but you can do any combination). Each section allows you to do --aimesh {repack, empty, keep, <sco-path>}, with repack being the default.

Run mab_sco_unpack.py -h and mab_sco_repack.py -h for help, it will show more examples. Please report back.

https://discord.com/channels/411286129317249035/411291372704890921/1022603640579575818
Tried to turn it into kind of a Swiss army knife for sceners. It should be easy to use, for the most basic needs: running mab_sco_unpack.py scn_my_scene.sco, editing the files in the new scn_my_scene folder and then running mab_sco_repack.py scn_my_scene again to overwrite the original scn_my_scene.sco should be enough.
 
Last edited:
Quick heads up, the bugs reported by @kraggrim, @Tocan and @Dalion on Discord should now be fixed in the latest version of the SCO tools. Most of the struggle seemed to be caused by infinite-looping when reading totally-blank ground layer files.

Also improved the mission object reindexer to also load and check plants and items and their IDs against the scene, added support for auto-deleting obsolete mission objects and made everything more robust.

Thanks a lot for testing. Download here: https://github.com/Swyter/mab-tools/releases/tag/swy-sco-tools


--
PS: @kraggrim posted a really cool use of the toolset on Discord; exporting collision meshes for staircases from OpenBRF, tweaking them a bit, and importing them as part of the AI mesh, saving a lot of time and manual work.

Same goes for cloning pieces. Only having to reposition the parts in the in-game editor. :party:

mb1054.jpg
 
@Dalion reported that the latest version of the scene mission object/prop reindexer didn't open Warband's item_kinds1.txt correctly after I added M&B 1.011/TLD support. It should be fixed now.

I have also added initial support for unpacking slightly older (v3 instead of the current v4) .sco files from older M&B versions, like scn_zendar_merchant.sco. The game still seems to understand these, as they are the same but lack the whole AI mesh block.

It should be fixed in the latest version. When it fails exit() should work correctly and return the right error/exit code. When the item_kinds1.txt or flora_kinds.txt exist but are wonky it will report it and ignore them. And I also fixed a problem where it only detected M&B 1.011 item files as valid, but not Warband's. I mistakenly typed or instead of , and because I was always calling it with the same hardcoded files by mistake I didn't notice.

https://github.com/Swyter/mab-tools/commit/c9b87c82916f0364c03fed26045038569f9fa14a

Sorry for the silly mistakes, the tools can be downloaded here: https://github.com/Swyter/mab-tools/releases/tag/swy-sco-tools
 
The latest version includes a handy mab_sco_drag_and_drop_here.cmd file that you can use to drop your .sco files into, and it will call the mab_sco_unpack tool to extract it and create a folder.

If you then drag-and-drop that very same folder it will automagically call mab_sco_repack to convert it back and replace the original .sco. You can also drop the mission_objects.json from the created folder into it to call mab_sco_unpacked_reindex. It should be smart enough to figure it all out.

The window that pops up stays so that you can see what happened and the neat messages it spits. It's the closest thing to a graphical mode, and a great way to come to grips with it without knowing how to use the command-line/terminal. At least for a basic usage.

The tools can be downloaded here, scroll all the way down to get it: https://github.com/Swyter/mab-tools/releases/tag/swy-sco-tools
 
Last edited:
Switched the Python to .exe conversion method from PyInstaller to Nuitka. This should make the binaries faster (now they are turned into native code) and more lightweight (from 10.4 to ~3.51 MB). It should also reduce the number of false positives, some people reported their antivirus was triggered when downloading the files. Let me know how it goes. ¯\_(ツ)_/¯
 
@Dalion asked on Discord for an easy way of massively renaming scene props across the whole mod .sco data, so given that it seems like a very common usage pattern I have included a new mab_sco_mo_remap_all.cmd in the latest version of the .zip file.

Turning the whole affair into a simple double-click! ¯\_(ツ)_/¯

--

Basically, you copy the contents of the .zip into your SceneObj folder, edit mab_sco_mo_remap.txt which has simple lines like this one:
Code:
spr_my_old_prop_name = spr_new_prop_name
itm_my_old_item_name = itm_my_new_item_name
my_old_plant_or_rock_name = my_new_plant_or_rock_name

Double-click mab_sco_mo_remap_all.cmd, and it will use the three tools on its own to unpack the mission objects, rename them if the patterns match, and repack it back into the original scene files, for every single .sco it finds in that folder you are running it from.

The good thing is that this .cmd file is a simple and well-commented Batch file that you can copy and tweak to make it do whatever you want and use as a nice getting-started sample for the whole toolset. The easiest tweak is to remove the --dont-reindex argument, to make it actually reindex, which is done for every element and will be done after renaming in a single pass.

Or even add --removemissing to make it automatically clean up your files; deleting any props, flora and items that no longer exist and may cause problems. Without this it will still warn you once per found element. Sky's the limit.

You can also simulate the SceneObj and mod folder structure somewhere else and just paste there the .sco files you want to modify. To avoid further problems, or avoiding editing some .scos by mistake.

The tools can be downloaded here, scroll all the way down to get it: https://github.com/Swyter/mab-tools/releases/tag/swy-sco-tools

PS: Make sure you have a good backup, in case it doesn't do what you think it does and messes up your scenes.
 
Last edited:
Quick update, forgot to mention the other day that after some feedback I have added support for auto-renaming mission objects (scene props/items/flora) to match the ones in the current .txt files. This arose from a @Vetrogor comment on Discord about stuff like WRECK making the props all-lowercase. And, because the .sco files still have the old soup of names, the game being case-sensitive means it can't match them and ends up doing it by ID/index, spawning the wrong thing and making all that scene work unusable. Like this:

mb118.jpg


So, this version of sco_unpacked_reindex now fixes these problems by default. If your spr_maingate is still stored as spr_MainGate in the scene itself or vice versa it will take care of it. You can disable it with the new the --dont-replace-case argument.

Using it is very straightforward; just extract the program's files in your mod's SceneObj folder and double-click mab_sco_mo_remap.all.cmd and let it run. I recommend making a backup of the whole thing (mainly the precious .scos) first, just in case.

imagen.png


PS: This was also possible in previous versions by just unpacking, making the JSON lowercase and then packing it back. Or using the prop remapping functionality, with one (spr_MainGate = spr_maingate) line per change. Now it's just automatic. The whole thing is intentionally very flexible.
 
Last edited:
Quick heads up. I've made a lot of changes to make it more robust against malformed or older versions, and added a thing to log to a file. I've also added support for v2 and v3 SCO files in many places, instead of the v4 format that the final game saves in. If you see that the tools don't work with some Native files (e.g. scn_random_scene.sco, scn_salt_mine.sco, scn_town_1_siege.sco and scn_training_ground.sco and the oldest of the bunch by far; scn_conversation_scene.sco) is because they are probably ancient and need to be resaved from the in-game editor. The game is still compatible with, and perfectly loads, like four to five obsolete SCO versions.

@Dalion asked for a way to save what you see onscreen into a log file when you run the mab_sco_mo_remap_all.cmd. So I have added another mab_sco_mo_remap_all_with_log.ps1 script that will also save everything into a handy mab_sco_mo_remap_all.log file. Keep in mind that, as this one uses PowerShell instead of plain Batch, to launch it you need to right-click the .ps1 file and click on Run with PowerShell.

--

Still, I think now the whole thing is way easier to use now. Just unpack everything into your SceneObj folder and either use the drag-and-drop thingie to manually unpack or repack, or the _all script to try to auto-fix all your *.sco files. If you only need to process a few, the easiest way is just to make a parallel SceneObj folder and place there what you need, the tools will still auto-find the right mod data.

Easy peasy, if anyone needs help or wants to suggest something I miss, feel free to ask here or on Discord. Happy to help and make it more useful.
 
Last edited:
Added an updated version of the old terrain code generator here. Fixes some problems with the layout, and it should work better when pasting codes of various lengths, as well as when linking and bookmarking, you can go back in your browser to undo:

https://swyter.github.io/mab-tools/terrain#0x1d63c005114300006228000053bf00004eb9

The checkboxes were broken on Firefox, as well as a bunch of other stuff. Now it also validates the format and shows the background in red when you paste something invalid.
 
To make things more visible for anyone not in the chat. Yesterday @iJustWant2bPure asked the following on Discord. I'll paste the response and pseudo-tutorial here for completeness.

Have you ever thought about doing a custom terrain generator that creates more natural looking terrain than the native one?

Swyter said:
This can be done with a flat terrain code and using something that generates heightmaps with hydraulic erosion. I'm sure there is stuff like World Machine and Blender plugins you can combine the with the unpacker/repacker. That's the beautiful thing about the whole thing being modular; you can chain the best software for each step.

With the SCO tools you get the raw/lossless heightmap data in the 32-bit floating point .PFM format. It's important to at least work in 16-bit formats when exporting from World Machine. Or you essentially get a stair-casing effect instead of soft hills. (Because 256 shades of gray/height isn't enough vertical resolution)

Something like this: https://docs.cryengine.com/plugins/servlet/mobile?contentId=25534772#content/view/25534772

I was trying out the best workflow, and it seems like World Machine Basic -> export to OpenEXR (32-bit) / .EXR, and then exporting the .EXR to .PFM via the GIMP works and it's lossless. Just make sure the .SCO heightmap dimensions match, or resize it. I'm sure the other material color exports can be exported from World Machine in a very similar way.

Stratospheric. 🐧
imagen.png


This looks better.
imagen.png


This is the kind of stuff you can do with the SCO tools after some work.
imagen.png



When I have tried in the past the whole surface was just grass, how did you set it up so that the steep sections were different?

Swyter said:
In this case, just using the color export from World Machine as terrain layer paint. Haven't made masks for each terrain material, yet. I think it's just painted grass.


To go from World Machine to this you export the heightmap as EXR and the color map as PNG. You can then download a portable version of ImageMagick, which is a handy command-line image format converter and editor from here, extracting it in the Documents/World Machine Documents folder and run this:

Code:
ImageMagick-7.0.8-23-portable-Q16-x64\magick.exe "New Project Height Output-1025.exr" -evaluate multiply 200 -resize 251x251 layer_ground_elevation.pfm
ImageMagick-7.0.8-23-portable-Q16-x64\magick.exe "New Project Bitmap Output-1025.png" -resize 251x251 layer_ground_leveling.ppm

You need the -evaluate multiply 200 step or the whole thing will only have one single meter of height (it normalizes the range from 0.0 to 1.0, instead of 0.0 to 500 meters, or something). If you make the scale too big the terrain will spike like in the first screenshot. The game calls ground_leveling to the painted RGB colors, it's a misnomer, which I kept.

Repack and re-enter the scene to reload the SCO data and see the results, you don't need to restart the game. Just switch between two scenes in-game.

Here's the matching World Machine terrain for the example.

Original one as seen in the viewport.
imagen.png


In-game.
imagen.png


Put a random e-mail to download the basic version from here, they don't verify it: https://www.world-machine.com/download.php
 
Last edited:
By the way, in the latest version of the SCO tools from today, and as a bit of retro-compatible usability improvement, the unpacker spits and the repacker accepts «layer_ground_tinting.ppm» as a stand-in filename for TaleWorlds' «layer_ground_leveling.ppm» which as a layer name sucks and is practically misleading, as people confuse this RGB color tinting data with the «ground_elevation» gray heightmap. Me included.

Any unpacked folder done before this change will still work, as the original layer filename is still detected and should still repack fine. Just keep it in mind, if you are searching for it and scratching your head.

But yeah, TL;DR: leveling is now called tinting.
 
Back
Top Bottom