Scene .SCO Toolset - heightmap converters, object placers, and texture importers! Oh my!

Users who are viewing this thread

Hey guys, I'm trying to define a scene terrain mesh from a heightmap. According to mbmodwiki.ollclan.eu/Scene, the SCO file contains information such as how the terrain mesh is modified. Does it have to modify the terrain mesh or can I use the SCO file to flat out define a terrain mesh (e.g. from my heightmap)?

Attempting to satiate my curiosities about the mysterious SCO format, I've managed to parse it using the code at mbmodwiki.ollclan.eu/SceneObj. I then added a hill to the La Haye Sainte scene using the default editor. My train of thought was to parse and compare the two SCOs and see what changed. Well even with a relatively small change the SCO files diverged significantly, so I was wondering if anyone knows how to define a scene's terrain mesh from a heightmap?

EDIT: Solved. Changing title.
 
No, you can't use the SCO file to define a terrain mesh. The terrain mesh is defined from the terrain code in scenes.txt, and the SCO file can only change it.
What you can do is use a completely flat terrain code and then alter the elevation in the SCO file based on a heightmap.
 
Thanks for the reply, that wiki seems to be filled with predominantly your contributions and it's already been very helpful.

Would you mind explaining how the SCO represents modifications to the terrain code? I assume the mesh is first generated procedurally from the code and then each alteration is stored somewhere in the SCO. I don't think the alteration is stored as a mission_object_t as I didn't see the number change after drawing some hills, and neither ai_mesh_t nor ground_paint_t seem like appropriate places (though they did change).
 
It's ground paint. If you look at my C example in the wiki, there are several layers of ground paint. Most are what you'd think they are (paint), but there is a layer with a special ground spec no (GROUND_PAINT_ELEVATION_MAGIC) that describes height difference from the procedurally generate terrain.
 
Got this working with y'all's help (textures too): http://steamcommunity.com/id/disgrntld/screenshot/594714782049253346

Here's the scoWriter I made to use in conjunction with scoReader. I also made a few small changes to scoReader, so that's why it's a tarball: http://dl.dropbox.com/u/70112191/scoUtils.tar.bz2

The bundle still has some problems, specifically AI meshes, but I figure those are easier made inside the editor anyway so I probably won't invest the time to fix them.
 
I'd be very interested in testing this, as I'd much rather do some bitmap edits than have to jack around in the ingame editor, but it failed to compile under VS 2010. 
 
Hmm, maybe it's the lack of header files for the reader and writer. Try the link again, it's been updated. At least for me, it compiles cleanly with:
Code:
gcc -Wall -o testWriter testWriter.c scoReader.c scoWriter.c
 
Original message:
You've tidied it up,... nice!
Do you mind if I mirror it on the wiki?


I'm yet to get a working heightmap re-importer.



Disgrntld said:
Hmm, maybe it's the lack of header files for the reader and writer. Try the link again, it's been updated. At least for me, it compiles cleanly with:
Code:
gcc -Wall -o testWriter testWriter.c scoReader.c scoWriter.c

It compiles perfectly for me too. I'm using TCC. The code is fine.
I've been playing around with it since yesterday. Thank you.

Edit: How many times have you changed your avatar, every time I reload the page it comes with a new cat :smile:
 
Go right ahead, I would very much appreciate a mirror!

I plan to post the importer tool once I clean it up, it's currently not utilizing the auto-fill functionality and it's very clunky. I do have an example of procedural object placement though:

Code:
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include "scoReader.h"
#include "scoWriter.h"

void print_vector(vector_t *vector) {
	printf("{%f, %f, %f}", vector->x, vector->y, vector->z);
}

void print_matrix(matrix_t *matrix) {
	printf("[");
	print_vector(&matrix->v0);
	printf(", ");
	print_vector(&matrix->v1);
	printf(", ");
	print_vector(&matrix->v2);
	printf("] ");
	print_vector(&matrix->o);
}

void print_mission_object(mission_object_t *mission_object)
{
	printf("%s: meta_type = %d, sub_kind_no = %d, variation_id = %d, variation_id_2 = %d, position = ",
			mission_object->id,
			mission_object->meta_type,
			mission_object->sub_kind_no,
			mission_object->variation_id,
			mission_object->variation_id_2);
	print_matrix(&mission_object->position);
	printf(", scale = ");
	print_vector(&mission_object->scale);
}

void make_pie(mission_object_t *mission_objects, char *name, vector_t *origin, double radius, int n)
{
	int lcv;
	for(lcv = 0; lcv < n; ++lcv)
	{
		double angle = 2 * M_PI * lcv / n;
		mission_objects[lcv].id = name;
		mission_objects[lcv].meta_type = 0;
		mission_objects[lcv].sub_kind_no = 743;
		mission_objects[lcv].variation_id = 0;
		mission_objects[lcv].variation_id_2 = 0;
		mission_objects[lcv].position.v0.x = sin(angle);
		mission_objects[lcv].position.v0.y = -cos(angle);
		mission_objects[lcv].position.v0.z = 0;
		mission_objects[lcv].position.v1.x = cos(angle);
		mission_objects[lcv].position.v1.y = sin(angle);
		mission_objects[lcv].position.v1.z = 0;
		mission_objects[lcv].position.v2.x = 0;
		mission_objects[lcv].position.v2.y = 0;
		mission_objects[lcv].position.v2.z = 1;
		mission_objects[lcv].position.o.x = radius * cos(angle) + origin->x;
		mission_objects[lcv].position.o.y = radius * sin(angle) + origin->y;
		mission_objects[lcv].position.o.z = origin->z;
		mission_objects[lcv].scale.x = 1.8;
		mission_objects[lcv].scale.y = 1;
		mission_objects[lcv].scale.z = 1;
		print_mission_object(&mission_objects[lcv]);
		printf("\n");
	}
}

int main(int argc, char **argv)
{
	if(argc < 3)
	{
		printf("Usage: %s input heightmap output\n", argv[0]);
		return EXIT_FAILURE;
	}

	FILE *in = fopen(argv[1], "rb");
	FILE *out = fopen(argv[2], "wb");

	if(!in || !out)
	{
		printf("ERROR: file(s) not found\n");
		return EXIT_FAILURE;
	}

	printf("Reading %s\n", argv[1]);
	sco_file_t sco_file;
	read_sco_file(in, &sco_file);

	int lcv;
	for(lcv = 0; lcv < sco_file.num_mission_objects; ++lcv)
	{
		print_mission_object(&sco_file.mission_objects[lcv]);
		printf("\n");
	}

	for(lcv = 0; lcv < 5; ++lcv)
	{
		int n = 32;
		mission_object_t *more_mission_objects = (mission_object_t *)realloc(
				sco_file.mission_objects,
				(sco_file.num_mission_objects + n) * sizeof(mission_object_t));
		if(more_mission_objects)
			sco_file.mission_objects = more_mission_objects;
		else
		{
			printf("ERROR: out of memory\n");
			return EXIT_FAILURE;
		}
		vector_t origin;
		origin.x = 310;
		origin.y = 277;
		origin.z = 9 * lcv + 33.23;
		make_pie(&sco_file.mission_objects[sco_file.num_mission_objects], "spr_bridge_modular_a", &origin, 50, n);
		sco_file.num_mission_objects += n;
	}

	printf("Writing %s\n", argv[2]);
	write_sco_file(out, &sco_file);

	return EXIT_SUCCESS;
}
That should generate a colosseum that looks like: http://steamcommunity.com/id/disgrntld/screenshot/594714994604098759

Edit #1: Haha, yea, I think the cat pics are an April Fools' Day gag.

Edit #2:
Code:
#include <stdio.h>
#include <stdlib.h>
#include "scoReader.h"
#include "scoWriter.h"
#include "tga.h"

void set_layer(ground_paint_t *ground_paint, int layer, tga_data_t *map)
{
	int max = ground_paint->size_x * ground_paint->size_y;

	if(ground_paint->layers[layer].cells == NULL) {
		ground_paint->layers[layer].continuity_count = malloc((max + 1) * sizeof(int));
		ground_paint->layers[layer].cells = malloc(max * sizeof(float));
	}

	ground_paint->layers[layer].continuity_count[0] = max;
	ground_paint->layers[layer].continuity_count[max] = 0;

	int x, y;
	for(y = 0; y < ground_paint->size_y; ++y)
	{
		for(x = 0; x < ground_paint->size_x; ++x)
		{
			int offset = (y * map->w + x) * map->depth / 8;
			short int r = map->data[offset];
			short int g = map->data[offset + 1];
			short int b = map->data[offset + 2];
			if(r != g || g != b) {
				printf("ERROR: map was not grayscale\n");
				return;
			}
			ground_paint->layers[layer].cells[x * ground_paint->size_y + y] = r / 255.0f;
		}
	}
}

int main(int argc, char **argv)
{
	if(argc < 12)
	{
		printf("Usage: %s input heightmap output\n", argv[0]);
		return EXIT_FAILURE;
	}

	FILE *in = fopen(argv[1], "rb");
	tga_data_t *height_map = tga_data_load(argv[2]);
	tga_data_t *gray_stone_map = tga_data_load(argv[3]);
	tga_data_t *turf_map = tga_data_load(argv[4]);
	tga_data_t *steppe_map = tga_data_load(argv[5]);
	tga_data_t *earth_map = tga_data_load(argv[6]);
	tga_data_t *desert_map = tga_data_load(argv[7]);
	tga_data_t *forest_map = tga_data_load(argv[8]);
	tga_data_t *village_map = tga_data_load(argv[9]);
	tga_data_t *path_map = tga_data_load(argv[10]);
	FILE *out = fopen(argv[11], "wb");

	if(!in || !height_map || !gray_stone_map || !turf_map || !steppe_map || !earth_map || !desert_map || !forest_map || !village_map || !path_map || !out)
	{
		printf("ERROR: file(s) not found\n");
		return EXIT_FAILURE;
	}

	printf("Reading %s\n", argv[1]);
	sco_file_t sco_file;
	read_sco_file(in, &sco_file);

	int lcv;
	for(lcv = 0; lcv < sco_file.ground_paint->num_layers; ++lcv)
	{
		printf("no: %u\n", sco_file.ground_paint->layers[lcv].ground_spec_no);
		switch(sco_file.ground_paint->layers[lcv].ground_spec_no)
		{
			case 0: // gray_stone
				set_layer(sco_file.ground_paint, lcv, gray_stone_map);
				break;
			case 2: // turf
				set_layer(sco_file.ground_paint, lcv, turf_map);
				break;
			case 3: // steppe
				set_layer(sco_file.ground_paint, lcv, steppe_map);
				break;
			case 5: // earth
				set_layer(sco_file.ground_paint, lcv, earth_map);
				break;
			case 6: // desert
				set_layer(sco_file.ground_paint, lcv, desert_map);
				break;
			case 7: // forest
				set_layer(sco_file.ground_paint, lcv, forest_map);
				break;
			case 9: // village
				set_layer(sco_file.ground_paint, lcv, village_map);
				break;
			case 10: // path
				set_layer(sco_file.ground_paint, lcv, path_map);
				break;
			case GROUND_PAINT_ELEVATION_MAGIC:
			{
				printf("Updating terrain\n");
				int x, y;
				for(y = 0; y < sco_file.ground_paint->size_y; ++y)
				{
					for(x = 0; x < sco_file.ground_paint->size_x; ++x)
					{
						int offset = (y * height_map->w + x) * height_map->depth / 8;
						short int r = height_map->data[offset];
						short int g = height_map->data[offset + 1];
						short int b = height_map->data[offset + 2];
						if(r != g && g != b)
							return EXIT_FAILURE;

						// 100 is the max height in-game, 886 is the max used height in your terrain software, 2625 is the max height available in your terrain software, and 1 is the water depth
						sco_file.ground_paint->layers[lcv].cells[x * sco_file.ground_paint->size_y + y] += (100.0f * 886.0f / 2625.0f) * r / 255.0f - 1.0f;
					}
				}
				break;
			}
		}
	}

	printf("Writing %s\n", argv[11]);
	write_sco_file(out, &sco_file);

	return EXIT_SUCCESS;
}
the importer I said I would upload. You should only use this if you want to import textures into your terrain in addition to adjusting the heightmap (Swyter was kind enough to write a lot more polished tool for strictly editing the heightmap that he posted later in this thread). It's purposefully missing tga.h as I don't want to maintain another tarball and this is quite frankly too clunky for most people to use.
 
Oh, it's probably just me being a klutzo; I'm sure I could have pushed it into working eventually. 

Just pressed for time atm, I was hoping it was going to be a ready-made tool with a loader UI to feed it a filename at least, I want to test it but I'm having enough trouble finishing the artwork and code atm and running the beta  :lol:
 
Okay, thanks to our new polymorphic friend we can have this:

Code:
22KiB | [IMG]http://i.imgur.com/j85OU.png[/IMG] [color=navy]scopng[/color] <sceneobj file> <optional png output filename>
(get it) (src) Saves the heightmap in PNG, TGA or BMP. Just change the extension


Code:
72KiB | [IMG]http://i.imgur.com/44YXw.png[/IMG] [color=navy]pngsco[/color] <sceneobj file> <png input file> <optional sceneobj output>
(get it) (src) Supports PNG, PSD, TGA, GIF, JPG heightmap files as input


Finally, as an extra goodie...

Code:
69KiB | [color=navy]psdtopng[/color] <input image> <optional png output filename>
(get it) (src) Converts PSD, TGA, GIF, JPG, BMP to PNG. Just drop the file on the program's icon


Let me know if you have any doubt.
Portable C, no external deps.

__________________
PS: I wrote this text the 1st of April, if that makes any sense. This has taken me 3 days to get in a decently working state. After lots of testing.
 
That sounds fabulous.  Really will try and find time to use this and think about use case and workflow.  Definitely want a drag-and-drop for it; it'd be nice to drop SCOs on it, get the PNG, do edits, drop PNG and get an edited SCO, simple and clean.  A preview of the mesh would be a big bonus, as there is obviously a scalar involved.  One thing to bear in mind, too, is that I would think that the values are floats, so it may be necessary to perform a gaussian blur operation to reduce stepping on the final results.



Another idea I want to throw at you guys, especially Disgruntled, since he seems to be interested in the procedural side of things, in terms of content integration; if I read that right, somebody's dreaming of a semi-automated way to, say, sketch out forests, city scenes, etc.

The single biggest annoyance / hassle on the engine, in terms of scene editing, is not the height map or even placing objects, per se, even though I can certainly appreciate the need for a tool to automate it if it can deal with certain problems. 

The really big issue is dealing with the AI mesh.  AI meshes not being even basically conforming to the upper surfaces of major shapes (buildings, large rocks, etc.) is  major problem.  Even something that merely detected larger objects and then cut out the AI mesh grid around them automatically, or better yet, first re-meshed the map at 1/2 the grid size and got a tighter fit would be a huge time-saver; if it could do some ray-testing and actually build some conforming quads that went around the ground-level sillouette, it would cut out about 90% of the drudgery associated with building a scene.  Heck, for that matter there's a performance tradeoff in scenes for the pathfinder code and how many sectors there are; a tool that merged sectors where only X meters of height differed between points (i.e., flat areas) to optimize performance might be worth doing, and that's probably not too terrible.

Anything higher than ground level would, naturally, be in a realm where human labor is probably more appropriate; I've seen some cutting-edge demos of even that problem being tackled, but that's really at the high end of the art atm and IIRC involves some fairly brutal surface analysis problems, so that's almost certainly not in scope.

Anyhow, sorry to ask for anything that big, but like I said, this is an area that's both poorly-understood and something where it's so much work that a lot of people can't even be bothered, even though it's necessary to get a good result with the AI.

I don't suppose that it is possible that you might be willing to consider looking at this problem? 

In theory, we have the data, including Marco's code for grabbing the collision meshes from the BRFs to place, scale, rotate etc. within the data and analyze... probably asking for a bit much, but hey, it's one of those things that came to mind when I realized that yes, somebody might be willing to look at SCO for a bit.
 
xenoargh said:
That sounds fabulous.  Really will try and find time to use this and think about use case and workflow.  Definitely want a drag-and-drop for it; it'd be nice to drop SCOs on it, get the PNG, do edits, drop PNG and get an edited SCO, simple and clean.  A preview of the mesh would be a big bonus, as there is obviously a scalar involved.  One thing to bear in mind, too, is that I would think that the values are floats, so it may be necessary to perform a gaussian blur operation to reduce stepping on the final results.

My software already does that. Look at the source. Just drag and drop files over them. I've added some logic for avoiding hassles, it's all in the source. I've tried to comment it as much as possible. I've cross-compiled them with GCC also, so it won't throw warnings.

The mesh is the same, the images are in 1:1 format, the precision lost is minimal because it does a pixel comparison and if both are really close to each other it picks the more exact one. I wanted to add 16bit support but most part of the programs don't support it. They already have trouble for grayscale ones.

And over all, I want to keep this tiny and clean. In my debug logs I can even see the image in integers over plain text.
Don't expect me to include an OpenGL window when you can just exit and re-enter the scene for seeing your changes in-game.

Finally as I've written before, I've been trying and testing the workflow and possible mistakes. I'm a modder myself after all!

Examples:
1. select
Code:
scn_ponyheaven.sco
-> drop on
Code:
scopng
-> it will create
Code:
scn_ponyheaven.sco.png

GnosP.png


...edit the png file with advanced professional software like mspaint ...

Nhm5j.png


2. select
Code:
scn_ponyheaven.sco
-> drop on
Code:
pngsco
-> it will recognize and use
Code:
scn_ponyheaven.sco.png
, using as output
Code:
scn_ponyheaven.edit.sco
so it doesn't gets overwritten by mistake.

2b. If you want to use another png, first select the
Code:
sco
and then the
Code:
png-you-want-to-use
and drag and drop both of them.

It will detect automagically if the sizes are matching and do it for you.
If not, it will tell you, accompanied by the right dimensions.

Base color is gray 38  (R: 38 G: 38 B: 38 / #262626) Darker than that is negative deep, and brighter, cool mountains.

____________
And if you want batching or alternative filenames, then I don't think that the command line is a problem here, looks more like an advantage for me. It's perfect for that.
 
Looks really good, Swyter!

One small issue, though. From my understanding, the terrain code defines a procedurally-generated base terrain, call it A. The SCO file contains the modification terrain, B, that when added to A produces the final terrain, C. So, C = A + B.

With scoWriter, we can modify B. However, we want our heightmap to define C. Well that sounds simple enough to fix, we think. We'll just subtract A from the heightmap before we dump it in (B = heightmap - A && C = A + B => C = heightmap). Alas, that's the rub. We don't have A. It's generated by code we're not privy to. This mean that there will be small undulations in C.

I did think of a way around that, though. We use the Mount&Blade editor to flatten our terrain. These terrain modifications are stored in B, and the end result is B = -A. Then, instead of overwriting B with our heightmap, we _add_ our heightmap to the existing B (B = -A + heightmap && C = A + B => C = heightmap).

Basically, I think you should add the heightmap to existing B so people can feed in pre-flattened SCOs.
 
Back
Top Bottom