Scene .SCO Toolset - heightmap converters, object placers, and texture importers! Oh my!

Users who are viewing this thread

xenoargh said:
Tested :smile:

Procedure:  copied a SCO from mod SceneObj file to location of Scopng.  Dragged SCO onto Scopng. 

Result:  a brief trip to the Console; no other output.  Are the directories non-relative, or did it crash during PNG creation?

It may sound obvious but only outputs a heightmap when there's a heightmap to output. It doesn't works in interiors and the like.
Post the console log here. It's pretty verbose.

The only problems I ever had were some memory leaks in the scoReader, in very specific cases.


________
Oh, and they don't need to be in the same directory, works everywhere. Just drag and drop.
If the log says that the layer exists and isn't empty, but the image creation failed then it's permission related.
 
Oh, duh... the scene I picked had the interior flag set  :roll:

OK... tried another one, and got the result.

Definitely think that it needs to use a 0.25 or even 0.125 meter scale per pixel value; the output result lost a lot of details, and it'd be very hard to work with all the values being restricted to such low numbers. 

Better idea is to have it stop and ask what we want the scalar value to be, for 0-255.
 
Just to have an idea of why I'm talking about the height values being a little hard to work with, here's my sample output, from a plains area that has a lot of gentle rolling terrain, a lake and other features:

scn_00002_plains.sco.PNG
 
xenoargh said:
Oh, duh... the scene I picked had the interior flag set  :roll:

OK... tried another one, and got the result.

Definitely think that it needs to use a 0.25 or even 0.125 meter scale per pixel value; the output result lost a lot of details, and it'd be very hard to work with all the values being restricted to such low numbers. 

Better idea is to have it stop and ask what we want the scalar value to be, for 0-255.

Just do it, that's why I made it open source. Just change two lines of code by the scalar you want, one per program.
I don't have use for it right now. And it's always better to do it well.

I included a homebrew-style makefile. Doesn't have deps. Takes 5 minutes to have it working.
If you use TCC you can use it like a scripting language. Running the source directly.

Adding procedural objects like Disgrntld did with his coliseum is a good point also.

xenoargh said:
Just to have an idea of why I'm talking about the height values being a little hard to work with, here's my sample output, from a plains area that has a lot of gentle rolling terrain, a lake and other features:

scn_00002_plains.sco.PNG

psimagen.jpg


Those hills are part of the terrain key, the only thing visible in your map is a slightly deep lake.
Keep in mind that this is linearly projected into the procedural terrain, here are only saved the changes that you made.
 
Oh, it doesn't reflect the auto-gen terrain?  Interesting!  So basically if we run this, are we overriding all of the terrain and those areas not marked will be totally flat?
 
xenoargh said:
Oh, it doesn't reflect the auto-gen terrain?  Interesting!  So basically if we run this, are we overriding all of the terrain and those areas not marked will be totally flat?

Nah, where did you read that? If you want a flat terrain just use this tool and tweak the terrain code like that.
Once you have a perfect canvas you can use the heightmap tool with a terragen-like program, or the in-game editor for getting your very own result.

The SceneOBJ file format works like a diff file, it stores the heightfield relatively respect to the procedurally generated terrain, per cells.

Code:
/* just an easy diagram for quick understanding of the structure...
   surely more readable than the original implementation.
    --swy
    
    
    ->PAINT MAGIC
    ->NUM_LAYERS
    ->SIZ_X
    ->SIZ_Y
    _ (per layer)
     |  
     | ->SPEC_NO
     | ->SPEC_ID (STR)
     V 
       ->CONTINUITY_COUNT (SIZX * SIZY)
     |  
       _ (per X of every Y)
     |  |
        | ->CONTINUITY_COUNT (X * SIZY + Y)
     |  | ->CELLS (X * SIZY + Y)
        V
     |   
*/

Everything but the objects (plants/ scene props/ spawn points/...) and the AI mesh is stored as paint layers, and that includes, the heightfield, the ground texture paint, and the vertex coloring.

With this toolset we are only dumping and overwriting them in case they exists.
 
We discussed ways around this earlier in the thread, xenoargh:
From my understanding, the terrain code defines a procedurally-generated base terrain, call it A. The SCO file contains the modification terrain, B, that when added to A produces the final terrain, C. So, C = A + B.

With scoWriter, we can modify B. However, we want our heightmap to define C. Well that sounds simple enough to fix, we think. We'll just subtract A from the heightmap before we dump it in (B = heightmap - A && C = A + B => C = heightmap). Alas, that's the rub. We don't have A. It's generated by code we're not privy to. This mean that there will be small undulations in C.

I did think of a way around that, though. We use the Mount&Blade editor to flatten our terrain. These terrain modifications are stored in B, and the end result is B = -A.
If you don't care about a completely flat base you can get pretty close by lowering hill height and not have to worry about creating -A.
 
Well, ideally, I'd want the exported PNG to reflect the entire actual heightmap as generated + modifications; but I guess I could use the generator script to build a totally-flat terrain, then go from there.  Most of the time, though, I tend to run the randomizer many times and select something that's vaguely like what I was looking for, then customize from there, so that workflow's going to take some getting used to :smile:
 
Yea, it's a pain, especially like in your case where you want to start from the generated terrain. One thing that could be done for people who want flat maps though, is generate a few different sized terrains, flatten them out, and package them with their code as a sort of starter pack. Who knows, maybe someone else can even help us decipher how to make our own generator that we could use for both workflows.
 
cmpxchg8b said:

Okaaay, Now... do you have a source code for that or... are expecting us to reverse and reimplement your reversed and reimplemented code?
I like the map-like txt output though, but people only wants to use a program, not a collection of dependent tools.

I'm not feeling like trying to learn to use IDA for that. Also, I'm seeing fun stuff all over the place, including some
Code:
log10
functions and
Code:
asin
,
Code:
acos
,
Code:
exp10
,
Code:
atan
. I don't know you, but logarithms and family aren't my specialty.

Code:
pow, exp, exp, log10, log10, log, log, pow, asin, acos, pow, pow, exp10, log, log10, exp, atan, ceil, floor, modf, sin, cos

Creepy. But mark me as impressed. :smile:


Edit:
Code:
IsDebuggerPresent
? Seriously? You must be joking.  :shock:
Mate, this is a tool for recovering terrain codes, not Assasin's Creed III, Are you from Ubisoft or something?
 
Unfortunately there's quite a lot of WSE2 code in there that I'm not ready to share. When I have some spare time I might "clean" it to a point where I can release the source.

PS: judging from the functions that call IsDebuggerPresent (_call_reportfault, _invoke_watson...) I'd say that isn't my code... :smile:
 
cmpxchg8b said:
Unfortunately there's quite a lot of WSE2 code in there that I'm not ready to share. When I have some spare time I might "clean" it to a point where I can release the source.

PS: judging from the functions that call IsDebuggerPresent (_call_reportfault, _invoke_watson...) I'd say that isn't my code... :smile:
I'm using Olly, it doesn't gives me the function names.
Now, I've been itinerating along the lots of function calls and doesn't looks like something made intentionally.

Anyway, the original WSE also had some basic protections. I had to patch the CRC check for using it with my revamped scene editor UI.

But that is understandable. Thanks for this, anyway.
It's probably one of the most complex pieces of code in the game.
 
Aw, c'mon, Cmp, you know nobody's going to be able to build a WSE2 from whatever's in there, I'm fairly sure you're the only one with enough background to even know what's what :lol:

Anyhow, that at least puts that a little closer to practical.  As for blanks, Swyter's tool provides that, I think, by setting the terrain values all to zero, correct?  So it's merely a matter of setting all of that to zero, declining to add a river, and the terrain is all set to 2 meters or thereabouts, IIRC.  From there, I just need to find some time to recompile with the 0.125 meter / pixel setting, and voila, it's a practical tool.  I'm having RL issues this week and I'm juggling this with work on the second Beta for Blood and Steel, unfortunately, or I'd have just gotten it done already.  I may be able to get to this tomorrow afternoon, though.
 
cmpxchg8b said:
Just because you ask nicely.
If something doesn't make sense it's probably because I snipped everything around it.

http://dl.dropbox.com/u/30551702/TerrainGen.rar

This is insane. How many hours have you dedicated to it? I barely understand how perlin noise works even after reading this:
Code:
http://webstaff.itn.liu.se/~stegu/simplexnoise/simplexnoise.pdf

I'm totally in another league, maybe in a year or two.
Also, if you've taken care of reconstructing all that, WSE2 is going to be something unexpected, in a really good sense.

__________
I doesn't compiles cleanly with
Code:
g++
, all those macros, I'll have to install msvc, once again.
 
Swyter said:
This is insane. How many hours have you dedicated to it? I barely understand how perlin noise works even after reading this:
Code:
http://webstaff.itn.liu.se/~stegu/simplexnoise/simplexnoise.pdf

I'm totally in another league, maybe in a year or two.
I think a couple of days, definitely not more than a week (including the stuff that I snipped, like building manifold and meshes). You'd be surprised to know how many of it I don't understand, but the thing is, you don't need to understand all of it, as long as it works.
Swyter said:
Also, if you've taken care of reconstructing all that, WSE2 is going to be something unexpected, in a really good sense.
It will, yes.
Swyter said:
__________
I doesn't compiles cleanly with
Code:
g++
, all those macros, I'll have to install msvc, once again.
I haven't tested it with other compilers, so it's very likely that it won't work, but... what macros? I don't recall adding anything more than includes, defines and pragma once.
 
Thanks, CMP, that's a useful glimpse inside the sausage factory!  I understand most of that, although the Perlin table's... interesting.  Bet it's just a tiling noise bitmap that was converted to data :smile:

Amongst other things, it would appear that:

1.  We can, indeed, force just one terrain layer in a seed?  I presumed that was the result of a flat map without any rivers, but it's good to know that for sure.
2.  Terrain doesn't have to necessarily just be linear, i.e. overhangs may be possible via vertex manipulation perhaps?  Probably not; I wouldn't be surprised if characters use the heightmap for ground collision detection, and that would hose it...
3.  The maximum number of terrain faces is an interesting limitation.  So if you've got a 1023^2 heightmap at 5M resolution, in theory you can push that all the way out to just under 1024+256, but not really.  That's a weird stopping point.  I wonder if the engine handles pathfinding outside 1023^2, amongst other things.
4.  It confirms my research in the shader thread about the methods for construction and drawing method of the meshes.  I really wish it didn't work like this, it's so much overdraw and it doesn't really have any effective optimization, nor can it be very efficiently optimized as designed, unfortunately.  Ah well :smile:
 
xenoargh said:
1.  We can, indeed, force just one terrain layer in a seed?  I presumed that was the result of a flat map without any rivers, but it's good to know that for sure.
I'm not getting this one.

xenoargh said:
2.  Terrain doesn't have to necessarily just be linear, i.e. overhangs may be possible via vertex manipulation perhaps?  Probably not; I wouldn't be surprised if characters use the heightmap for ground collision detection, and that would hose it...
Heighmaps are linear vertical, so, no. If you want to do fancy stuff, like caves or dungeons, model your own terrain mesh. Games like Gothic I/ II are composed by kilometers of solid, handcrafted subdivided surfaces.

Agents use the ground meshes, for colliding into. Forget about the height-map, is just another layer used in a later stage by the terrain generator, that isn't really used in the "game".

xenoargh said:
3.  The maximum number of terrain faces is an interesting limitation.  So if you've got a 1023^2 heightmap at 5M resolution, in theory you can push that all the way out to just under 1024+256, but not really.  That's a weird stopping point.  I wonder if the engine handles pathfinding outside 1023^2, amongst other things.
The engine handles pathfinding within the navigation mesh, otherwise it just do advanced stuff like go from A to B and getting stuck, agents are restricted by the bounding box, haven't you experienced empty scenes? When the agent it's spawned outside you can only rotate him, it gets teleported back to a playable area.

xenoargh said:
4.  It confirms my research in the shader thread about the methods for construction and drawing method of the meshes.  I really wish it didn't work like this, it's so much overdraw and it doesn't really have any effective optimization, nor can it be very efficiently optimized as designed, unfortunately.  Ah well :smile:
The shaders are a disaster compared to the terrain generator. It's pretty well done, but hardcoded.
I don't know why I expected something less arcane,... and full of hacks.


cmpxchg8b said:
Swyter said:
__________
I doesn't compiles cleanly with
Code:
g++
, all those macros, I'll have to install msvc, once again.
I haven't tested it with other compilers, so it's very likely that it won't work, but... what macros? I don't recall adding anything more than includes, defines and pragma once.
I meant platform specific code, workarounds, build systems, and all what is needed to keep portable code nowadays. 
Code:
*shrugs*
That's one of the reasons I don't get seriously with the heavies.
 
Back
Top Bottom