B Info Module System Face code module system trickery | Scripts to read more face attributes, measuring string lengths and doing string-to-number conversion

[Swyter]

For those interested in reading face attributes for random characters, here's the extended function/polyfill for face_keys_get_morph_key. For the first time ever you'll be able to insult the player by measuring how big their nose or ears are at runtime, or whatever:

# face_keys_get_extended_morph_key
#  Stores face key's morph key value (0-20) into reg0; by exploiting some sneaky
#  string trickery we can read outside of the limited 0-7 range TaleWorlds gave us. >:)
#
#  This is a replacement or 'polyfill' for the limited (face_keys_get_morph_key) operation.
#  You can use this handy online tool to inspect and generate these face codes:
#        https://swyter.github.io/mab-tools/face
#
#  Note: Keep in mind that the actual range of available keys goes from 0 to 41.
#        But in this implementation we can only access until 'Eyebrow Height' in Native.
#        So if you want to read 'Eyebrow Depth' onwards you are out of luck.
#        Still, 0-20 is still much better than the ridiculous 0-7 range.
#
# Input:   param1: string_index containing the 64-hex-character face code, usually as returned by (str_store_troop_face_keys) or (str_store_player_face_keys), doesn't matter if they are in upper or lowercase, as long as they only consist in numbers from 0-9 and letters from a-f. face_keys_get_morph_key only cares about the 64 leftmost characters and discards the rest, that's why this trick works.
# Input:   param2: key_no, the morph key index you are interested in; from 0 to 20, both included.
# Output: reg0: selected face key's value
("face_keys_get_extended_morph_key",
[
    (store_script_param_1, ":string_index"),
    (store_script_param_2, ":key_no"),
#                                               |
#                                               V first three bits for morph key 0
#                                            ----
#                                       ----  111    <- fk00
#                                | ----   11 1       <- fk01
#   bit pattern repeats at key 4 V    1 11           <- fk02
#                             ---- 111               <- fk03
#                        ----  111                   <- fk04 (three bits at start again, like fk00)
#                   ----   11 1                      <- fk05 (three bits like fk01)
#                      1 11                            ...
#                   111                        
#                      (There are four 3-bit morph keys in each 3 4-bit nibbles/hex characters, 3*4=12 bits)
#                
# https://swyter.github.io/mab-tools/face#0x000000000000000070070070070070070000000000000000000000000000000a
#
#
#  0,  4,  8, 12, 16, 20 (same bit pos as face key 0)
#  1,  5,  9, 13, 17     (same bit pos as face key 1)
#  2,  6, 10, 14, 18     (same bit pos as face key 2)
#  3,  7, 11, 15, 19     (same bit pos as face key 3)
#
#
# 0x00000000000000007007007007007007           0/4=0*3  <- how many 4-bit hex characters we need to prepend to move the string to get it at the face key 0 position
#    0x00000000000000007007007007007 +3 =  3   4/4=1*3
#       0x00000000000000007007007007 +3 =  6   8/4=2*3
#          0x00000000000000007007007 +3 =  9  12/4=3*3
#             0x00000000000000007007 +3 = 12  16/4=4*3
#                0x00000000000000007 +3 = 15  20/4=5*3
#
#        if index > 20:
#          exit
#  
#        pad = (index/4)*3
#
#        while pad--:
#          'A' + key
#  
#        if (index % 4) == 0: face_keys_get_morph_key(0, 0)
#        if (index % 4) == 1: face_keys_get_morph_key(0, 1)
#        if (index % 4) == 2: face_keys_get_morph_key(0, 2)
#        if (index % 4) == 3: face_keys_get_morph_key(0, 3)

    (try_begin),
        (this_or_next|lt, ":key_no",  0),
        (             gt, ":key_no", 20),
        # swy: due to limitations of this string-prepending method we only support keys
        #      that are left-ward from the base fk0 position. from 21 onwards they are
        #      in the right (c) part, and we can't seemingly shorten strings for now,
        #      only prepend extra letters.
        #                                   <--|
        #      00000000000000006b1a20a72efac68814e5df58d1053977000000000000000a
        #      [      a       ]^^^^^^^^^^^^^^^^[      c       ][      d       ]
        #      00000000000000006b1a20a72efac6880000000000000000000000000000000a
        (assign, reg0, -1),
    (else_try),
        # swy: we were asked for a key between 0 and 20
        (set_fixed_point_multiplier,   1), # swy: we want integer division without decimals for this to work
        (store_div, ":pad", ":key_no", 4),
        (  val_mul, ":pad",            3), # swy: pad = (key_no / 4) * 3
 
        (str_store_string_reg, s2, ":string_index"),
 
        (try_for_range, ":dummy", 0, ":pad"),
            (str_store_string, s2, "@F{s2}"), # swy: move the useful part of the string to the right, one hex character at a time
        (end_try),
 
        (store_mod, ":key_no_mod_four", ":key_no", 4),
 
        (try_begin), (eq,":key_no_mod_four", 0), (face_keys_get_morph_key, reg0, s2, 0), # swy:      0,  4,  8, 12, 16, 20 (same bit pos as face key 0)
        ( else_try), (eq,":key_no_mod_four", 1), (face_keys_get_morph_key, reg0, s2, 1), # swy:     1,  5,  9, 13, 17     (same bit pos as face key 1)
        ( else_try), (eq,":key_no_mod_four", 2), (face_keys_get_morph_key, reg0, s2, 2), # swy:    2,  6, 10, 14, 18     (same bit pos as face key 2)
        ( else_try), (eq,":key_no_mod_four", 3), (face_keys_get_morph_key, reg0, s2, 3), # swy:   3,  7, 11, 15, 19     (same bit pos as face key 3)
        (  try_end),
    (end_try)
]),

Here is a test function with a funny code that shows how to use it, check out the result here:

          (try_for_range, reg99, 0, 22 + 1),
            #(str_store_string, s70, "@00000000000000007db6b6d9244922490000000000000000000000000000000a"),
            (str_store_string, s70, "@00000000000000006b1a20a72efac68814e5df58d1053977000000000000000a"),
            (call_script, "script_face_keys_get_extended_morph_key", s70, reg99),
            (display_message, "@SCRIPT RESULT: {reg99} {reg0}", 0x289128),
          (try_end),

Next step would be to have another function that lets you set those face attributes at runtime, too. That should also be possible for morph keys 0-41 and let you build your own Pinocchio depending on how the current quest is going, or dialog responses.

The code is pretty short, I just added a lot of comments and explanations, because how it works is kind of backwards. If you want more information about how these funky face keys work, they are documented them extensively here, scroll down.

 

[Swyter]

Here is a module system function that converts a decimal quick string to a normal module system integer; from "@1234567890" to 123456: https://gist.github.com/Swyter/0b03...malink_comment_id=4273522#gistcomment-4273522

It only works for numbers up to six digits right now, and they must not contain spaces or anything weird.

But still, darn. This is pretty powerful, imagine the things we could do in vanilla Warband. Exploiting the side effects of operations.

 

[Swyter]

Improved it so that one can actually convert integer strings of arbitrary size, now it can also find the length in bytes via some extra tricks: https://gist.github.com/Swyter/0b03...malink_comment_id=4273761#gistcomment-4273761

So if you give it "@1476419641" it knows it has 10 digits, and it gives you a decoded version as 1476419641 in a numeric register.

 

[Swyter]

I wanted to mention that thanks to the str_encode_url operation (which essentially turns characters into stringified hexadecimal digits) in combination with the weird stuff above we can also detect and parse symbols like . , or - in plain module system without WSE, unfortunately not g to z letters or Unicode glyphs. The function above only works for positive numbers so far, but it doesn't have to be the case.

My kingdom for a hack that lets us turn an indexed string character into a number. Or at least a way to compare and slice/cut strings.

There has to be a super convoluted way of doing this in the typical Rube Goldberg machine style we know and love.

 

[Eärendil Ardamírë]

Wow, looks like you have been busy like a bee in the last days. Thanks for sharing your findings with us!

 

[Swyter]

Always wanted to know more about how these codes were made, and recently came across them when seeing how limited we are in the string manipulation department for something TLD-related, given that this is the only avenue I found of bridging the string register to integer register divide, it seemed ripe for some fun experiments. Now there's face code format documentation, a handy web tool and some extra functions that make the original operations way more useful, hopefully other people will also find it interesting. ¯\_(ツ)_/¯

The next project I want to tackle is some tool to kind of extract SCO files into human readable files and being able to pack them back.

 

[Eärendil Ardamírë]

Well, that would be also interesting to have. Especially if one can switch out easily scene prop ids to be able to convert scenes across mods more easily

 

[Swyter]

That's exactly what I have in mind. It isn't even too hard to make, I have done it in an ugly way for SWC way back, like ten years ago.
It would be a matter of having the .sco and two versions of the scene props .txt file; old and new, and let the program remap them in a jiffy by matching their actual names.

Another potentially interesting thing is being able to resize scenes without losing your work, or having that fixed black border in the expanded part. I'll think about it.

 

[Eärendil Ardamírë]

There is also the tool of Janycz: https://github.com/cuellius/ScoTools/tree/master/ScoUtils
It's however not really well explained on how to use it. He only supplied the upfollowing script (copy pasted from here) additionally:

repl_list.txt -- list with replacements
X -> Y

Spoiler: Script

#include <iostream>
#include <fstream>
#include <string>
#include <map>
#include <cctype>
#include <functional>
#include <algorithm>
#include "ScoReader.h"
#include "ScoWriter.h"

inline std::string trim(const std::string &s)
{
    const auto f = std::find_if_not(s.begin(), s.end(), [](int c) { return std::isspace(c); });
    return std::string(f, std::find_if_not(s.rbegin(), std::string::const_reverse_iterator(f), [](int c) { return std::isspace(c); }).base());
}

std::map<std::string, std::string> load_repl_list(const char* filename)
{
    std::ifstream f(filename);

    std::map<std::string, std::string> map;
    while (!f.eof())
    {
        std::string s;
        std::getline(f, s);

        auto p = s.find('#');
        if (p != std::string::npos) s.erase(p);

        p = s.find("->");

        if (p == std::string::npos) continue;

        const auto a = trim(s.substr(0, p));
        const auto b = trim(s.substr(p + 2));

        map[a] = b;
    }

    f.close();

    return map;
}

int main()
{
    auto map = load_repl_list("F:\\repl_list.txt");

    sco_file_t scene;
    read_sco_file(fopen("E:\\Mount&Blade Warband\\Modules\\DNO_Historical_Battle_0.96\\SceneObj\\scn_town_nov_center.sco", "rb"), &scene);

    for (int i = 0; i < scene.num_mission_objects; i++)
    {
        if (scene.mission_objects[i].meta_type == MT_SCENE_PROP)
        {
            std::string id = scene.mission_objects[i].id;
            id.erase(0, 4);
            //std::cout << "scene prop '" << id << "'" << std::endl;
            if (!map.count(id)) continue;
            std::cout << "replace '" << id << "' with '" << map[id] << "'" << std::endl;
            id = "spr_" + map[id];
            scene.mission_objects[i].id = strdup(id.data());
        }
        else if (scene.mission_objects[i].meta_type == MT_FLORA)
        {
            std::string id = scene.mission_objects[i].id;
            std::cout << "flora '" << id << "'" << std::endl;
            if (!map.count(id)) continue;
            std::cout << "replace '" << id << "' with '" << map[id] << "'" << std::endl;
            id = map[id];
            scene.mission_objects[i].id = strdup(id.data());
        }
    }

    write_sco_file(fopen("scn_town_25_center.sco", "wb"), &scene);

    return 0;
}

I didn't understood it immediately and there it sits since then, waiting as a bookmark for me to spend time with it again

Another potentially interesting thing is being able to resize scenes without losing your work, or having that fixed black border in the expanded part. I'll think about it.

That would be surely interesting for some sceners although I think there is some game engine mechanic behind it. Will be the more tricky challenge for you.

 

[Swyter]

That would be surely interesting for some sceners although I think there is some game engine mechanic behind it. Will be the more tricky challenge for you.

That black thing happens because the terrain paint grid size is fixed when you first generate the SCO. I think the width and height for the scene get set in stone, like the Photoshop canvas. So no matter how much you change the dimensions internally afterwards the terrain only uses the original ones until you regenerate the SCO.

It's true that to keep the original scene centered everything probably would need to be moved around to shift it away from the corner, and if the terrain code you chose is bumpy it may not match anymore with the underlying orography. So there may be some complexity involved. Or not, I haven't looked into it properly yet. But it seems more useful than this.