Warband Refined & Enhanced Compiler Kit (W.R.E.C.K.)
Download W.R.E.C.K. 1.0.0 (1.25 Mb file size) from:
Nexus Mods | Lav's Warband Workshop
Updated version with bugfixes by Vetrogor
Download W.R.E.C.K. 1.0.0 (1.25 Mb file size) from:
Nexus Mods | Lav's Warband Workshop
Updated version with bugfixes by Vetrogor
W.R.E.C.K. Description and Features
W.R.E.C.K. is designed as replacement for the module system compiler provided by Taleworlds, providing drastically improved performance, enhanced error reporting and a number of advanced features potentially useful in Warband mod development.
W.R.E.C.K. is very easy to try. You don't have to make any changes to your module system file - just copy the W.R.E.C.K. files to your module system (they don't overwrite anything) and you're good to go! You might not get full specter of W.R.E.C.K. features this way, but you'll be able to try it for yourself - at no risk to your module.
W.R.E.C.K. requires Python version 2.6.x or 2.7.x! Unlike vanilla compiler, it will not be able to run on Python 2.5.x versions.
Basic W.R.E.C.K. features:
- Compilation speed is several times higher than vanilla compiler.
- A number of potential errors which are not detected by vanilla compiler are actually detected and reported by W.R.E.C.K. Also, unlike vanilla compiler, W.R.E.C.K. will not flood your screen with tons of error messages because of a single bug.
- W.R.E.C.K. provides modder with the tools to easily change item modifiers, game UI strings and user hints, which sometimes is extremely useful. For that purpose, it introduces three new optional modules: module_item_modifiers.py, module_ui_strings.py and module_user_hints.py, which are compiled into Data/item_modifiers.txt, languages/en/ui.csv and languages/en/hints.csv files respectively. If module files are missing from your module system, no files will be generated (making Warband fallback to standard ones).
- Vanilla compiler requires two sets of entity references: static references contained in ID_*.py files (icon_player) and quoted references ("icon_player") which are calculated at compile-time. This creates a number of problems. For example, it's impossible to simply add a new icon and a new party which will use that icon: modder must add a new icon, compile the module to regenerate the ID files, and only then he can add a new party. W.R.E.C.K. offers a solution to this problem by introducing a dynamic reference (icon.player) which can be used freely and everywhere, replacing both types of vanilla references. Note that while W.R.E.C.K. sticks to standard Warband prefixes, strings are referenced as s.<string_name> instead of str.<string_name>. This is the only deviation from the scheme, necessitated by the fact that str is a global keyword in Python.
- Global and local variables can also use this syntax, with ":variable" becoming l.variable, and "$variable" becoming g.variable.
- More than that, new dynamic references can be freely used in mathematical expressions. For as long as the expression can be calculated at compile time, it will be calculated at compile time and the calculated value will be used in the resulting code.
- To further expand on this feature, it is actually possible to use mathematical expressions that include run-time variables (locals, globals, registers) in the expressions within a code block. For such expressions, compiler will auto-generate the module system code which will calculate the mathematical expression and insert it into the code, completely transparently for the modder. Note that for obvious reasons it is not recommended to use this feature inside conditional operations which are part of this_or_next operation blocks.
- W.R.E.C.K. provides modder with ability to access properties of many game entities at compile-time, using them as script parameters or putting them directly in the module system tuples. Syntax like (assign, reg11, trp.swadian_footman.strength), becomes possible with this feature.
- Vanilla module system relies on header_skills.py file to correctly declare skill-related constants, which makes it unnecessarily difficult to alter game skills as any changes to module_skills.py must be mirrored manually in header_skills.py as well. W.R.E.C.K. compiler automatically and dynamically calculates all necessary constants.
- Dynamic references (imod.<modifier> and imodbit.<modifier>) are available to be used instead of vanilla imod_* constants.
- W.R.E.C.K. will transparently generate skl_*, knows_*_*, imod_* and imodbit_* constants from module files for backwards compatibility.
- W.R.E.C.K. provides support for a fairly advanced plugin system. It is now possible to create plugin files which can include all types of module system data, inject code and data entries into the existing module system, extend operations syntax and perform arbitrary processing over module data. Plugin functionality is described in more detail below.
- A number of artificial limitations have been lifted from the Module System. W.R.E.C.K. supports numeric, string and position registers up to reg127, s127 and pos127 respectively. There are no artificial limits on item weight and max_ammo parameters (not very useful for ammo items, but may be pretty nice for food and trade goods).
- New ATTR() and SKILLS() functions are provided to be used in troop definitions in module_troops.py file. Modder can use ATTR(8, 9, 6, 4, 3) instead of str_8|agi_9|int_6|cha_4|level(3) and SKILLS(trade = 2, riding = 6) instead of knows_trade_2|knows_riding_6, resulting in better readable code. Additional benefit is that this syntax is not limited by the constants defined in header_troops.py, making it easy to define attributes values above 30 and skill levels above 10.
W.R.E.C.K.ing Your Mod: Installation and Integration
W.R.E.C.K. is designed to be usable right out of the box. Just copy all W.R.E.C.K. files from copy_n_forget folder to your module folder and you're good to go. That's actually all you need to get a much faster compiler, better error reporting and ability to mod item modifiers and UI strings directly from your module system.
Of course, many advanced W.R.E.C.K. features will still be unavailable to you.
To properly integrate W.R.E.C.K. with your module system, you will need to modify your module_*.py files. In all module_*.py files EXCEPT module_info.py AFTER all other import declarations but BEFORE all other code, include the following line:
Code:
from compiler import *This line will enable full support for all W.R.E.C.K. features for your module. You can remove build_module.bat and all process_*.py files at this point as W.R.E.C.K. is not using any of them.
Note that generally you don't actually need to modify module_info_pages.py and module_strings.py files as they contain nothing but text. Also W.R.E.C.K. compiler does not use module_variables.py file so it can be safely deleted.
You can do even more, though this is purely optional. See, while you have already enabled support for all W.R.E.C.K. advanced features, a lot of vanilla module system limitations still apply. Namely, you're still fully dependent on ID_*.py files as module system code contains a lot of references to constants declared in those files. So you might want to replace all those references with W.R.E.C.K.-style dynamic references instead.
Once that is done, you can remove ALL other import directives from module_*.py files, leaving from compiler import * as the only import directive.
Note that W.R.E.C.K. prefers header files from headers/ subfolder if it's present in your module system. So once you remove all other import directives, you can replace all header_*.py files in the module system folder with a single headers/ folder from integrated_ms pack, reducing file clutter in your module system.
You can also change the path where ID_*.py files will be generated or simply disable their generation, further reducing clutter in the module system folder.
As an example, Native 1.165 Module System fully integrated with W.R.E.C.K. is included in this archive. You can find it in integrated_ms folder.
Possible W.R.E.C.K.age: Compatibility Issues
Note that some libraries and packs may have compatibility issues with W.R.E.C.K. Generally, if you do not adapt your module system to W.R.E.C.K., there shouldn't be any issues no matter what you use. However if some pack or tweak is using Python code to process module system data, it may cause issues with an adapted module system.
In other words, if you're using such packs, DO NOT integrate your module system with W.R.E.C.K. You will not get any advanced features, but you will still get faster compilation time and improved error reporting.
This includes: ModMerger framework (as it's doing extensive processing on module system files), kt0's auto-resolve scripts (which traverses items and troops lists expecting to see simple numbers there, which is not the case with W.R.E.C.K.), etc.
Scene Chooser script should not cause any problems even with integrated module system, but will not detect any scenes added in plugins. For as long as you don't use scene-adding plugins, it should be fully compatible.
Designing a W.R.E.C.K.: New Module Options
W.R.E.C.K. will recognize a number of new directives in the module_info.py file in addition to export_dir parameter:
- Option write_id_files may be used to change the path where ID_*.py files are generated by the compiler or disable their generation completely. See "Installation" chapter for more details.
- Option show_performance_data can be set to True or False to enable or disable output of compiler performance information.
- Finally, any plugins that modder wants to use are imported in module_info.py file with the import plugin_file_name directive.
Plugin System Basics
W.R.E.C.K. supports extension of Module System via plugins. Note that unlike ModMerger plugins, W.R.E.C.K. assumes that all plugin-related information is stored in a single file. This doesn't prevent you from keeping your plugin in multiple files, but one of the files must be designated as the "master" file. It is the file that W.R.E.C.K. will load as the plugin, and it will be responsible for loading it's children files.
There are no specific requirements for naming your plugin files, but a recommended naming convention is "plugin_*.py".
Each plugin MUST have the following two lines at the very start of the file:
Code:
from compiler import *
register_plugin(__name__)The first operation is the same as for regular module files and it will pre-load all headers, constants and other parafernalia for the module system code within the plugin to work correctly.
The second operation is to let W.R.E.C.K. know that this file is actually a plugin and must be treated as such.
Now that you have a plugin, you can add any module data to it, using the same syntax as you would use in respective module system file. For example, you can add the following lines to your plugin:
Code:
meshes = [
("lco_background", 0, "mp_ui_bg", 0, 0, 0, 0, 0, 0, 1, 1, 1),
]During compilation, W.R.E.C.K. will load the module system and plugin. Reference mesh.lco_background (or "mesh_lco_background") will be accessible throughout module system, and the mesh itself will be appended to the end of module system meshes list.
If you have several plugins that add new meshes, their meshes will be added in the same order as the plugins are imported in module_info.py.
Plugin Dependencies
Note that plugin code can easily reference module system data outside of plugin itself, including data inside other plugins. It is perfectly possible to declare a new hero troop "bob_the_uncle" in plugin_heroes.py and actually use trp.bob_the_uncle (or "trp_bob_the_uncle") inside a script in plugin_family.py.
Such situation is called "plugin dependency", and in this situation we have plugin_family.py being dependent on plugin_heroes.py. To let the compiler know of this dependency, you need to add one more line:
Code:
require_plugin("plugin_heroes")With this directive, if for some reason you import plugin_family in module_info.py, but forget to import plugin_heroes, compiler will give you a warning, requesting you to import the missing plugin.
If your plugin requires several other plugins to work correctly, list several plugin names inside the require_plugin directive:
Code:
require_plugin("plugin_basic_math", "plugin_advanced_math")Data Injection
Of course, there are situations where you don't want to simply append some data entries to the existing ones. If your plugin defines a new mercenary troop, you will probably wish to insert it somewhere in the middle of module_troops.py, right after the standard mercenaries.
Similarly, your plugin may wish to modify an existing game script by inserting some additional code (which will probably deal with situations specific to plugin-defined data).
This is done through the process of "data injection".
W.R.E.C.K. does not limit a modder in what and where can be injected. You can inject entire data entries (troops, meshes, dialogs). You can inject code snippets. You can inject several new beards into male skin definition in module_skins.py. Essentially, if it's a list (i.e. some stuff separated with commas) - you can inject into it.
To inject some data into the module, you need to define an injection point (to let the compiler know WHERE to inject the data), and the plugin needs to provide the injected data.
To define an injection point, insert a bogus list element inject('injection_name'). You can define any string as injection name, it will be used by compiler to determine what injection to replace it with.
Example:
Code:
meshes = [
# (...)
("pic_arms_swadian", 0, "pic_arms_swadian", 0, 0, 0, 0, 0, 0, 1, 1, 1),
("pic_arms_vaegir", 0, "pic_arms_vaegir", 0, 0, 0, 0, 0, 0, 1, 1, 1),
("pic_arms_khergit", 0, "pic_arms_khergit", 0, 0, 0, 0, 0, 0, 1, 1, 1),
("pic_arms_nord", 0, "pic_arms_nord", 0, 0, 0, 0, 0, 0, 1, 1, 1),
("pic_arms_rhodok", 0, "pic_arms_rhodok", 0, 0, 0, 0, 0, 0, 1, 1, 1),
("pic_sarranid_arms", 0, "pic_sarranid_arms", 0, 0, 0, 0, 0, 0, 1, 1, 1),
inject('new_faction_pics'),
("pic_castle1", 0, "pic_castle1", 0, 0, 0, 0, 0, 0, 1, 1, 1),
("pic_castledes", 0, "pic_castledes", 0, 0, 0, 0, 0, 0, 1, 1, 1),
# ...
]In this example, we have defined injection point named 'new_faction_pics'. If your plugin is adding a new faction into the game, it can use this to add that faction's picture into module_meshes so all faction pics will still be listed in the same order as factions themselves.
Now, your plugin can define the data injection. Instead of just declaring meshes list with our new mesh, it will instead contain:
Code:
injection = {
'new_faction_pics' = [
("pic_gothian_arms", 0, "pic_gothian_arms", 0, 0, 0, 0, 0, 0, 1, 1, 1),
],
'some_other_injection' = [
# (...)
],
# (...)
}Obviously a plugin that's supposed to add a new faction will inject MUCH more than just a single mesh, we're just keeping things simple for easier understanding.
Obvious question: what happens if we define an injection point but there's nothing to inject?
Answer: nothing is injected.
Obvious question: what happens if several plugins attempt to inject using the same injection point?
Answer: they will all inject successfully in the same order that they're imported in module_info.py file.
Module System Pre-Processing
Sometimes there are situations where a plugin must perform some additional processing of the entire module system (or part of it). W.R.E.C.K. provides functionality to do so, however at this point it requires a good amount of Python knowledge to use this feature effectively.
If a plugin declares preprocess_entities() function, it will be called by the compiler after all plugins are loaded, all injections (with the exception of script injections) are complete and all data references have been calculated. This means that preprocessor scripts cannot and MUST NOT change data identifiers, insert data, delete data or move data around. However preprocessor scripts are perfectly capable of changing data contents for as long as they keep the identifier.
Note that this feature enables a plugin to affect data entries everywhere, including the base module system and other plugins, as the data arrays preprocessor script is working with will already include all that information.
For example, it is perfectly possible to write a preprocessor script will will add 20% to swing damage of all one-handed cutting weapons - and the script will work correctly even if modder will later add a new plugin with several new items.
Functionality of preprocessor script is used extensively in plugin_ms_extension.py plugin to fill the strings with actual names of attributes, proficiencies, skills and item modifiers which are used in the mod, dynamically retrieving them from their respective lists.
Syntax Extension
Another powerful feature of W.R.E.C.K. plugin system is it's ability to extend module system scripting syntax with new macro operations.
For example, a modder can add the following code to his plugin:
Code:
def troop_set_attribute(troop, attribute, value):
return [
(store_attribute_level, l._cached_, troop, attribute),
(store_sub, l._cached_, value, l._cached_),
(troop_raise_attribute, troop, attribute, l._cached_),
]
extend_syntax(troop_set_attribute)This will effectively create a new macro operation (troop_set_attribute, <troop_id>, <attribute_id>, <new_value>), which can then be freely used anywhere in the module system. Of course, it is not a "true" operation as it will actually be replaced with the code returned by the troop_set_attribute function in the plugin file.
New operation can even be used in the other plugins, though in this case it can only be used in plugins which are loaded *after* the plugin which defines the new operation.
Note that while it is possible to create can-fail macro operations, such operations can not and must not be used in combination with this_or_next or neg prefixes, not even as the last operation of a this_or_next condition block.
Last edited by a moderator:
