Reconstructing With Fire and Sword's Module System -- My new hobby :)

[Swyter]

Hey there, mates.
Some time ago (three weeks?) I had a talk with Comrade Temuzu about sound flags in the Mount & Blade Mod Makers Q&A Thread.

Back then, I did a fast program to check the flag usage and almost without intending, ended with a complete reversed version of the Ogniem i Mieczem's

module_music.py

. I found it pretty fun and heart warming. As I love procedural stuff.

Here's the current status. I'll be updating it as the reconstruction advances.
Some parts are completely usable, other ones almost compile, and the lesser part is only shown here as reference for readability.

[!] Due the 40000 characters per post limit. I'm going to use Ubuntu Pastebin, let me know if you have any problem.

[color=navy][SIZE=13px]--module_info_pages.py — 100%[/SIZE][/color]
http://paste.ubuntu.com/626832/

[color=navy][SIZE=13px]--module_music.py — 100%[/SIZE][/color]
http://paste.ubuntu.com/626837/

[color=navy][SIZE=13px]--module_quests.py — 100%[/SIZE][/color]
http://paste.ubuntu.com/626842/

[color=navy][SIZE=13px]--module_scripts.py — 80%[/SIZE][/color]
http://paste.ubuntu.com/626804/

[color=navy][SIZE=13px]--module_skins.py — 20%[/SIZE][/color]
not yet -- use warband's one instead

[color=navy][SIZE=13px]--module_sounds.py — 100%[/SIZE][/color]
http://paste.ubuntu.com/626844/

[color=navy][SIZE=13px]--module_strings.py — 100%[/SIZE][/color]
http://paste.ubuntu.com/626848/

That's it.
Let me know if you find it useful.

 

[xenoargh]

Very cool 

 

[Swyter]

Very cool 

Thanks, xeno.
Any specific suggestion? 

 

[xenoargh]

Troops is just Warband's yes?

Module_mission_templates.

Oh, and... can you grep the hex values to coorespond to Warband's values in the headers?  Other than the new ones, that is?

 

[Swyter]

Oh, and... can you grep the hex values to coorespond to Warband's values in the headers?  Other than the new ones, that is?

Troops are easy. Didn't needed for anything so I left it for later.
The only important code chunk left is doing the ID correlation and identification, as I'm writing the whole thing in Lua this isn't that cool.

Had to write my own artisan syntax parser. The logical and easy step would be reversing the inner workings of the Python scripts.
The problem. I've a basic knowing of real Python. And anyways, I'm not feeling comfortable.

I'll keep you informed of my forays. 

 

[Yoshiboy]

I realize this is only a hobby but I shouldn't have to tell you that decompiling code (even in this sense, which is as you put it is more like "reconstructing") is illegal. I'm not going to lock the thread, but this is just a warning that if taleworlds come knocking this stuff will have to be removed and the project shut down.

I'm also going to make it clear now that With First and Sword's code should not be used in any modules. Any resources from the game also most certainly should not be used. The devs have already spoken on this matter.

 

[Swyter]

I realize this is only a hobby but I shouldn't have to tell you that decompiling code (even in this sense, which is as you put it is more like "reconstructing") is illegal. I'm not going to lock the thread, but this is just a warning that if taleworlds come knocking this stuff will have to be removed and the project shut down.

I'm also going to make it clear now that With First and Sword's code should not be used in any modules. Any resources from the game also most certainly should not be used. The devs have already spoken on this matter.

Interesting.

That's exactly my position about module decompiling. About Native's code... that's another story...
As you can see. You can't even compile that bunch of files. As the most part with the IDs are missing.


It's sort of ironical that I defended the right of the modding community about code privacy and rights and you're saying me that, Yoshi.
Now I was only trying to observe the changes implemented in the game... About the lack of Module System. I find it, discouraging.

Only tried to keep the spirit of the series alive.
Thanks for the warning, mate. I didn't know.

 

[Yoshiboy]

Yeah I understand the frustration and your motivations for the project, but unfortunately this is the developers position on the matter and so we have to respect that.

It should be fine to reconstruct individual files out of curiosity, inspection and personal use - this is something even covered in the laws about decompiling - providing you are not planning on releasing the equivalent of a WFAS module system, because I feel that would most certainly be against the developers intentions and also illegal.

Thanks for understanding

 

[Swyter]

If someone has interest in how my code looks like look here
I'm not accustomed to showing my sources, but this is all the main loop:

log("---------------------\r\nstarted!");
--// text files loader
for a,b in pairs(txt) do
	_G[cut(b)] = TextFile.ReadToTable(orig.."\\"..b); z = _G[cut(b)];
	log("-file "..b.." loaded  ["..itemsno(z[1],z[2]).."]");
end

--// info_pages procedural parser - finished!
local y = ""; -->
log("parsing "..INFO_PAGES[2].." info_pages");
for b=1,tono(INFO_PAGES[2]) do
	local g= explode(" ",INFO_PAGES[b+2]);
	
	g[1]= c(cut(g[1],"ip_"));
	g[2]= c(dash(g[2]));
	g[3]= c(dash(g[3]));
	
	y = y..tuple(g[1],g[2],g[3])..jmp();
end
inject("info_pages",y);

--// postfx procedural parser - finished!
y = ""; -->
log("parsing "..POSTFX[2].." postfx");
for b=1,tono(POSTFX[2]) do
	local p= explode("  ",POSTFX[b+2]);
	
	for h,j in pairs(p) do
		local g= explode(" ",j);
		if h==1 then --base desc
			y= y..tuplehead( c(cut(g[1],"pfx_")) , g[2] , g[3] );
			addID('postfx',g[1]);
		else --matrix
			y= y..tuplemed( t(g[1],g[2],g[3],g[4]) );
		end
	end
	y = tupletail( y )..jmp();
end
inject("postfx",y);

--// music procedural parser - finished!
y = ""; -->
log("parsing "..MUSIC[1].." music tracks");
for b=1,tono(MUSIC[1]) do
	local g= explode(" ",MUSIC[b+1]);
	y= y..tuple(c(cut(g[1],".ogg")),c(g[1]),flag('music',g[2]),flag('music',g[3]))..jmp();
	addID('music','track_'..cut(g[1],".ogg"));
end
inject("music",y);

--// strings procedural parser - finished!
y = ""; -->
log("parsing "..STRINGS[2].." strings");
for b=1,tono(STRINGS[2]) do
	local g= explode(" ",STRINGS[b+2]);
	y= y..spc(2)..tuple(c( cut(g[1],'str_') ),c( dash( g[2] ) ))..jmp();
	addID('string',g[1]);
end
inject("strings",y);

--// sounds procedural parser - finished!
y = ""; -->
log("parsing "..SOUNDS[2].." sounds");
--local t = {};
for b=1,tono(SOUNDS[2]) do
	local g= explode(" ",SOUNDS[b+2]);
	addID('soundfile',g[2]);
end

local x=SOUNDS[2]+2;local i=SOUNDS[tono(x)+1]+x+1;
for b=x+2,i do
	local g= explode(" ",SOUNDS[b]);
		local xx=tono(g[3]);
		if xx~=0 then
			sf={};
			for u=4,4+1+((2*xx)-1),2 do
				if g[u]~='' then table.insert(sf,c(id_soundfile[ tono(g[u]) ])); end
			end
			sf=table.concat(sf,',');
		end
	y= y..spc()..tuple(c( cut(g[1],'snd_') ), flag('sounds', g[2]), t( sf ))..jmp();
	addID('sound',g[1]);
end
inject("sounds",y);

--// quests procedural parser - finished!
y = ""; -->
log("parsing "..QUESTS[2].." quests");
for b=1,tono(QUESTS[2]) do
	local g= explode(" ",QUESTS[b+2]);
	y= y..spc(2)..tuple(c( cut(g[1],'qst_') ),c( dash( g[2] ) ), flag('quests',g[3]), jmp()..spc(3)..c( dash( g[4] ) ))..jmp(2);
	addID('quest',g[1]);
end
inject("quests",y);

--// skins procedural parser - 20%
y = ""; -->
log("parsing "..SKINS[2].." skins");
for b=1,tono(SKINS[2]) do
	local g= explode(" ",SKINS[b+2]);
	local h= explode(" ",SKINS[b+3]);
	local i= explode(" ",SKINS[b+4]);
	y= y..tuple(jmp()..c(g[1]),g[2],jmp()..c(h[1]),c(h[2]),c(h[3]),jmp()..i[1]);
	addID('skin',g[1]);
end
inject("skins",y);

op= getopcodetable("operations");

--// scripts procedural parser - 75%
y = ""; -->
log("parsing "..SCRIPTS[2].." scripts");
for b=1,tono(SCRIPTS[2]),2 do
	local g= explode(" ",SCRIPTS[b+2]);
	--h= explode(" ",SCRIPTS[b+3]);
	x = [[  #script_]]..g[1]..[[ >
  # Input: 
  # Output: 
  ]];
	y= y..x..tuple( c( g[1] ) , jmp()..'   '..t( opcode(SCRIPTS[b+3]) ) )..jmp(2);
	addID('script',g[1]);
end
inject("scripts",y);

log("done! :)")

Welcome to Lua. It's awesome. The son of CPP + PHP with Python reminiscences.
I've even added PHP functions like

explode()

And before anybody asks. Aha, that's hacky, raw code. I know, my variables are quite explicative.
Minimalistic coding.

 

[xenoargh]

I love Lua.

Oh, and there's nothing illegal about decompiling their stuff, nor sharing it.  Distributing something made with it, on the other hand, is not necessarily legal until they say so.  Thought I should clarify that.

 

[Pumpkin Lord]

Are you serious, now are you gonna create the WFaS module system yourself?

 

[Yoshiboy]

I love Lua.

Oh, and there's nothing illegal about decompiling their stuff, nor sharing it.  Distributing something made with it, on the other hand, is not necessarily legal until they say so.  Thought I should clarify that.

No, decompiling to source code is illegal under almost all circumstances. Exceptions being checking for malware, reengineering of legacy code, and recovery of globally lost source code. This certainly doesn't fall into any of those circumstances as the WFAS devs own the source code, own copyright on it and have explicitly stated and licensed saying that it will not be released.

Sharing it most definitely would be illegal and distributing something based upon it breaks a whole host of copyright laws.

Don't believe me take a quick look on google or I can hit you up with some links.

Sorry if it sounds blunt but I need to make it pretty clear that this is something not allowed. It is illegal or at the very least immoral. If you had a object you didn't want me to see, touch or use for my own purposes, it wouldn't be right for me to try and gain access to it.

 

[MadocComadrin]

No, decompiling to source code is illegal under almost all circumstances. Exceptions being checking for malware, reengineering of legacy code, and recovery of globally lost source code. This certainly doesn't fall into any of those circumstances as the WFAS devs own the source code, own copyright on it and have explicitly stated and licensed saying that it will not be released.

If this was, say machine code to assembly, I might concur; however, this is a lot simpler than that. We have the text files, and frankly, they aren't that hard to edit directly. It's a lot more like a direct translation than a decompilation.

 

[The Shadow]

The text files aren't exactly compiled into anything.  It's data that the engine reads.  I'm taking a shot in the dark without researching, but this is no different than editing the .brf files and putting those up as a mod.  Or taking a texture, making minor tweaks, and distributing that as a WF&S mod.  If the developers don't want people messing with the source code, then shouldn't they have some form of encryption?  Until then, these are just random .py file with no relation to the game.    No where in the file does it say copyrighted by such and such, when I click on the .txt file, it doesn't say anything about the assembly, unlike the .exe which is what is illegal to modify. 

By the way Swyter, great job. 

 

[Specialist]

The text files aren't exactly compiled into anything.  It's data that the engine reads.  I'm taking a shot in the dark without researching, but this is no different than editing the .brf files and putting those up as a mod.  Or taking a texture, making minor tweaks, and distributing that as a WF&S mod.  If the developers don't want people messing with the source code, then shouldn't they have some form of encryption?  Until then, these are just random .py file with no relation to the game.    No where in the file does it say copyrighted by such and such, when I click on the .txt file, it doesn't say anything about the assembly, unlike the .exe which is what is illegal to modify. 

By the way Swyter, great job.

Exactly!
In this case though, the project would only become illegal if you sell the WfaS Module System. Otherwise, nobody really can care, or stop  you.

 

[The Shadow]

I should say, on the other hand, that I can see why Yoshi might end up having to take these down.  Obviously, if the developers make a fuss, it is the TaleWorlds forum and it'd be the same reason why SWC and TLD can't have sub-forums here.  TaleWorlds is trying to run a business here.  Unofficial forums is one thing, but on their own...

 

[Swyter]

Thanks for the support, guys. I've even done a sort of enhanced/ better structured msys.
Which includes Python, even x64 binaries. Without the environmental variables hassle. And clear indentation. Everything in less than 10mb.


I'll probably release it.
At least for Warband and the Original game.

Might be useful for newcomers.

 

[theAthenian]

To say the truth I am glad WFAS won't be modded. I was afraid that everyone would move on to that game and abandon good old Warband...

Turns out that it isn't mod-friendly and many think it sucks by itself(including me) so I am relieved.

 

[MadocComadrin]

Minimalistic coding.

My number one fetish!

 

[GetAssista]

Nice effort.

It would still be major P.I.T.A. (or plain impossible) to reconstruct all the constants & name conventions used in the code for WFaS, like slots and such. Although most are hardly changed from WB, and I doubt WFaS developers brought a lot of new original stuff into the code.