"Engine" in this instance is the .exe core that processes things like battles, graphics rendering, and translates the game campaign scripts, etc into action. The engine is indeed Warband's (as one can see from how battles work, graphics are rendered, etc).
"Module system" is the source of all the campaign scripts, item definitions, troop definitions, etc, etc, that gets compiled into the myriad of .txt files within the \Modules\<Module Name>\ folder. It provides the instructions on what to do and then the engine carries it out. The module system here is a mix of original M&B campaign scripts (with the lack of a player faction, marriage, etc) and Warband multiplayer mode definitions, etc.
(Another clear distinction, the "module system" is open for Warband and vanilla M&B and contains things modders can change to their heart's desire...the "engine" is not open for modification, without breach of license.)