Sadly the setting doesn't seem to save itself, you have to manually set it again after each time you load a game.
Anyhow, what kind of differences have you observed with the different settings ?
AI smarter ? Or just getting bigger armies and higher tier troops ?