Restriction to cut down magic in sieges is artificial IMHO.
Its natural it excells at sieges. I did sieges in Phantasy Caldaria it wasn't OP - it was very powerful. Mages die from one arrow hit, and higher level troops had resistances for magic.
The problems you mention its coz there walls make people cluster but don't give protection at the same time.
It is because beacuse there is no collision check in spell scripts.
There is a solution here that is good and diminishes the problem (and in general makes things more realisitc)
Explosion code that takes into account obstacles and already hit units
a) Take three orthogonal vectors that pointed from explosion to two forward directions and one backward and name them (L, R, B) Its important to take left-forward, right-forward and back-up.
b) See if those points are visible form center of sphere. ( *position_has_line_of_sight_to_positiion*),
Multiply vectors by the the visibility weight, VI, Vectors should have been normalized previously.
Visibility weight should be 1/3 if point is visible form center or less than 1/3 (1/9 lets say) if not.
c) When you apply damage (my guess is you're using granade code) - and must calculate relative position of agent to the center of explosion.
Normalize this and name following way P = (AgentPOS - Explosion_POS) / length(P)
Your damage is (TOTAL_DAMAGE is whatever you had before)
TOTAL_DAMAGE_PHYSICAL = TOTAL_DAMAGE * (
max(0, VL * dot (L, P)) +
max(0, VR * dot ( R, P)) +
max(0, VB*dot(B, P))
)
You now have damage that accounts existence of surfaces in vinicity of the spell its approximatly accurate but it 100 better than not taking cover into account at all.
If you want more accuracy you sample with more vectors.
I may edit this post to replace formulas with actual code.
1. Note that you have change one script, the damage calculation - NOT every spell if gsanders was involved I'm sure he made the damage calculation "grande" code separated.
2. This is very performant - you make visibility check three times per explosion, not per enemy!
What is the second issue with spells in sieges - the fact that is strategically good choice?
If you don't want spamming apply miscast chances for bad outcome should raise (i.e 5% for damage each time u cast a spell)