I use Realistic Battle Mod which makes the AI even more defensive, and yes, I do find it annoying when a low tier enemy turtles behind his shield when all I want to do is quickly kill him so I can get to where I'm going. But c'mon, that is exactly what a scared soldier would do.
Battlefields are not the place for duels. As said, flanking is the best way to tear apart your enemy foot soldiers. And quite frankly I wish the soldiers would be even more defensive. Even with Realistic Battle Mod I don't have time to attempt maneuvers. I'm trying a Cav-less Sturgian army this round, and I would love if I could use my line-breakers effectively to either flank or to... well... break the line. But then I'd need some shielders to protect them from the enemy ranged, plus split my regular troops into two, three or maybe four divisions so I can either spread them out or pull back, plus have a reserve which I could use to add men to flagging divisions.
But this will never happen. In real life two equally sized battle lines could fight for hours until exhaustion set in. A soldiers first and main priority was staying alive. Spears kept everyone a healthy distance away from each other, and the shields were never ever lowered. Now I'm not saying every battle should last several hours, but even if you had two equal armies fighting face to face with no flanking or any tactics it should at least last ten minutes.
So I say lets make AI even more defensive, and give recruits shields. It's odd that the worst soldiers would not have one of the cheapest pieces of equipment to produce.