A big part of the problem lies in the fact that village defense seems to be very low on the AI's list of priorities. Any lord who isn't in an army is either off raiding the enemy or riding around gathering troops, weak and unable to help much. Relief armies for besieged cities and castles are common but it seems like the only time they defend a village is when they happen to see one under attack on the way to something "more important."
I miss the Warband ability to suggest that a friendly lord go and defend a particular village. They'd usually listen if you had decent relations and you could coordinate an effective defense without needing to personally be everywhere at once.
Also, we should be able to donate food/draught animals/money to raided settlements to get them back on their feet. Hell, we can choose to donate looted equipment to our troops for XP, why not also allow us to donate gear to settlements to produce more and/or stronger militia?