I examined situation. There are several reasons for this :
1-There were so many defections. Today with new patch defection probability is decreased and defections are rare now. Do not forget during defection lords also take their settlements with them to new faction. Weak factions were losing their lords and settlements to strong ones this was not so frequent but even 1-2 in a year this was effecting huge.
2-There are lords going in financial crisis stay with no money and they cannot recruit men. (will be examined tomorrow)
3-There were big starving penalty for starving castles and towns. For each 4 missing food 1 garrison were dying. This effect is now for 8 missing food with today's patch. Because they were dying they were making sally out and losing their defensive bonus.
4-There were so many sally outs and during sally out garrisons were losing their defensive bonus. This probability is decreased with today's patch now sally outs are rare compared to previous versions.
5-Currently there is no war decleration mechanic to a very powerfull kingdom or making peace mechanic to get unite aganist most powerfull kingdom. (will be added in next months, not quick one)
I continue searching for other reasons. However do not forget in Bannerlord there is no stable world even we fix these problems one faction can rule all the world without player interaction but this should take more time like 20 years maybe. We are working on that. If you want to show you effect on world is much conquer all world earlier than 1090.
Thanks for your reply. Even in early access, the game is already great.
I want to play for my child, but so far the game ends too quickly.
I hope in the future, there will be an opportunity to play for more than 15 years and see a generational change.
I hope the mechanics of "aging and children" was introduced for a reason. And the game will be long enough to reveal these mechanics.