I don't know as for the others, but I for one would like to play Bannerlord, and not Warband, while not having the age feature operating in my game. I'm staying with my opinion regardless.
As it's an early access game, I won't yell "refund" or whatever like there was some "no kingdom" madness going on around. That one feature can and will be modded easily, with a workaround via the developer console of main character's age modification being one available right now. I'll use that personally, because I can.
I also think that since this "core component" is being discussed quite intensively then that could be feedback upon which TaleWorlds could see about. Feedback's feedback after all.
Overall, the game's really good, and the amount of care provided by constant patching & updating is neat. Keep the good work up and don't let it stress you out!