I would say when you in one way or another have power equal to a large faction. Either by creating one, being voted the leader of one (that's in a good position/powerful) or have independently taken a factions worth of of fiefs demonstrating your clans power to take out a faction.What's the definition of late game?
I would say it's when the "struggle" has ended and you can confidently take on any enemy and grow your faction/holdings as the apex power.
I doesn't mean you have too, you could take it slow, but just having that power make it a different game state then before.