As my base of power, I try to got with a town and at least 2 castles. It can take awhile for your renown and relations to grow enough for the king to give you a few walled fiefs, but it will be there soon enough.
I've not seen much difference between castles per se. Perhaps if you plan for a trap fort to siege defend to death attacks on your budding kingdom?
So I'd rather think in terms of the town.
First is the faction you want to join, if you have one of their original cities, it can help avoid a two-front war against your old king, and the king whose culture your city belongs to.
Second are KOs. If you want to use one of the non-qualis orders that are tied to a city as your "spam order", pick that city or next to it.
Other considerations:
Singal has slave trader, which is nice.
Holding a fief near the Noldor Woods causes friendly parties to occasionally dump noldor prisoners for you for free. Also when you are friendly with Noldor, their patrols can help drive off raiders.
Holding a central town can help a lot in the mid-game, where going off to do stuff (like hunting for a Qualis army) sees you going there and back again faster.
But in the end, any will do without too much difference.