Currently, as our investigations show, the profitability problem stems from a lack of relative scarcity of goods. I.e. all goods are prevalent in all areas (while it's possible for player to exploit certain trade routes with use of intuition, AI uses a more algorithmic approach and cannot find this routes.) Also it's not ideal to maximize the profits but also improve the distribution of the goods as well. Minor tweaks to prevalence of many items through increasing demands actually helps with the average profit of caravans a lot. (About %50 increase in profit) This is the current way we are approaching the problem. Of course we still require manual testing so that the trade is fun and meaningful to the player character as well, but we can see from the data that the caravans benefit from high demand and scarcity.
Another additive benefit of this scarcity is to introduce some heterogeneity accross the lands of Calradia, where different places have very different market configurations (not just high number of the goods produced by the bound villages, but e.g. no dates in Sturgia).
For the trade penalty of the caravans, yes we can change them, however currently trade penalty is already very low for caravans (about ~2-3% of the price for buying and selling.) Though we will look into it as well.