Also I'm getting -9 from lack of rest even though I turned off rest effects morale.
Turn it back on, rest, check it is 0, and then turn it off.
I also have a problem with morale. It only takes about 1.5 days to go from Excellent to Average even if I win every battle AND give the men some loot AND have plenty of food AND am not at party size limit.
You can easily counter it with a variety of food. Easiest thing - buy chicken or pork in village, use it for a day and then sell in town for profit. Get boar meat, it doesn't go bad. Basically if you can get 6 items with morale modifier of 6 and above you should eliminate the party size penalty completely.