The problem is bad design; if there is a food deficit, population growth and prosperity growth should slow down and at some point come to a halt. There can't be significant growth when there is a food deficit. In real life, if a prospering city has reached the size where food becomes scarce; price of food will increase by a lot, poorer people will start leaving the city and merchants will be motivated to come do food runs to the city.
Maybe what they mean by prosperity is actually population. If there is a lot of population in a city, but not enough food, the people will start going hungry and angry toward the local noble.
There should be a way to lower prosperity, for example by removing prosperity buildings.