I was somehow able to accidently move a layer below the base layer, despite the popup telling me I couldn't. It was an accident so I'm not 100% sure how I did it, but it did mess up all the rest of my textures. I've attached a screenshot with the popup and the layers flipped (previously rock was above default).