You're working with the assumption that settling in large groups, creating a "state", and tending to agriculture represents progress for the people involved. I'd recommend checking out Scott's "Against the Grain: A Deep History of the Earliest States" for an alternative look at those assumptions.

https://www.amazon.com/Against-Grain-History-Earliest-States...

For evidence based narrative of the oldest known state like agricultural cultures I can't recommend enough James Scott's "Against the grain" https://www.amazon.com/Against-Grain-History-Earliest-States...

All evidence at this time points that agriculture based city states were challenging to boot into a "proper civilization". If you raised taxes, your citizens would just leave since the area you could control was quite small. And then you get new diseases that suddenly could spread dense in urban population - wars - all that good stuff.

Based on the book, maintaining a city state over more than few generations, before the knowledge how to keep it going has been learned, is really, really really hard.

This is not to counter the suggestion that there could have been cities at e.g. coasts tens of thousands of years ago, but the fact that they seem to be so frail in the beginning points out that it's not also improbable that first "higher civilizations" were indeed the ones we know of.