I’d suggest building a game engine and a sandbox on top of it, and developing the two together, so the sandbox drives what the engine needs and the engine opens up new things to try in the sandbox.
In the sandbox you could have small situations to resolve, on grounds about the same size as the ones you have now, like a flooding river over the main road in a diorama, or clearing fallen trees and felling new ones. Trees would be a great one for physics too, because they are usually not modelled very well in games.
There are many open-world games already, but with Ground Works I am just happy to dig in the first place. If I didn’t have to deal with grand strategy and could simply enjoy the mud, rocks, sand, dust and trees, that would be something I’d find really interesting.
Each diorama could be a puzzle to resolve. Also you can vary the topic, like farming, mining, felling, demolishing building, paving road, etc.
So these situations first, the engine matures, and then you can think about something bigger.
Update: oh and also water, flowing.