This comes dressed up as advice for game designers, but don’t let yourself be distracted by that — it’s solid advice for anybody trying to build any kind of system: https://www.derekyu.com/makegames/deathloops.html

Derek Yu puts out a lot of good advice about indie game development, which as you said applies to a lot of domains in software