Only halfway through the latest podcast ep but my imagination is piqued. Is there a version of, like, unit testing, where the software generates some randomized states based on its internal state and unknown inputs, presents the outcome as part of a report, and asks humans to correct any erroneous assumptions or missing contingencies? IDK how to even describe what I'm thinking of but I feel like it must exist, or maybe be theoretical in the emerging AI models of creating software. Kind of fuzz testing, I guess. My brain is going somewhere bigger with this, though, maybe I'll check in with it again once I find time to finish the podcast.

Have you seen quickcheck (in Haskell) or similarly inspired libraries in other languages (like hypothesis for python)?

Those are new to me but I have heard of property based testing! It was on my list of things to learn about, then I forgot. Thanks for the pointer!

“generative testing” is the broadest umbrella term

Also reminds me of program synthesis—the idea is that you go from human provided examples to an algorithm. Then there’s also a quick check related tool that infers properties from your implementation (commutativity, monotonicity, etc).