Grant Forrest

07/08/2023, 11:39 PM
Only halfway through the latest podcast ep but my imagination is piqued. Is there a version of, like, unit testing, where the software generates some randomized states based on its internal state and unknown inputs, presents the outcome as part of a report, and asks humans to correct any erroneous assumptions or missing contingencies? IDK how to even describe what I'm thinking of but I feel like it must exist, or maybe be theoretical in the emerging AI models of creating software. Kind of fuzz testing, I guess. My brain is going somewhere bigger with this, though, maybe I'll check in with it again once I find time to finish the podcast.

Naveen Michaud-Agrawal

07/09/2023, 1:34 AM
Have you seen quickcheck (in Haskell) or similarly inspired libraries in other languages (like hypothesis for python)?

Grant Forrest

07/09/2023, 2:01 AM
Those are new to me but I have heard of property based testing! It was on my list of things to learn about, then I forgot. Thanks for the pointer!

Jack Rusher

07/09/2023, 9:03 AM
“generative testing” is the broadest umbrella term

Justin Blank

07/10/2023, 12:41 PM
Also reminds me of program synthesis—the idea is that you go from human provided examples to an algorithm. Then there’s also a quick check related tool that infers properties from your implementation (commutativity, monotonicity, etc).