We're experimenting with a novel form of "Programming by Example", similar to Excel macros: https://www.plato.io/blog/introducing-macros I think it can scale quite well to a complete programming system with branching, looping, etc.. Thoughts?

How would branching work? Drag/press a condition button? Record two macros and mark them as branches visually?

@Mike Austin We're experimenting with a few variants. The most straightforward is your first suggestion, but the challenge is maintaining a consistent state. Macros are recorded with a "recording context", or an example of the parameters needed to run the macro (e.g. a single row of a table). But once you introduce conditionals, those parameters might not be valid for all branches (e.g. update if a record exists, insert if it doesn't). So one option is to prompt the user to choose a different context (e.g. pick a different row). Another option is more in line with your second suggestion (took us longer to come up with it, though :)). The user records a macro for a single context, then we alert them to a failure or possible failure for other contexts (not always possible, but usually is), and then they record a separate macro for that context. This is more in line with classic "Programming by Example" systems such as the one described by Bret Victor (http://worrydream.com/MagicInk/), where multiple examples are provided and the system tries to generalize.

where multiple examples are provided and the system tries to generalize

This approach fails. Just have the user write the branch condition explicitly.

Why do you say it fails? To clarify, we're not trying to do any "smart" inference. Just trying to come up with a scheme that's more inductive. For instance, if you record a macro that accepts some input, searches for a row, then updates the row, that may work for your first example, but it will fail when the row doesn't exist. We can prompt the user when they hit that failure, and ask them what they want to do. I agree it won't work in all cases, but we think there might be common patterns that will get us pretty far.

Is this representation editable, or must you redo the demo if one of these inferences is wrong?

It's not editable yet, but we're working on it.

We can prompt the user when they hit that failure, and ask them what they want to do.

Maybe Eager did this? It sounds okay. But what you don’t want to do is implicitly try to infer the branch condition. I’ve seen many PBD systems and the amount of work to demonstrate to try to get the branch condition is very much greater than the amount of work to just say the branch condition.

I didn't know about Eager. Thanks! Are you aware of any PBD systems that only record explicit actions? All of the research projects I'm familiar with try to do an "observe and infer" approach, which I agree is too fragile.

Yeah less inference is better. If you could also add the loop after the fact, that’s even better. That’s where the editable program representation is helpful.

Yea, SmallStar is the most practical I've seen. But all of these systems operate in the desktop metaphor. Have you seen any that operate on structured or relational data? That's our context, and the problems end up being quite different.

“Why Programming by Demonstration Systems Fail: Lessons Learned for Usable AI” Tessa Lau. 2009. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=37c50b0cc3c701b269c18014dd3a0c07f9c2b4b8 Most of these are solved with an editable program representation.

Awesome. I have some nightime reading 🙂

Have you seen any that operate on structured or relational data?

Surprisingly, not off the top of my head (nor in my quick search). Closest I can find is Gneiss, which is great in its own right. https://www.cs.cmu.edu/~shihpinc/gneiss.html

Wow, SmallStar goes waaay back. I was going to say something like PhotoShop actions.

Photoshop actions doesn't solve the hard problems like conditionals, AFAIK, though.

True. Just one example of macros that came to mind. I'd imagine they would have solved some interesting problems being how old PS is, though. I haven't use it in many years.

As strange as this might sound, the most powerful and usable PBD system I’ve ever seen is… emacs keyboard macros (the name’s a misnomer, they also record other types of events, like mouse gestures).