https://futureofcoding.org/ logo
#share-your-work
Title
# share-your-work
m

Michael Gummelt

11/10/2023, 7:37 PM
We're experimenting with a novel form of "Programming by Example", similar to Excel macros: https://www.plato.io/blog/introducing-macros I think it can scale quite well to a complete programming system with branching, looping, etc.. Thoughts?
m

Mike Austin

11/10/2023, 9:08 PM
How would branching work? Drag/press a condition button? Record two macros and mark them as branches visually?
m

Michael Gummelt

11/10/2023, 9:11 PM
@Mike Austin We're experimenting with a few variants. The most straightforward is your first suggestion, but the challenge is maintaining a consistent state. Macros are recorded with a "recording context", or an example of the parameters needed to run the macro (e.g. a single row of a table). But once you introduce conditionals, those parameters might not be valid for all branches (e.g. update if a record exists, insert if it doesn't). So one option is to prompt the user to choose a different context (e.g. pick a different row). Another option is more in line with your second suggestion (took us longer to come up with it, though :)). The user records a macro for a single context, then we alert them to a failure or possible failure for other contexts (not always possible, but usually is), and then they record a separate macro for that context. This is more in line with classic "Programming by Example" systems such as the one described by Bret Victor (http://worrydream.com/MagicInk/), where multiple examples are provided and the system tries to generalize.
b

Brian Hempel

11/10/2023, 9:51 PM
where multiple examples are provided and the system tries to generalize
This approach fails. Just have the user write the branch condition explicitly.
m

Michael Gummelt

11/10/2023, 9:54 PM
Why do you say it fails? To clarify, we're not trying to do any "smart" inference. Just trying to come up with a scheme that's more inductive. For instance, if you record a macro that accepts some input, searches for a row, then updates the row, that may work for your first example, but it will fail when the row doesn't exist. We can prompt the user when they hit that failure, and ask them what they want to do. I agree it won't work in all cases, but we think there might be common patterns that will get us pretty far.
b

Brian Hempel

11/10/2023, 9:55 PM
Is this representation editable, or must you redo the demo if one of these inferences is wrong?
m

Michael Gummelt

11/10/2023, 9:55 PM
It's not editable yet, but we're working on it.
Also, those aren't inferences, they're recordings of explicit user actions.
b

Brian Hempel

11/10/2023, 9:59 PM
We can prompt the user when they hit that failure, and ask them what they want to do.
Maybe Eager did this? It sounds okay. But what you don’t want to do is implicitly try to infer the branch condition. I’ve seen many PBD systems and the amount of work to demonstrate to try to get the branch condition is very much greater than the amount of work to just say the branch condition.
m

Michael Gummelt

11/10/2023, 10:01 PM
I didn't know about Eager. Thanks! Are you aware of any PBD systems that only record explicit actions? All of the research projects I'm familiar with try to do an "observe and infer" approach, which I agree is too fragile.
For instance, our plan for looping is to make the user explicitly say they want to run a loop over a collection, rather than to try and infer that they're repeating actions for some set of elements.
b

Brian Hempel

11/10/2023, 10:08 PM
Yeah less inference is better. If you could also add the loop after the fact, that’s even better. That’s where the editable program representation is helpful.
m

Michael Gummelt

11/10/2023, 10:10 PM
Yea, SmallStar is the most practical I've seen. But all of these systems operate in the desktop metaphor. Have you seen any that operate on structured or relational data? That's our context, and the problems end up being quite different.
b

Brian Hempel

11/10/2023, 10:13 PM
“Why Programming by Demonstration Systems Fail: Lessons Learned for Usable AI” Tessa Lau. 2009. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=37c50b0cc3c701b269c18014dd3a0c07f9c2b4b8 Most of these are solved with an editable program representation.
m

Michael Gummelt

11/10/2023, 10:15 PM
Awesome. I have some nightime reading 🙂
b

Brian Hempel

11/10/2023, 10:18 PM
Have you seen any that operate on structured or relational data?
Surprisingly, not off the top of my head (nor in my quick search). Closest I can find is Gneiss, which is great in its own right. https://www.cs.cmu.edu/~shihpinc/gneiss.html
m

Mike Austin

11/11/2023, 2:30 AM
Wow, SmallStar goes waaay back. I was going to say something like PhotoShop actions.
m

Michael Gummelt

11/11/2023, 2:31 AM
Photoshop actions doesn't solve the hard problems like conditionals, AFAIK, though.
m

Mike Austin

11/11/2023, 5:16 AM
True. Just one example of macros that came to mind. I'd imagine they would have solved some interesting problems being how old PS is, though. I haven't use it in many years.
j

Jack Rusher

11/11/2023, 7:36 AM
As strange as this might sound, the most powerful and usable PBD system I’ve ever seen is… emacs keyboard macros (the name’s a misnomer, they also record other types of events, like mouse gestures).
2 Views