I've been working on a voice driven product, and it's got me thinking about programmability. Specifically users start to learn a set of phrases to accomplish things, which can become the building blocks to string actions together. It's kind of natural language programming, but I'm not confident that's the right way to think about it... just because the individual phrases are natural language doesn't mean the assembly should be. Anyway... I don't have any particular question or assertion here, nor have I thought about this for long, but I thought I'd mention it because I'd be interested in any thoughts.

@Ian Bicking I was thinking of a solution for natural language programming a while ago. This is what I came up with: Transform English to code A simple parser will try to convert English sentences to code with the following steps: The verb of the sentece will be the function that’s going to be called. For the moment, if the sentence has no verb it’s an error. The other words are the parameters of the function. Articles, prepositions, and other words with no meaning will be ignored. The parameters are converted to a Hash. The keys are the categories of the words, the values are objects with the words and more information. The hypernyms (https://en.wikipedia.org/wiki/Hyponymy_and_hypernymy) of a word are the candidates for the category (this is IMO the most complex problem to solve for the project, might need some heuristics). For example the sentence:

Draw a red circle

Would get transformed to the code:

draw(color: 'red', shape: 'circle')

I was thinking about it for a text based language, here is the POC I did https://github.com/nlpl-lang/nlpl-editor

Ah I forgot about how threads are used here, apologies Your design reminds me somewhat of mozilla lab's canned ubiquity project, though voice driven. Though it seems to me that your project aims to be more composable than ubiquity? Incidentally, ubiquity got ported to webextensions, and thus can be tried out here: https://gchristensen.github.io/ubiquitywe/

@Ian Bicking I assume you're aware of Inform 7? We discussed this talk here a while back: http://inform7.com/talks/2018/06/09/london.html

I periodically remember and forget inform ;) I will check out that talk! And I'm hoping the project I'm working on can revive some of those ideas from Ubiquity (https://github.com/mozilla/firefox-voice)

The time is definitely ripe for revisiting speech-to-code. Between neural NLP and program synthesis, all the tech is now there there for an effective and usable speech interface for inputting lines of code. Someone just needs to do it.

i’ve had this half-idea brewing in my head: i think you want to start from the api rather than the speech recognition end of things. any voice interface can be summed up as a fuzzy search over existing capabilities—you don’t have to understand all of grammar to interpret that, and if you could understand all grammar it wouldn’t be much help if your api can’t do the vast majority of requests you could make

Yeah that’s a reasonable way to approach the problem