Does anyone have tools / processes to recommend for rapidly iterating the design of a language? I’m looking for strategies to produce a document that captures the design, evolution, and potential variations of a language interface separate from implementation concerns. Good examples of this would also be very much appreciated!

There is Gramada: https://github.com/hpi-swa/Gramada

I haven't used it, but PLT-Redex is a racket lang for roughly the same task. There's a series of lectures on how to use it on YouTube from (IIRC) the Oregon programming languages summer school. For anyone who has used it, I'm also very interested in your experience.

These are both great systems that I hadn’t heard of, thanks! But I’m thinking of a high-level document, that uses purely speculative code examples (no implementation) to explore language design before diving into defining grammars or informal compiler design. To ask in a different way, if you asked a relatively novice programmer to create example programs informing the design of a new language, what would that document look like? What are effective language “sketches” and how do you show iteration and variation? I tend to just dive in and start implementing after writing down a few small code snippets, but I’m hoping that people here might have experience with other ways to approach design before beginning implementation.

I've been working on language design for a few years now, and for design work I've not found any better solution than interlinked textual notes (Roam is so much better at this than anything else) plus digital sketches (I use an iPad + Pencil, and just Apple Notes). I export the sketches into the note app (screenshot -> Airdrop to laptop).

the best i've come up with so far is Markdown, using Typora (WYSIWYG). I write and refine all my random thoughts in text, and use the code blocks for trying out the syntax. It sorta works because whatever syntax I'm working with, I try to pick a language for the code box that matches it. Minimally i can usually get keyword or operator highlighting and it makes it easier to read.

Reviving this thread 👋 One thing I’ve found helpful is to consider: What sorts of things do you want people to be able to do with your system? What goals can they accomplish? How will their thinking be changed by using your system? Why should they turn to it, instead of something else? Don’t think about how you’ll do this yet, just focus on human needs that need meeting. From there, you can start to sketch out ideas (I’d start on paper, as it’s the most free form). Remember the needs and contexts of the people you’re trying to help! Are they best met with text files (and things like git, other source code tools) or are they better met with something more graphical? (Or really far out: are they best met with an entirely new kind of computer??) At this point, I like to move into something more like a drawing tool (I use Sketch on the Mac) that lets me mock things up and write notes, spatially. I like to imagine different kinds of UIs for solving issues, and then I can write notes along side them, etc. As far as actually prototyping these things, that’s where I struggle to do so rapidly (as far as real working systems go). But I think there’s something very powerful (and cheap!) about mocking things up in a drawing tool (or in a text file — but beware, if you do everything in a text file, you’ll probably narrow your ability to work on programming UX things provided by your environment)

@Jason Brennan the original thread seems to be about designing languages. I interpret that as about designing textual programming languages. Does prototyping those still have a limitation with text files? (Earlier phases certainly benefit from the ability to draw lines and boxes and so on.) In general I'm a bit bemused by this thread. As a programmer it all feels quite waterfall-y. If you try to "mock up" a language before giving it a grammar you're quite likely to end up introducing ambiguities that need modifying the language to resolve. Similarly with many other aspects of the activity. If an activity eventually needs to be mapped on non-linear primitives (as computer programs do), it seems to me that it benefits from lots of feedback loops. Funny story: the original waterfall diagram [1] had feedback loops between phases, so even waterfall practiced right can be quite effective. Of course, that's not what anybody means by the term anymore.. [1] http://www-scf.usc.edu/~csci201/lectures/Lecture11/royce1970.pdf, figure 3

Yep! I still think there’s a ton of benefit to mocking this stuff up in a drawing of some kind, even if you’re doing a textual language. There’s lots of “programming experience” I hope everyone in here aims to include for their languages, like auto-complete, debugging tools, visualizations, etc. In my view, it’s better to design these things along with the language itself (not as goodies added on later), as they’ll help you steer how you want the language to work. Of course, as you say, it’s still important to have a somewhat working implementation to know the limits of what can be reasonably expressed in your language, but I look at that more as a limitation than a guiding tool

The tendency of grammars to become ambiguous is not a limitation of some single tool, it's more like a limitation of the universe you live in. It's a little bit like the speed of light. I suppose you could see it as a technical limitation of current technology rather than a property of the universe, but either way it seems as unfair as asking a carmaker why they haven't managed to add a fusion drive yet to their creation. If you create an ambiguous grammar, all your tools like autocomplete, visualizations, etc. have to now deal with the grammar. And the drag (amount of implementation needed) compounds over time. Perhaps this is getting off-topic. I'll just reiterate that programming isn't just something you do once you know what you want. It's a tool in the toolbox for arriving at good designs. "Writing doesn't just communicate ideas. It generates them." -- Paul Graham (http://www.paulgraham.com/writing44.html)

Ohhh, @Kartik Agaram, have you read A Timeless Way of Building? If you haven’t, you’d probably enjoy it.

I've read bits and pieces of it as others have pointed them out. But I really ought to. Particularly since I cited it in http://akkartik.name/akkartik-convivial-20200607.pdf 😕

I never really got that Alexander had generative grammars in mind for what he calls pattern language. But that becomes quite clear in ATWoB. That elevates the concept to pretty much your description above. Obviously, I can’t make you read a 500 page book, but please do and then tell me (us) what you think. :-)