Has anyone stumbled upon interesting research around visual statecharts and state machines. Trying to come up with a really intuitive visual that captures the total power of something like x-state (fancy stuff like parallel state machines, recorded history on state machines....)

I'm not aware of any but I'd be willing to help drum up some research. As a nerdy poet without the imagination required to be a mathematician (look up David Hilbert's quote), I feel helping derive visuals of complex subjects may be the only contribution I can bring to the science that I love. Creating something like this could be the single most important contribution in the formation of a benevolent general artificial intelligence.

hehe, I love that quote

I’ve done a lot of practical work with statecharts. I lean towards cleaving paradigms apart - e.g. statecharts = (1) concurrency and (2) hierarchical state machines. I compile diagrams to code and am working on a description of the techniques. Interesting, related, stuff: FBP (Flow-Based Programming), Drakon, PEG parsing, Full Metal Jacket. My reading of Harel’s StateCharts paper is online.

The most interesting recent work that I'm aware of is InterState

I'm gonna shoot a video of my problem and send it over in a bit to give you fine folks a sense of the magnitude of the challenge I'm dealing with

I almost asked this same question! I ran into the concept of "behavior trees" while looking for stuff about HFSMs, and thought they looked interesting (BTs start on slide 12): https://web.stanford.edu/class/cs123/lectures/CS123_lec08_HFSM_BT.pdf They apparently originated in video game AI, but in the context of these lecture notes are meant for robot AI. They struck me as an intuitive and powerful state machine formalism, almost a general-purpose code formalism. Anyway, I would love to see field accounts of these ideas, especially if put in the hands of ostensible non-programmers.

Why don't we have a group brainstorm session if we still need to explore it?

So we're using state diagrams in a peculiar way which is likely different. We kinda took the concept and added a ton of bells and whistles to it and made the state diagram the backbone of defining the behaviours of your backend. The state diagram is hyper away of the rest of our abstraction (our pre-built auth system, our permissioning system) and that's a big part of how we decrease the surface area of code.

Have you read Harel’s original paper? It has visuals for both problem 2 and 3 in your video.

I'm actually reading through right now

This might sound glib, but I'm 100% serious. The best solution I've seen to problem #1 (how do you get people past the initial unfamiliarity / reluctance?) is to make your GUI look enviably cool. Make the GUI just explode charm, style, je ne sais quoi, mystery. Make it draw people in. Make it look like nothing they've ever seen before. Make it elicit and reward curiosity. Don't allow people to grossly assume that they know what this GUI is, what it does, and that it's not worth the 20 minutes of attention to learn how it really works. This is a very hard problem, but it's not an NP hard problem. It's do-able, and people do it all the time. (I answered a slight variation of your question, because the other half of the question — how do you make the GUI self-revealing to new users — is something it sounds like you already have a good idea of how to do. Click targets, tooltips, progressive disclosure... typical UI design jazz.)

^ great feedback. I should have added one crazy caveat, we have a "no progressive disclosures" rule right now which makes every design challenge infinitely harder. The reason for this is we think there's going to be some weening required to get devs acclimated to even the slightest GUI mostly cause of legacy rhetoric. So the idea behind our UI is that it has to be entirely keyboard navigatable. So we figured that out on storage with great difficulty. On the behaviour page (the statechart) obviously it's impossible. But we're still holding out in hopes that we'll come up with a way to structure this diagram to make it more auto-layout and then, hopefully intuitively keyboard navigatable. There aren't that many great ways to do a progressive disclosure and still keep the hand away from the mouse.

You might consider more of a railroad-diagram look (or at least organizing principle). Maybe oriented vertically rather than horizonal like the common examples (it will be more compact and look more like code that way for skeptical devs). I agree that auto-layout is really important for something like this, as I never ever want to fiddle with box positions.

I seriously doubt that xstate is the "state of the art" of statecharts. I bet you'll probably find more inspiration looking at Yakindu that is a really polished (alas, non-free) implementation of statecharts (including modeling orthogonal states). When it comes to GUI, I like the Enso approach: hybrid/bidirectional visual and textual syntax. There are lots of projects like SMC that define a special DSL. I imagine I'd spend most of my time on the textual mode if I had that, and then use the visual one to browse. A good textual DSL with a cool browsable visual representation (perhaps even retaining just some interactive editable elements) seems like an easier sell than a "1000-clicks-no-code-ui", but that's just me 🙂 . Good luck with your project! Looks pretty cool.

Hard agree with @Ivan Reese on the value of aesthetics here. See: Don Norman's book on Emotional Design for a longer treatment, with a brief preview of the material here: https://www.interaction-design.org/literature/topics/emotional-design

(I failed to add a link to my reading of the StateCharts paper: https://guitarvydas.github.io/2020/12/09/StateCharts.html) (HTH).

That link at the bottom is gold. Very hard to read Harel's original paper given how the schematics are laid out. Thanks for sharing

@Mo the loom video 404’s now, but i’m extremely interested in this, especially the keyboard-navigability part. one of the things i harp on a lot is that you can in fact design a set of ui primitives that match the primitives of the algebra for the data you’re working with (vim pretty much does this for text, although the actual commands tend to be obscure and not right for editing most code)

Here is my understanding of your problems: 1. problem #1: on-boarding 2. problem #2: user history in the tool 3. problem #3: allow users convenient access to parallel state machines [I think that you are generating code from users’ diagrams. I think that you are joining the generated code to the XState library. There are 2 different use-cases: (1) designing your tool, (2) generating code from users’ diagrams. XState can be used for both use-cases (or just one)]. 1. problem #1: How to get users to love dealing with details. They don’t. Give them eye-candy until they buy-in. Problem #1: Yes, designing using StateCharts makes you think about details. Users live in a fictional reality hoping that you will solve all of their detailed problems. That’s why neophytes like to use BASIC and Excel (and JavaScript, Python, etc). Problem #1: Give them layers, but, don’t make them use layers immediately. Users want “flat”, but that leads to later trouble. Give them flat until they need more. (Also, see Ports, in #3). Don’t mention the fact that the “flat” layers will interconnect, once they need them to. 2. problem #2: History. The UX depends on your users - what do they want to see? Your job boils down into (a) designing a decent UX and (b) hooking what users see, in the UX, into the generated XState code. 3. problem #3: Parallel State Machines. Harel’s notation for Orthogonal States falls short (IMO) here. in StateChart notation, state transitions are explicit (curvy arrows) except when you use Orthogonal States, which can only be implied by reading the code. How to present this to users? My suggestion: Peel orthogonality out of user-level HSM’s and put explicit ports on the user-level HSM’s (HSM === Hierarchical State Machine). Join ports up using explicit lines (arrows, different color, whatever). Design it so that the users must (explicitly) wire up output ports to input (watcher) ports. Design it so that every user-HSM is completely self-contained. Design it so that a user-HSM can only send “state-changed” events to one of its own output ports. Every user-HSM can observe (listen-to) “state-changed” events from other user-HSMs only-iff their ports are connected in the UX. Can an HSM send other kinds of events to its output ports? (A: yes. Ask me for more discussion, if this interests you. I am happy to discuss further). 4. problem #0: Go to a whiteboard and draw the design of your tool on the whiteboard. (Use StateChart notation, if you wish). (Most programmers skip this step and go straight to implementation using JS, Python, Rust, etc.)