📋 How Visual is Your Language? Semantic Mutation Testing How many "box and arrow" languages are just scratch with arrows between the blocks? https://instadeq.com/blog/posts/how-visual-is-your-language-semantic-mutation-testing/

Very useful idea. In some instances the lack of semantic meaning from mutating the dimension, such as order, is "correct", if you want to visually communicate that order is irrelevant. So not using a dimension may be desirable. Making visual interconnections provide more information by providing shapes also runs into problems when using un-typed or dynamically typed language. Doing it in typed languages, you quickly run out of useful shapes. I've seen people try to use color to solve that problem, and it's visually jarring.

Consider that for many box arrow tools, layout has about as much semantic content as whitespace in textual languages. More completely, layout gives you more: layout also is like the order of things in a source file (often not semantically relevant to the computer but important to authors) and still more. I was going to say layout is like a comment, but the whitespace analogy felt too strong.

My problem with layout is that the default is messy and requires active work to make it "tidy", with white space the default/least effort option is the "tidy" one. I say it because I built a box and arrows[1] tool and I saw how much time was spent rearranging the layout for no functional reason. Same with the dashboard designer, it was free form, people spent ages aligning stuff (there layout had a meaning at least) [1] https://www.event-fabric.com/

I think the main advantage of 2D in programming is not the ability to move or orient things, but the ability to zoom, which is way under-utilized. If I call a library function, I should be able to zoom in on the visual element for the call, and "inside it" see the code for the function. Zoom could make all kinds of nested graphs easier to navigate. Vlojure does a version of this.

@Jason Morris No reason that plain text programming couldn't do more to show items related to the focused part.

This is something discussed in “_The ‘Physics’ of Notation_” paper (https://futureofcoding.slack.com/archives/C5U3SEW6A/p1647551973870069) . This screenshot below is referenced from from the “_Semiology of Graphics_”. The former paper presents evidence that shape seems to be an important visual variable

Of all visual variables, shape plays a special role in discriminating between symbols as it represents the primary basis on which we identify objects in the real world. In fact, theories of object recognition differ only to the extent that they consider object representations to be based only on shape or if other features are also involved [9], [78], [114]. For this reason, shape should be used as the primary visual variable for distinguishing between different semantic constructs.

Hierarchy and subsequently depth are other interesting visual variables:

Hierarchy is one of the most effective ways of organizing complexity for human comprehension as it allows systems to be represented at different levels of detail, with complexity manageable at each level [30]. This supports top down understanding, which has been shown to improve understanding of SE diagrams [94]. Simon [125] proposed hierarchy as a general architecture for structuring complex systems.

In fact, its sublte but depth is something I have explored adding to the visual syntax of statecharts. Adding drop shadows to hierarchical/parallel states and inset shadows to history states. The later is interesting since history states are deeply knowledable about the state they are in and inset highlights.

I would love to have something more like "zooming in to a different depth of hierarchy" that I could use as an alternative to collapsible trees, e.g. Haven't found any free, easy to use libraries for it, yet.

@Jason Morris Harel had some ideas about this in his original paper on statecharts. Havent seen it implemented in any tools though. Seems to only be possible by using containment to visually represent hierarchy.

That diagram is almost identical to one I wrote in my notebook last year while thinking about a visual annotation for procedures. So strange that there is no open implementation of it. Beyond my skill, unfortunately.

@Jason Morris @Paul Sonnentag posted about this today: https://twitter.com/paulsonnentag/status/1521111393613008901

Yeah, I showed someone the diagram, and they said "congratulations, you have reinvented hierarchical state charts." I'm not persuaded what I want to do is exactly the same thing, though. The state of a program and the state of a process are not identical. The state of a program describes all possibilities. I need to be able to note when a process has gone outside the lines, and figure out how to return it to compliance. But the visual interface would be similar. I need access to the interface, not state machine editing tools.

There was some research in the late 90s on combining zoomable UIs with level-of-detail rendering (from the Ben Bederson group at UMaryland - ). Would be useful for focusing in and out of different complexity levels in a software system (in particular, I've thought it would be a good approach as general UI paradigm for a Smalltalk system).