New visual programming languages don't fail as vis...
# thinking-together
j
New visual programming languages don't fail as visual programming languages. They fail as programming languages, which is what most new programming languages do.
i
I agree. But I also think they fail don't fulfil their potential in other regards. • They fail at visual communication. Some visualize data types (common in block-based). Some visualize complex control flow (common in node-wire). Some visualize runtime state (common in state charts). Few give the programmer any choice in the matter. None (that I've seen) let the programmer build their own visual affordances in the editing environment. None meaningfully use animation. • They fail at interactive communication. Some are structural, preventing invalid program configurations (blocks). Some use interaction as part of the programming process (PbD/PbE). Some use tight cycles that approach the sensation of dialectic (fast TDD). None give you the flexibility to define what interaction ought to mean as part of the activity of programming (though some GPT experiments are verging on this). None meaningfully use theatre. It's easier to make a mediocre textual programming language than it is to make a mediocre visual one. I imagine the same would be true if one wanted to make a good visual programming tool — but I don't think anyone has done that yet, so I can't say for certain :)
j
Your reply just planted the idea that we might need to figure out consciousness to figure out how to create a truly good visual programming language. Somehow our brains are able to blend visual and symbolic thinking into one coherent model, but how that's pulled off remains a mystery.
i
I think it's easy to come up with a long list of concrete things programming tools (visual or not) could do better. It's just prohibitively effortful to do the design, implementation, etc. work needed to realize those ideas.
So while you might be right in the limit, let's not preclude the possibility of making progress in the meantime.
j
I want a spoken language based on the potent succinctness of an APL
j
Also consider the possibility that human beings evolved in the environment of text, not the other way around.
j
Analogy is potent, it taps an experienced knowledge to apply it to another context. I think the real future visual programming language that takes root is potentially far more flexible in its initial form and gradually becomes abstract as a user expresses fluency with the underlying principles. I like the mental exercise of imagining how i would attempt to introduce someone to programming from a raw visual representation of sections of memory flowing through various processor types. Re-introducing assembly and the abstraction of higher level languages on top of that.
c
@Ivan Reese There was something quite profound to me in the way you worded this! Thank you 🙏
None meaningfully use animation.
I was wondering if you could expand on this?
i
If you have a programming environment that makes effective use of graphical imagery, it's easy to imagine communicating even more information by making those graphics move somehow. Exactly how depends on what is being visualized. Let's assume a graphical program editor that is able to access the stack and state of the running program. It could show data flowing through functions. Or functions moving across a visualized state space updating values as they go. It could overlay the program source with a heat map representing time spent in each area, updating on the fly as the program ran. Even without access to the running program, an editor could use animation to convey information. We already use color (syntax highlighting, diff, themes) — and that's arguably a narrower channel.
c
@Ivan Reese Totally agree with what you’re saying! Besides being an new channel to use, it helps slow down the computer’s imperceivable execution and provides object permanence when morphing between representations. I’d also love to see more VPLs exploring physics engines. One of my favorite parts of @Samuel Timbó’s Unit is the forced directed layout. It adds a liveliness that I dont see with many other VPLs.
k
@Ivan Reese I'm thinking something similar, I'd like to see a 4D programming environment where the activation of the program and its state is visualised in real time. And it directly maps to the 1.5D textual representation.
The best way I would describe it is like this. Let's say that reality is like a state computation machine. How would it be possible to create a more accurate model of reality than we have currently? It would have to go beyond words, and would have to include visual, kinaesthetic constructs. It would have to be a higher dimensional representation, to the extent that we as humans can perceive a higher dimensional representation. So this means, higher than the 1D text we have, or its 2D UI, single frame in time representation. This would look like some kind of 4D temporal model, where it's possible to see the execution of a program in its entirety, all at once. It would not only show semantic relationships in a graph sense, but also model the data structures and their operation over time. So the idea of "stocks" of data, and "flows" of data (from Donella Meadows, Systems Thinking) would apply here. React.useState might be represented as a small orb of energy, and if we setState, then it would visually change. When we assign const a = b, then it would visually indicate a flow of information from b -> a. I think this would be a much clearer representation of what a program is doing, and free up cognitive load to focus on more important tasks, as opposed to the programmer having to hold a mental model in their head. No matter how good a programmer is, if he or she possesses a better more accurate kinesthetic model of what is happening, their brain power is freed up to solve more problematic and relevant abstractions. I've been mocking up a prototype of how this would work, and the basic idea is overlaying a tool like VSCode with "vibes" as well as flow relations. It's like an annotation layer over existing code, but this is just the basic start of how it would look like. Ultimately, it would look like a mini universe. www.eusaybia.com for a non-programming context demonstration of what I'm talking about.
d
@Kongwei Ying your followup to that post went to the channel: https://futureofcoding.slack.com/archives/C5T9GPWFL/p1710504901802799
There are some of us that see the whole "computers and programming" thing as being about simulating or modelling reality. In other words, the "computer" that computes mathematics stuff or the "office computer" that just does paperwork are really just thin slices of a bigger conception of what we're all doing here. As is the "gaming computer" that is literally modelling realities in 3D. And on to "Spatial Computing"... A good programming language has to allow us to express our mental model of a live simulation/model into the machine for it to manifest that, for us to check against the mental model and make adjustments. So it has to be able to represent human cognitive elements for building that, for converting the cognitive model into an external live one. So to be humane it must not have any elements that are machine-oriented, like disk, memory, network, CPU/thread, etc. It has to have elements that are human-oriented, like property, identity, sequence, collection, evolution, adjacency, colour, size, words, sentences, sounds, animation, behaviour, ... (see Roget's Thesaurus for inspiration there) https://www.gutenberg.org/cache/epub/10681/pg10681-images.html
k
I think the best model to express computation would be something that resembles the human being, at all levels, from the spiritual, to the mental, emotional and finally physical. In our society and especially in technology there's a tendency to over emphasis the mental and physical and no where more is this prevalent than in STEM fields where everything is over abstracted into rigid, linear concepts. I also think that a good programming language would have very little to do with low level concepts but instead, would model aspects of being human. The problem I have with the Thesaurus you linked is that it's still overly conceptual. It's just a bunch of words, and while I get what he's hinting at, I find this frame of reference to be inadequate. Instead, I think that certain spiritual schools of thought have a much better vocabulary with which to explore this area. In esoteric schools of spiritual thought, the human being is thought to have an underlying blueprint which describes its true nature. So in Daoist sects, there is something called Jingshen, which roughly translates to spirit. In some Hindu schools, there are models of this energetic blueprint that model the human not as atoms or quantum particles but as a conglomeration of energy that's distributed around the body and stored in chakras. While I'm not saying you should believe this, I'm merely saying that this model of the human being, is something that would be good to emulate in a programming language. Something that can model the human being at this level would be far more powerful and expressive than the logos based languages that we have today.
So basically the idea is, humans have a spirit and this spirit is the blueprint for our lives, it's like a virtual container which simulates every aspect of our being at a mental, emotional and physical level. If we want to make a super powerful programming language and computation machine, it would resemble this spirit and be able to define everything that our spirit does.
The optimal programming language would be able to create an AI version of a human, down to all its smallest details. Anything less is just really primitive.
d
I'd like to refer followers of this thread to this new related thread: https://futureofcoding.slack.com/archives/C5U3SEW6A/p1710572601034079