I know a lot of you guys are pro-visual programming. How do you reconcile the massive drop in human brain-to-computer bandwidth when moving from keyboard to mouse? Visual languages may be better at getting information from the computer to the brain but text is still orders of magnitude better at information flow from brain to computer. And it's not for a lack of trying; UX design is mostly a field about how to get ideas from human brains into computers using a mouse and/or phone touchscreen, and we're still miserably bad at it. What changes to make all the programmers stop using terminals and the accountants stop using Excel?

Visual programming primarily means using a richer visual representation of a program than a grid of fixed width ASCII characters. We have these powerful graphical displays: let's make better use of them. Better visuals do not mean you can't use the keyboard as an input device. As for the mouse, it depends on how you use it within the UI. My Curv language environment became much more productive for certain programming tasks when I added the ability to tweak numeric parameters using the mouse and get continuous visual feedback, updated at 60 FPS, showing the effect of the parameter change on the program output.

As well as the above "scalar" selection, you can still use the keyboard within a system that constrains choices - e.g. use autocomplete to save keystrokes; plus not allowing syntax errors to be chosen / typed

As mentioned before, nothing about visual programming dictates that it requires a mouse, or that mouse is the primary mode of interaction. You can easily design a visual programming language where mouse and keyboard usage is roughly equal (Unreal Blueprints), or where keyboard is the primary input. Some of us are working on visual editors for platforms were keyboard input is non-standard, slower (virtual keyboards) or non-existent. Smartphones, game consoles, virtual and augmented reality all have that constraint.

If I can click the spot I want to click directly with a mouse is this really low bandwidth information exchange or am I swapping my visual/spatial/motor system effort for my language effort? The brain processes much more than just language. And I choose that spot using some sort of internal language which is then translated in the position I chose. If the visual representation is more efficient in describing my intent, was it actually less information flow going from me to the computer when I created that visual representation? I still use a code editor but I spend about half of my effort in a low code environment where I just put attributes and conditions into forms.

In addition to the points above about keyboards not being precluded -- I'd actually be willing to trade off some inconvenience while writing (brain->computer) for a better experience when reading (computer->brain).

@Charlie a great question. I recall someone long ago, probably Tog, found that people think their faster with the keyboard when in fact they're faster with the mouse – at least for some tasks all the standard caveats apply. Surely, the mouse (or similar controllers) are terrible for text entry compared to keyboards... not that keyboards are necessarily the best... I think I just saw a stenographer wave. It's even disappointing seeing most apps on tablets using single touch over multitouch. As an adult who likely uses a phone (which is smaller and therefore favors single touch since you're holding it in the other hand) you will not see how wrong wrong this is unless you see a small child flip out about it. And to close up this steam of consciousness... spacial computing... AR and VR... is a real game changer, because established screen centric conventions suck in that medium.

Just thought I'd add the Tog articles mentioned here: https://www.asktog.com/TOI/toi06KeyboardVMouse1.html https://www.asktog.com/TOI/toi22KeyboardVMouse2.html

(plover stenoHi I'm the aspiring stenographer who wavedplover steno Though I will note that for programming in particular, heavy use of !@#$ ascii symbols (and kebab-cased camelCased etc identifiers) does privilege the keyboard somewhat. A steno interface for writing

function foo(a,b){ bar(4) }

would ideally be something like `FUN foo a b LINE bar CALL 4 FIN`; at which point it may be prudent to notice what you're doing and switch to writing ruby/TCL/somesuch directly, or perhaps a lisp with some autocomplete trickery for placing parens)

@fyr Ah, I thought you were here somewhere. 🙂

Interesting; I hadn't thought of a keyboard controlling visual representations. Total blind spot on my part--thanks for illuminating!

There’s certainly a drop in information bandwidth, as you’re no longer adding lines and lines of code to the program. But how does that map to bandwidth of meaning / intent? I’d argue that visual input approximates intent much better than textual, so the drop in information bandwidth may be superficial.

It all boils down to "tap vs stroke" at the end of the day..

For the sake of argument, I’d grant that building a program visually is slower than using a text-based language. Most problems I’m usually facing in programming occur when I need to change something. I think laying out and trying different variations visually can be much faster and can lead to better results than going through the same process with textual code. (Maintaining and modifying a mental model.) Then there’s the question about which is more important: brain-to-computer or computer-to-brain bandwidth? Here I agree with @Kartik Agaram. I think understanding quickly trumps writing quickly. And finally, as @Duncan Cragg pointed out, there’s no reason for keyboard not to have a role in visual programming. Either in navigating the code, or editing certain properties.

And text is actually both two dimensional and structured, so it's already visual

Else there would never be an argument about indentation. For programming, text does very well at the "paragraph" level, somewhat complicated expressions across a few lines, but for navigating a program, easy search seems to be its primary advantage. I mean if I open up a 500 line file, I have little idea what the overall structure. Honestly, node-edge graphs don't necessarily look much better. Semantic zoom friends, semantic zoom! These can be textual and often IDE have some sort of various summary views (going all the way back to the Smalltalk browser). These are often to the side though rather than being spacial.

I've been using semantic zoom for some time with traces generated by my programs: https://git.sr.ht/~akkartik/basic-whitebox-test (bottom section, but it may be helpful to read from the top). It takes only a few lines of code, and it works well with text. Add the ability to click from lines in the trace to the lines of code that generate them, and they give you a scalable way to navigate programs.

I agree that visual programming doesn't necesseraly imply mouse only. I have experimented developping a node and link language (à la PureData) which one can practice with keyboard editing and navigation only (with modes like in vi).