"Visual Programming Languages and the Empirical Evidence For and Against" (Whitley, 1996)

The past decade has witnessed the emergence of an active visual programming research community. Yet there has also been a noteworthy shortage of empirical the resulting research. This paper summarizes empirical data relevant to visual programming languages both to show the current empirical status and to act as a call to arms for further empirical work.

https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.31.8423&rep=rep1&type=pdf

First I want to say thanks for posting this. And definitely want to encourage you to post any papers you find interesting over at #C037X8XMFB3. Really would love for that channel to become a wealth of papers on these topics. I haven't read the paper. Just skimmed it. But sadly it has reinforced my biases. I really want to be charitable to empirical research in CS. But they make it so hard.

Visuals can simply be poorly designed or be inappropriate for a task. Therefore, visuals do not

always provide a benefit and can sometimes encumber the programming process

Of course! No one thought otherwise even before a study was done. Even the argument for is bad.

An underlying premise of visual programming research is that visuals will improve the human-

machine interface by catering to human cognitive abilities.

Or that visuals are a fun way of doing things. Or that visuals might be actually harder, but less scary. Or that visuals do not come with the cultural baggage associated with math. Or that visuals appeal to us aesthetically. Or that visuals are a distinct way of communicating that some people prefer. I could go on and on. There seems to be this obsession with efficiency or reducing errors, or speed of learning or various metrics like that when people do studies in CS. But is that the goal? Why should it be the goal? Poetry is not the most efficient manner to express propositions. But no one is doing a study to see if we should write poetry instead of just writing prose.

Moreover, within the current context of programming, in which software engineering is struggling

to move from being a craft to being an actual engineering discipline, scientific method beckons.

I know I am turning up the snark here. But I just can't help it here. I'm going to ignore the fact that is debatable that engineering follows science or if it is really science that comes along after to explain engineering. What I instead want to focus on is just the rhetorical effect of this sentence. The elevation of engineering over craft. The idea that unless empirical methods are being used the work is not "serious". The whole thing just bothers me. To try and bring it back to less snark. I just find this whole discourse of empirical methods confusing. So many times they seem to be investigating premises that are not ones anyone with a sophisticated view holds to. Even ignoring the methodology of the studies, the questions being asked seem so misplaced. Or a guess more charitably, to only want to investigate a particular lense, usually around efficiency or comprehension. I just want to step back and ask, assuming a perfect study with zero methodological issues. What should we do with this information? If visual programming was found to be worse than text based programming in terms of efficiency. What does that mean for us? Should we abandon it? Should we not use it? I don't see how any of that follows unless you assume efficiency is the only important measure.

Ultimately, any software engineering technique or

theory of programming is judged by whether it produces cost-effective results.

I guess that is the case for this author. Or maybe they are making an empirical statement? It is just true that people will judge it based on that? Or that people should judge it based on that? Either way, maybe that's what we need to change.

Oh cool, didnt know that was a channel!

@Jimmy Miller thank you for pointing at the obsession with efficiency! I think it’s important for much of the work in this community. Worth adding too that it’s usually ‘efficiency’ in a very narrow sense, often meaning economic efficiency and worker productivity. As if a programmers measurable addition to profit is the be-all-end-all of what makes computing interesting. IMHO usage of a term like efficiency has deep ideological implications, because the meaning of these terms are in historical and current conflict. Our usage of efficiency today takes a lot of meaning from the turn of the 20th century when it was used to imply less economic waste in factory production and cost-effective worker management (less “waste” of human labour). Many other terms are highly political/ideological/philosophical of course, but this is often left unsaid, even in an academic context.