perhaps even, the more successful they are, the less they're considered worthy of being called programming at all (spreadsheets, scripting, patching)

Isn't this just that the term "programming" is a bit scary for most people, so out of pure marketing self-interest, developers go for other labels>

Yes I agree but think 'scary' isn't quite the word - people don't want to be called programmers because of all the cultural baggage that comes with it. On the other side computer scientists / programmers don't consider spreadsheets etc programming because it doesn't resonate with them culturally, even though it clearly is by any objective measure.

Really good software creates the illusion that you're working with something directly, rather than trying to use some about interface to work with something. In other words, the interface is invisible. So maybe when it's done really well, you don't realize you're programming, because (for example) you're "just" reviewing and updating your budget.

@Dan Cook I'd disagree very strongly that interfaces should be invisible, that their job is to get out of the way.. Bret Victor's classic is useful here http://worrydream.com/ABriefRantOnTheFutureOfInteractionDesign/ Also Julian Oliver on seamlessness https://theinfluencers.org/en/julian-oliver/video/1 You can't add a layer of abstraction to something in a way that still feels like you're working on the underlying thing. You can bring together the abstraction with the end result so that you can make a change in one and see a change in the other. If it's a creative interface the relationship between abstraction layers should be rich and even surprising. A great interface that you're familiar with should support a productive state of flow, but that isn't about invisibility, but a balance between boredom and anxiety.

I think the level of interactivity depends on the use-case, context and even the preference of the users: sometimes you want a race car with direct control, the other extreme is ordering a taxi or Uber. Many use-cases fall between these extremes. See also the “Interactivity considered harmful” section on the MagicInk post by Bret Victor: http://worrydream.com/MagicInk/

Abstraction is fundamentally about removing detail. When you abstract, you make a choice what to keep and what to get rid of. You decide to ignore some details that you deem unimportant and you highlight those that you think are important. That is what design is about. Maybe calling it “invisible interface” is too limiting or too extreme, although sometimes that might be a good design. However, always being able to jump between the layers of abstraction and to merely consider abstraction as transformation and not getting rid of anything really does seem extreme either, even though it might also be good design in certain cases. In our field I sense a strong aversion to giving up control (aka removing things). Our deep understanding of technology enables us to deal with a much higher amount of complexity than people untrained in our field, who might get overwhelmed much easier than we would. That gets in the way when we try to design stuff for them like interfaces or experiences. I think this also lies at the core of why we can’t leave things like human-readable data formats, text editors, or command line terminals behind — it just gives us so much control we’re not willing to give up. Meanwhile, regular people do just fine without those.

Right. What I notice when doing UI design is also, that some fairly innocent-sounding flexibility in data models can cause huge problems when building user interfaces. For example, if you allow many-to-many connections between entities in a data model, then you need to take those into account in the UI if the user tries to delete any of the objects: “Are you sure you want to delete X which is in use by Y, Z and 23 other objects?” Answering these questions might be hard, or impossible, for the user. So, to keep UI easy-to-use, often you have to limit the flexibility.

Abstraction isn't fundamentally about removing detail. Run-length encoding is an abstraction but doesn't remove any detail. If anything it adds detail.

I think the idea that images are a more advanced interface than text is pretty unique to computer user interface designers. text is made out of images.

I think this is an unfair caricature of UI designers. Images are more general than text, they include text as a subset. By not restricting yourself to text, you admit more possibilities, which are useful in some domains. Text only photoshop would be pretty hard to use: you need to see what you are working on. Node+wire visual languages like Pure Data (music), or Nuke (graphics) are popular with some artists, because this kind of interface removes the need to parse and generate the syntax for a textual language. It's not an artist vs developer thing, though. It's just that not everybody's brain is wired the same way, and text syntax is not everyone's preferred interface. Some people are primarily visual thinkers, for example.

I've worked with blind kids, and can assure you that you don't need to see what you are working on to use a text-only drawing program.

@yaxu

Abstraction isn't fundamentally about removing detail.

Apparently, we come at this with a different understanding of what abstraction means. As you reject mine, would you mind explaining what abstraction means for you? You listed examples, but I need help to come to a generalization that would differ significantly from mine.

"Our deep understanding of technology enables us to deal with a much higher amount of complexity than people untrained in our field, who might get overwhelmed much easier than we would" that's the sound of my jaw hitting the floor

I was contemplating for a moment if "untrained" is enough to make clear that this is not about intellectual capacity but about time spent in a domain. I'm sorry if that came across as derogatory and I should've taken the time to make that clearer. I was thinking of people who are professionals in a different domain than ours yet still proficiently use computers, but without necessarily knowing how they work — not because they're not capable of understanding, but simply because they chose to spend their time differently because they have more important stuff to do. I have lots of those in my circle of friends and often think about how I could create better software for them. Is that what caused you to make the connection to privilege, or am I missing a totally different dimension here?

Do we agree that digital computing is an abstraction on top of analogue electronics? I suppose that does involve removing details, every continuous spike is slightly different, but is treated exactly the same.. A binary 0 or 1 being a generality of any continuous movement between high and low. So perhaps I'm splitting hairs, but I find it difficult to think of digital computing being an removal of detail from analogue electronics. I suppose fundamentally it actually is, but in practice, digital computation is an entirely different world from the analogue circuitry that supports it. Likewise, the idea of a symbol 'L' is a kind of removal of detail in that all Ls can be treated the same no matter what font or handwriting it is notated in. But the abstract notion of an L has nothing to do with a right angle.

@yaxu Not literally invisible, but conceptually. That is, what you're doing isn't trying to get some tool to do what you want, but the thing you want to do with it. But yes, tools and interfaces can user powerful ways of thinking and working and help conceptualize, etc. But a good interface is not so obsessed with itself that it gets in the way -- a good one is obsessed with creating a good experience for whatever you need to do with it.

@yaxu good points about symbols, syntax, and visual programming. For a while, I have been exploring what a programming language without words would be like. Start with shape, orientation, transformation, and see where it takes you.

@yaxu It's a strange myth that graph-based languages like pure data aren't text based.

puredata has no visual syntax - visual layout has no syntactical meaning.

That absence of visual syntax is a feature, it's the main point. The people who prefer these languages don't want to deal with syntax. Pure Data leans rather heavily on embedding a text based language inside those boxes, so it's not a good example. Less so for some other node based languages. @tbabb is building a node-and-wire language that eliminates text-based syntax altogether, although you can still use text as labels. https://lynxtool.com/What.html

@Doug Moen I agree that the beauty of pd is practically error-free syntax (imported from modular synthesis), I just think it's interesting that what we call a visual language is less visual than many textual ones. Of course the lack of visual syntax means that it's left to what Thomas Green calls 'secondary notation', people can lay things out in a way meaningful to them. There's a huge opportunity here though, to make truly visual languages which brings this secondary notation into primary syntax and semantics. The ReacTable is a great example of a front-end for pure data, where proximity gives both connectivity and value. But I'd still say it's a kind of symbolic text! But augmented with gesture, in the same way that speech is.

@yaxu yes we have gone to a funny place if a "language" is not some sort of symbol system. Usually a "symbol" is an arbitrary signifier, a string of characters in typical text-based programming language. Arbitrariness isn't a all-nothing thing. For example, a color swatch is far from an arbitrary way to represent a color, but it's still not the same as seeing your kitchen painted in it. Even a hex color

#7521AF

isn't an arbitrary symbol, it's part of a system. The

#

at the beginning is about as arbitrary as it gets though.

I agree that all programming languages are languages, and that all languages have syntax, in a formal sense. But that insight obscures an important distinction. It's the language centre of our brain that processes syntax (and music: https://www.sciencedaily.com/releases/2007/09/070927121101.htm). There are other modes of thinking, outside of language, such as visual and kinesthetic thinking, which engage other parts of the brain. We make the experience of programming accessible to more people when we build alternative programming interfaces that de-emphasize syntax and parsing, and emphasize other modes of thinking, such as visual and kinesthetic.

@yaxu Lynx's UI essentially causes direct changes to the abstract syntax tree data structures, which the compiler then directly converts to machine code using LLVM. There is no parser/lexer that's part of the language.

@tbabb Sorry I don't understand, that just seems like an implementation detail?

You were wondering how Lynx gets away from text— the answer is that it never generates it. I guessed you were picturing something like what lots of other systems do, and that is to use a UI as a wrapper on a textual language, and then parse/lex/compile the text language "normally"; where the text is the "real" representation. It's not always obvious to everyone that you can cut that step out; people often assume a programming language has to be text at some point because that's usually the only way we ever encounter it. But Lynx is no more text than a JPEG or a photoshop document is.

I do agree that computer languages could make far greater use of imagery (whether visual, sonic or kinaesthetic imagery).

For the user, objects refer to each other by connectivity. If you call a function, there is a wire from the function definition to the call site. Internally, the AST is a data structure, so its sub-parts refer to each other directly (i.e. pointers). The user doesn't have to invent a sequence of characters for anything in order to use it. (Though it is possible and helpful to give things labels)

Ok so I have a function and some input and output. How does the function have any identity without some kind of symbol?

Fundamentally what happens is that a wire is drawn from the definition of the function "reverse" to the call site, which also has a wire feeding the data to be reversed. For convenience, there is a menu which allows you to choose the function to place and connect to from a menu, which displays the labels and documentation for all the available definitions. But when you use it, it's really doing the above. In principle you could give no names to anything, or everything the same name, and it would still work just fine— but of course that's not a good idea because it'd be confusing! :)

Sorry still don't understand, and can't find the link to it now for some reason.

Internally, the data structure representing the call site has a pointer which refers to the function definition. The pointer is assigned as a result of the UI actions of the user

ok the first example, you select "+". To me, "+" is a discrete symbol - text.

Sure; yep, there is text on the screen. That's the label for the function which does addition, which the UI will then place with a (hidden) connection to. In this case, the example is complicated by the fact that the UI hides the wires for simple operations. We don't a zillion wires crisscrossing the canvas connecting every "+" to its global definition— but those can be exposed if need be.

It still seems to me that the text part is essential to a computer language.

Here— this is a mockup for a client which shows the connectivity between a definition and a call site fully exposed:

That said I do agree that augmenting text with graphs is a great way to go

The top square shows the definition for a function (labeled "my financial model"). The definition is carried out the bottom through the white wires, to the several call sites, where it is applied.

It's definitely a trade off. For example it's a lot easier to write a text program about text than a graph about graphs

Neat! Yes, node-and-wire is extremely playful and reactive. It's my hope to bring that playfulness and responsiveness to programming in general, and to more people.

That's great. My argument is really that text isn't bad, after all literacy rates are high these days. It's just that text is often better in combination with gesture. No-one wants to listen to a speech synthesiser that doesn't have prosody, for example.

Connectivity by proximity is an interesting idea. I haven't seen that in professional node tools, but it might be handy. FWIW, I am aiming to release the user from the burden of laying things out. This is not implemented yet, but I want the system to automatically layout and tidy up nodes so the user doesn't have to fuss with them. That can be a big slowdown when using pro node tools. (They should still be able to tweak the layout, though)

Heh careful, you might find the best layout is horizontal and vertical, and get back in the constraints of 'standard' text :)

@yaxu I can see where our understanding of abstraction diverges. Most of the examples you gave, I would call translations or transformations, because you jump between domains. I agree that transformations don’t necessarily remove or add details. My understanding of transformation is that it’s merely a change of form or appearance. My understanding of abstraction is that it is a more specific form of translation, with the added constraints of what I described as “removing detail” and also that you stay within the same domain. You abstract because you generalize more specific concepts by ignoring some of their details and lift the commonalities you want to highlight into a concept that’s higher up the taxonomy, still in the same domain. You do that so you can remove complexity and more efficiently think or communicate with the now abstracted, simpler concept, no longer having to deal with all the detail removed. For me that’s the whole point of abstracting — to reduce cognitive burden, to simplify. In my taxonomy of definitions, confusingly abstraction is therefore a more specific concept than transformation adding more specific requirements to what it describes. That’s how I think about this. Does that make any sense?

So run-length encoding isn't an abstraction because you can go in both directions? So an algorithm of how to make a thing isn't more abstract than the thing?

Yes, intuition… ever since you rejected my initial comment, I was questioning my own intuitive understanding and have since been reading the Wikipedia pages for transformation and abstraction (computer science), which leads to generalization, which introduces the domain constraint. And so I thought I share what I learned and refined the description of my understanding based on that. I'm not saying I've figured it all out and my interpretation is correct and you're wrong; who knows. It's kind of… abstract. 😉 Language is hard. What I wrote makes sense to me, seems to be aligned with the definitions in a reasonably authoritative source, and so I hope it has a chance to align with other people's understanding, and I feel comfort in that I'm not just making stuff up or rely on my intuition. I'm still just interpreting something I likely do not fully understand. But we have to agree on something, some kind of starting point, to have a meaningful discussion, and I think we didn't have that, and probably still don't have. And that's fine, as I'm not debating because I want to change your mind or I need to "be right". It was a good prompt for me to check my own understanding of these concepts and I coincidentally happen to write something currently where this exercise was just super useful and important and I want to be precise about this. Thanks for your comments that encouraged me to put in effort that I wouldn't have otherwise. Perhaps it'll make my work a little better.

Yes I'm a firm believer in cognitive semantics, where we communicate in order to get our personal systems of meaning closer to reach other. Turing equivalence poses a problem for your definition. Maybe we agree a DSL implemented in C is more abstract than its host. It's less general, and in some way embodies a model for a domain. I'd argue that whether or not the DSL is Turing equivalent is really incidental to usability/design issues around affordance. This implies that abstraction can be circular, as C could in theory be implemented in the DSL. How do you feel about this?

Here's my take. When you learn to count, you are learning the natural numbers. "Two" is an abstraction which represents the abstract concept of a pair of objects. There are infinitely many pairs of objects, but there is only one Two.

@yaxu At this point I'm more curious about your style of reasoning. As far as I can see, you haven't attempted to explain what abstraction means to you in abstract terms. Instead you always turn to examples and have posted quite a few of them here. Nothing wrong with that, in fact they would all be super helpful to illustrate an abstract description. By arguing like that, somewhat ironically, you leave out the crucial step of generalizing what the concept means to you and leave it to us to interpret your examples to find the commonalities ourselves. I've tried, and I am unfortunately not able to come to the conclusions you do using your examples. I tried to acknowledge that in my explanation of how I interpret abstraction by offering transformation as a more generic concept that I believe fits all your examples and also my understanding of abstraction itself, trying to create some common ground. Now on the other hand, I so far have only offered my abstract explanation and no specific examples. So let me try that and meet you where you are: Take an

apple

and a

banana

. In our (hopefully shared) English folk taxonomy, we would classify both as

fruit

. We're generalizing, ignoring some of the properties `apple`s and `banana`s have. Or take

green

and

yellow

. Both `color`s. Same process. That's what I call

abstraction

. Based on your examples I feel like you come into this claiming "yellow is an abstraction of banana". You jump between domains. And that's interesting. I don't really understand it. Also, it's not obviously wrong! It's somewhat clever because, yes, bananas are usually yellow, and just looking at the color of bananas feels very similar to the process I described above. Still I come to the conclusion to reject it as a form of

abstraction

and point out the requirement to stay within the same domain. Here's why: When we agree that an

apple

is a

fruit

, we gain a powerful tool as part of our language that allows us to communicate more efficiently. It's not just introducing a new name for something. It's more than that. We can jump between the levels of abstraction as we see fit and say "`fruit`s are healthy" and don't have to list all the individual fruits. At the same time we can walk into a supermarket and say "an

apple

, please" and get exactly the fruit we want. (From my understanding of Lakoff, there are kinesthetic image schemas container and whole-part at play here, and we use metaphorical structure to add additional meaning to the relationship between

apple

and

fruit

.) What do we get if we accept the same relation between

banana

and

yellow

? I can't see the same benefits there and to me it's mostly confusing, even though I can see why it appears similar. That's why I'd like to keep

abstraction

for the concept with these specific benefits, and leave

transformation

or

mapping

for the concept without. It feels like we have good words for all these things already, and watering down

abstraction

only makes it harder for us to communicate.

I'm not trying to offer an alternative, I'm trying to understand your position by asking you to respond to examples

Based on your examples I feel like you come into this claiming "yellow is an abstraction of banana".

"Yellow" is an abstraction of all yellow things, which would include ripe bananas, just as "fruit" is an abstraction of all fruit things.

Scott McCloud’s drawings and thoughts on abstractions in comics might be interesting here. The thread is so long that I’m not quite sure if anyone linked to these yet. Sorry if these were already mentioned! https://comicbookglossary.wordpress.com/realistic-vs-iconic-representation/ https://medium.com/@pattyjburns/understanding-comics-by-scott-mccloud-quick-synopsis-bcc2fa260075 I feel that these are related to signifiers in semiotics and categorizations by Peirce, I don’t know if the connection is a direct one: https://vanseodesign.com/web-design/icon-index-symbol/

"Yellow" is an abstraction of all yellow things, which would include ripe bananas, just as "fruit" is an abstraction of all fruit things.

Is it that simple? I don’t think the category “yellow” is structured the same way the category “fruits” is (Lakoff and other linguists don’t seem to either). There is indeed lots of research about colors as categories in language, and studies in that area are used as empiric evidence that mental categories are not objective, transcendental, and definable without taking into account the perceiving being. See George Lakoff, Women, Fire, and Dangerous Things for reference. Independent from how we structure our mental categories, let me try to address the domain requirement this way: If we want to ensure mutual understanding, it isn’t sufficient to just define a word, but also the domain we’re operating in. I can totally see a category “yellow” that means exactly what @Doug Moen defines it as. It’s just not the same category as “yellow” we more intuitively turn to as part of our folk taxonomy of colors.

Hm, I haven't looked into cognitive linguistics for some years, but didn't think Lakoff provided much empirical evidence for his claims.

No question, I find the title Lakoff chose for this book very unfortunate as well. The names you list are all over referenced in the book, I don't get any sense that this is not a thoroughly researched work backed by countless references. He wrote it to be comprehensive, which doesn't necessarily contribute to the accessibility of the work. As they say: all models are wrong, but some are useful. I find Experientialism extremely useful. Integrating the experiencing individual into the model of reasoning seems like a pretty good idea to me and a good step towards a more humane foundation we want to build our world on. Am I taking sides? Perhaps. Does that mean I'm wrong? Perhaps. Maybe I can come up with something useful anyway.

Yes I agree with that. I should give the book another go.