Retrofitting of Medium I tend to think every time a new medium is introduced, all the contents of the previous medium are ported over, without regard for fully understanding the new medium. For instance, films matured when it stopped porting over books, games matured when it stopped porting over movies, etc. An excellent example of this is Outer Wilds, a space-exploration time-traveling game which is simply impossible to recreate in any other medium. It fulfills the medium. Given that, I think two new forms of medium are being introduced in the next 5 years: LLMs and VR. If we accept the supposition that there will be a lot of retrofitting of the medium, i.e. chat into LLMs, games into VR, what are the content that will truly fulfill the mediums of LLMs and VR?

I think there is an interesting distinction to be teased out in your question between a few things • a new UX • what that new UX is used to do With your example of LLMs and PDFs — is there something distinctly LLM-ish about that UX solution? Could that same UX be achieved without an LLM?

I think one can potentially do the tree of concepts idea with an extensive knowledge graph. But at least feasibility-wise/commercially it's difficult. You need a flexible KG usually, especially if you want to do some link prediction based on the PDF at hand. Re: Nielsen piece, fully agreed. I think it's an interesting new medium.

I think another way to approach your question is to explore something like vocabulary or that moment when a new medium establishes its own norms for communicating through itself. So, for film, that would be building up a toolkit of specifically filmic story telling techniques that are born from film, rather than another medium (but may then find their way into other mediums) With video games, these techniques are often game mechanics — but also can overlap with animation, etc. By attributing “maturity” to this sort of technique, I think you can start to decouple the medium and the message a little.

That rings true. Words have power, a la call a demon's name to dispel it, express the psychological problem to solve it, i.e. Focusing. (Perhaps vocabulary doesn't have to be linguistic, but linguistic vocabulary has the highest virality) A tangent, but then what's the best medium for crafting a vocabulary around a new medium? Like could we build a meta-tool to help people understand the latent structure of another tool?

oh! I dig that question so hard! well, my academic background is in theory and art history — there, new vocabulary comes in through a few different processes, but, the easiest to trace is often borrowing, e.g. there exists a body of work that talks about painting, BOOM! Photography explodes into the art scene, so, at first, folks write about photography by borrowing terms from painting, often times focusing on the differences and similarities between the 2 mediums…over time, those terms transform, new terms/vocabulary are invented to begin to wrestle with that new stuff photography is bringing to the plate

That inspires a lot of thoughts. It reads to me like there is an isomorphism with scientists trying to explain a new phenomenon with existing theories, e.g. Newtonian mechanics not being enough to explain phenomena near light speed. I understand the "new stuff" you mention as different affordances/spaces/possibliites that must be wrestled with. I guess new vocabulary is created in the chemical reaction/wrestling/resistance between the borrowed terms and the new phenomena. So perhaps operationally, 1. Don't settle for the borrowed terms. Keep looking for new phenomena and "react" with borrowed terms. 2. Borrow not from the direct parent of the new medium, but somewhere else. Maybe vocabulary in games (e.g. juice) is useful for understanding LLMs. Determination of which arbitrary medium to borrow from would also be interesting. I am also thinking, which field has the most experience dealing with this phenomenon of creating new vocabulary? They would have the most meta-models regarding new vocabulary. The top candidate seems to be art, right? The ultimate medium of mediums. Tools for thought can also be rendered as tools for expression.

you know what we’ve just described here?

should i pop his book to the top of my reading list

the book is interesting, but I think the big points to take away can be pretty quickly surmised (I think @Jimmy Miller did it on a recent episode) as something like: • there is an existing understanding of a thing, with an existing model • someone pokes a hole in that understanding that throws it into question (triggering a crisis) • to deal with the crisis a new model is brought into being (paradigm shift) • that new thing becomes the next model until another crisis comes up

I feel like this is an on-paper thing that is really different when it's real and in your face.

arguably, Isaac Asimov’s Foundation series is about this, too, but a lot longer to read than Kuhn’s book 😆

I mean different as in it's hard for people to get

But also, there must be something different when applying Kuhnian paradigm shift to the shift of mediums, right? (in a truly meta sense)

I am not sure!

I am wondering, in a similar history of science way, whether there is a way to incorporate Feyerabend into this discussion?

sounds like I’m gonna have to read some more books! 😆

same here

math as a medium for art! — looking at you @Jack Rusher

The big change is that computers can be communicated with in a similar fashion to the way we communicate with each other so words contain more instructional information for a computer than before. So the UI is text right? Or even with current speech recognition the obvious UI is answer and response. That’s the most efficient way we communicate, but what we say rarely is what’s received but our internal error checking irons out most misunderstandings quickly. Also if the person you’re speaking to gives a puzzled expression that’s a huge amount of information for you and you’ll reformat your speech. LLMs have none of those abilities which makes them pretty inefficient when they misunderstand. LLMs are a flawed model of human communication. I hope we don’t bend ourselves out of shape to compensate. But the medium is not new IMHO.

But the medium is not new IMHO.

Big agree

I have to inquire about your definition of medium. In addition, I am curious about the effects of the definition, as in the utility of it. I am getting the sense that LLMs and VR are viewed as flawed versions of mediums that already exist in reality, thus not new medium. That feels like retrofitting LLM and VR to reality. Surely they look similar to reality rn (or try to), but they will mature. A weak example is electronic music, which developed into a field of its own. Or texting, which is different from phone calls, which is different from physical comversation. Messaging apps nowadays have reactions, which allow you to juggle talking about multiple things at the same time. All of this discussion (personally) is for developing interesting pieces of technology within VR, or through LLMs. I would imagine thinking of them as flawed rendition of reality would not help in furthering what can be done. I think we can go further than imitating human communication (which already is cool).

If I was to be rigorous about investigating it I’d make clear boundaries between the examples. Instructing a computer is tool using whereas and music (electronic or otherwise) is art. Computers are tools. Using that tool to make music does not change the nature of music. I don’t think theories of art can illuminate the effect of a new tool.

Using that tool to make music does not change the nature of music. I don’t think theories of art can illuminate the effect of a new tool.

@greg kavanagh, if you have a moment, would you mind elaborating on this? Do you mean that tools used to make art don’t change that art, e.g. the same art can be accomplished with one tool as another? From my perch, I think that tools and the art that they help to enable are very much linked, inextricably so, often times. I do think that theories of art can illuminate the effects of a new tool — I think much of art history is a project in understanding exactly what the ramifications of a new tool may be.

Hi. No that’s not my point. Obviously the invention of the camera changed visual art. My point is that LLMs are a method of instructing a computer. did a digital display on a camera change cameras? The conversation seems at points to equate LLMs with the invention of the camera. I think it’s important to clarify the nature of things as the metaphors can become mixed in ways that only LLMs can :) Regarding tools and music.. if I play a Bach piece on a piano, a glockenspiel or a synth what has happened to the essence of the fugue. And if I play it on a sampler that has sampled the original well tempered clavier Bach played it on to a degree of fidelity that is indistinguishable from the original has the tool changed the music? If I manipulate it then with the sampler am I changing music or am I composing? Has music changed? I was merely drawing attention to where those lines might be as I think it’s important to the discussion.