Naturalizing relevance realization: why agency and cognition are fundamentally not computational This is a psychology paper, so perhaps not what you usually consume, but I want to encourage you to give this a go. I think it’s super useful to leave our bubble from time to time and learn about what other disciplines’ perspectives on computation are. I’ll quote the summary at the end, but there is so much more building up to that that you don’t want to miss, certainly not before commenting on it. :)

The view that intelligence equals some kind of computational optimization is no longer tenable. It does not help us make sense of a large world. Therefore, claims that the study of intelligence is converging onto computational rationality as its ultimate foundation are not only premature, but outright misguided. Quite the opposite: we have shown here that the basic foundation of natural agency and cognition, and therefore of anything we could reasonably call “intelligence,” cannot be computational at all because it cannot be completely formalized. The dream of generating purely algorithmic systems able to think and act like human beings is and remains a pipe dream, because purely symbolic machines exist in small worlds, in which there is no problem of relevance to be solved.

👎🏻 This is like the dual of AI people who want to reduce everything to oversimplified computational models because they don’t know enough biology to realize how stupid that is.

@Jack Rusher If one was so misguided to be motivated enough wanting to teach those AI people just enough biology to realize their stupidity, what would one do?

I believe that computational thinking is a corner that we've painted ourselves into and is causing a great deal of self-flagellation and accidental complexity. I haven't finished reading the paper, but, it seems that this paper, as far as I've read, supports my belief. Does the paper go too far in the opposite direction by proposing new corner-y solutions in lieu of computational thinking?

@guitarvydas Oh, I think you’ll be very interested in finishing reading it, because I guess you will quickly make a connection between the adaptive dynamic process they suggest lies behind relevance realization (which makes it non-computational or rather non-formalizable) and independent parallel processing, which you seem to be interested in judging from several other posts here. Although I’d be particularly careful about that mapping, as of course our understanding of parallelism and concurrency is heavily colored by computational models.

@Stefan if I knew a short path to that goal, I’d have already written that essay 😆 One the other side, “computational” does not mean deterministic/formalizable — i.e. the fact that something is stochastic or heuristic doesn’t prevent it from being modeled via computation. Church/Kleene/Turing stands, though it’s clear that working strictly from a function call perspective is not always the right practical approach (which @guitarvydas has recently been trying to demonstrate).

@Jack Rusher I think you should read it. I’d love to hear your thoughts on that taking into account what they write about exactly this. I think they are very thorough in their definition and use of these words. The paragraph I chose to quote I did for grabbing attention. That’s what social media taught me to do. I realize that in your case that had the opposite effect, and I regret the error. From what I know about you based on your contributions here, I don’t think you’d let your assessment from above stand after reading it. Or you would at least have a profound insight about why you’d still think so. On the other hand I also totally understand if you don’t have time for reading dense 20 pages about something that may only be tangentially interesting to you. Btw, I think a concise answer to the question I asked you would be, “Write exactly that paper.”

I see a lot of miscommunication in this space due to people applying specific interpretations "computational" without checking that others' interpretation is sufficiently close. This paper (which I read a while ago, so I don't remember the details) uses "computation" in the Church-Turing sense, and under that interpretation I agree with the authors. And there are indeed people who make weird claims about life being "just computation" based on the same interpretation, so the criticism is somewhat justified. But then, there are also people with a wider view of computation. I used to be one of them, but I have moved to using less loaded terms (e.g. "information processing") just in order to avoid misunderstandings.

@Konrad Hinsen Can you say more about what exactly this misunderstanding is about? What makes you yearn for a different and less loaded term for computation or a wider view on it?

I don't think it's a minor point because computation in the Church-Turing sense is by definition/construction the action of a machine. Brains do information processing, but it's neither fully symbolic, nor fully deterministic, and in particular not decoupled from an organism's agency which is related to it pursuing goals shaped by evolution. Can you still call that "computation"? Some people say yes, others say no. And then there are those, against which the authors argue, who deny there's a difference.

I either read this paper as a preprint last year, or read a nearly identical set of arguments from the same authors. It is, in my opinion, a load of wooly metaphysics that doesn’t advance anyone’s understanding of anything. I put it in the same drawer as Wolfram’s philosophical stuff. Unfortunately, doing a proper job of explaining the ways in which it is not even wrong is a non-trivial undertaking (see https://en.wikipedia.org/wiki/Brandolini%27s_law). My best attempt at extremely lossy compression: I am sympathetic to — and frequently make — related arguments with regards to “AI” claims for the current generation of machine learning systems. They are not agents, and non-agentic things are not intelligent in anything like the definition we use for biological entities. Any attempt to make something actually intelligent would start from drives (something more like what we call emotions) than from simulating what many regard as intelligent activity (“write an essay based on a prompt”, &c). Failure to understand this is part of a long running tendency of clever humans who have been rewarded for “being smart” to over index on tasks for which they were rewarded rather than tasks that allowed 4 billion years of their ancestors to survive. On this general point I think I am in complete agreement with the authors. But this doesn’t actually argue against the idea that biological entities are made of networks of parts undergoing processes, and that the behaviors of those parts (the nature of those processes) could be encoded in a different chemical substrate. Nor does it suggest that a mechanism that can model arbitrary processes (e.g. Turing machine) cannot model the specific processes going on inside the parts within those networks. The arguments they make against Deutsch-style pancomputationalism hinge on things like determinism/syntactic formal systems that feel completely irrelevant in this context (Turing machines can model stochastic processes, and they are “symbolic” only in the sense that we use symbols to understand/encode them).

There's a fine metaphysical line between "Turing machines can model X" and "X is equivalent to a Turing machine". The authors of the paper argue against the second stance, adopting the opposite one. Personally, I am happy to remain uncommitted.

Hello from a cognitive science perspective I can recommend this paper: https://link.springer.com/article/10.1007/s42113-024-00217-5

@Jack Rusher Thank you for elaborating on this. I appreciate you taking the time to write this. I am very interested in positions against what they argue to make up my mind of how profound their ideas really are, and being left to my own (and at this point fairly biased) interpretations probably means I’m at risk to overlook something. So far, however, I only hear criticisms that seem to be rooted in the fear that the authors take some possibility away. I tend to see their attempt in differentiating algorithmic (both symbolic and stochastic) models from a totally different constraint-based dynamical systems approach that provides characteristics that the computational approach can’t as a revelatory insight that, while perhaps putting algorithmic approaches in their slightly more restricted place, doesn’t at all prevent those approaches from continuing to improve their approximations of reality. Which maps nicely to what’s happening in AI, mostly by increasing complexity in the form of increasing both data and compute. That probably also explains why I find their point about computation rather minor, @Konrad Hinsen. At the same time it paves the way for a new and different path of exploration that tries to integrate human cognition, which I personally find much more refreshing than the “If we just throw enough data and compute at it” approaches that are so en vogue in AI currently. There’s lots of space (and tons more resources, apparently) to build more and better models the “classic” way. I don’t think anyone is trying to take that away. However, there is also a refined understanding of an alternative path emerging, that seems (to me at least) to be based in good science. Of course, I am an enthusiastic amateur at best, and my judgement of how good the science is means nothing to anybody else. But having spent a lot of time with some of the adjacent research, mainly relevance realization (Vervaeke) and constraint-based meta-stability in complex dynamical systems (Juarrero), there is something here that endlessly fascinates me. And it is so sad that it is so difficult to share that fascination and excitement with others.

I seem to completely failed to communicate my point, for which I’m sorry. But I also don’t have enough bandwidth at present to make another pass at it. 🤷🏻‍♂️