Thinking about Schematics vs. Code. [This began as a short reply to @Stefan regarding my take on electronics schematics in another thread.] https://open.substack.com/pub/programmingsimplicity/p/2024-07-17-thinking-about-the-game?r=1egdky&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true

Thanks @guitarvydas for this detailed explanation with a nice example! There's one aspect I find missing from the discussion: CPUs. Early CPUs, the ones for which our languages, compilers, etc. were designed, have no accessible parallelism (and the very early ones no parallelism at all). There's a single instruction stream. In that universe, function-based programming did not any additional synchronicity constraint. Today, all CPUs and GPUs have some level of parallelism, but it's not under the control of the software. If I have a four-core processor, I can run up to four independent threads. That's a pretty severe constraint on asynchronicity in software as well. So I wonder how we could possibly get to your dream situation of "as many asynchronously working subsystems as our design requires" with today's processors and no software layer for multithreading on top of them.

The issue of CPUs is addressed, but, maybe not with enough brutality There’s approximately 60+ truly-asynchronous processes on the 1972 Pong schematic, but no CPU. The design employs massive parallelism and real internal asynchronousity. In 2024, we consider 8-ish cores to be a marvel. The cores share memory, so they cannot even be truly asynchronous. That’s some 50 years later. Something is wrong with this picture. A CPU was originally meant to be a simple sequencer circuit that was not reentrant. There is no reason to make CPU’s multi-threaded, other than co$t. In 1950, it was too expensive to own more than one CPU, so human brain-power was wasted on inventing and adding software to make CPUs multi-threaded. Today, we have access to very cheap hardware actors - Arduinos, Rasberry Pis, etc. (cheaper yet, if you discard Linux and reclaim internal chip space). N.B. “central” is a bad word these days. We probably want DPUs (distributed processing units). I previously tried to write about hardware actors in https://guitarvydas.github.io/2024/02/17/Hardware-Actors.html. So, one might imagine a reprogrammable electronic machine (avoid the use of the word “compute-er”) to be a collection of 1,000s of cheap hardware actors on-a-chip, where only one of them is a big honking Linux blob for running existing bloatware and for backwards compatibility. @Konrad Hinsen

Thanks @guitarvydas, that confirms my suspicion. Are you aware of any projects to create such massively asynchronous devices? All I remember having seen in this space is the MIT Connection Machine.

I am aware of, but, haven’t deeply explored… Chuck Moore’s GA144 (https://www.greenarraychips.com), LLMs, internet, Dave Ackley’s T2 Tile project, the human body, blockchain, transputers (https://en.wikipedia.org/wiki/Transputer), quantum computing, neural nets (and Bart Kosko?). LLMs are based on simple synchronous technology, but, to my understanding the real secret sauce is the zillions of weights derived from access to the New Library of Alexandria and a massively parallel version of the game of Plinko. @Konrad Hinsen

Interesting case study! (I heard about Pong circuits but haven't seen them) I wonder what you'd say about Scratch/Snap!'s programming model — it's a bit closer to circuits by running many small loops in parallel, each belonging to a single sprite. E.g. here is someone's 1-player pong. The UI doesn't offer "everything on one page" view, you have to click specific sprites (Ball, Paddle, ...) in bottom right, but for the most part that helps understanding. (There is 🗋->Export summary... but it loses the spatial organization)

@guitarvydas I prefer to stay in the realm of digital computation, so I accept neither the human body nor quantum computing. But the other examples are worth exploring! I agree that LLMs deserve much better adapted hardware than today's GPUs, but I suspect that everyone is so busy training their LLM to beat their competitors' LLM that nobody has any time to think about the fundamentals.

@Beni Cherniavsky-Paskin Interesting thoughts. “… but I've seen too many concurrency bugs …” I’ve come to the conclusion that a lot of concurrency bugs are due to self-flagellation cause by the use of the wrong notation / paradigm. FTR, I emphasize async because I perceive it to be underemphasized, but, I think that a hybrid of async+sync is the goal. To me, async enhances composition. …more mulling-over required…