I've been thinking about the Viable System Model as a framework for AI agents since I came across it, mentioned it here actually: https://futureofcoding.slack.com/archives/C5T9GPWFL/p1748381760844169 Finally put together some rubygems to explore it -

vsm

the framework and

airb

a CLI-based agent built with it and wrote up some more detailed thoughts about them on my newsletter: https://worksonmymachine.substack.com/p/the-system-inside-the-system

> Sometimes metaprogramming is dismissed as just being used for the sake of being clever, but this is exactly the kind of runtime flexibility that makes self-modifying systems possible. The kind that would let an AI agent literally write its own capabilities while it's running. But does an LLM-powered agent actually have to facilitate symbolic rule-making in order to coordinate with other parts of the system? I understand an agent might set parameters for other agents, but LLMs seem capable of developing their own non-human, non-deterministic languages to reason internally. Or would meta-programming facilitate a human in the loop?

Hm yeah, I don't think this is an either/or, it's more about providing a shared, modifiable structure that an LLM do whatever it likes inside and then ways to incorporate it. It needs to do symbolic rule-making at some level to interact with the rest of the system, right? The other thing is that LLMs do really well when you provide examples for them to work off of and constrain what they can do, so once you have a structure you can feed into an LLM with something like: Here's the interface and 5 working examples of {THING} -> generate me a new {THING} that does {new behavior} You have a really high likelihood of getting exactly what you want and being able to use it right away, but there's still a chance it needs some modifications before you can use it. But thinking more about your question at the end, maybe this is all about human in the loop...I was originally just thinking about it at the first level: just being able to approve/reject an action an agent wants to take and providing an interface for that. But there's a second aspect: being able to ask questions to see and inspect the structure the LLM has built and having the language to do it... And maybe even a third: having a common language to modify the system with the human being able to say "add a new governance rule to this sub-system" or an agent being able to say something like "I need a new sub-system for xyz capability" and they're communicating in the same structural language.

Creating the language between human and AI, to put a structure around the "their own non-human, non-deterministic languages to reason internally"

Indeed. I'm not sure how much introspection will ever be possible on these models. But bounding them (for safety) and then collaborating in a metaprogramming environment could be expressive and powerful.

You've given me a bunch of new things for my reading list, I'll say that! Need to look more into the viable systems model now. The thing that I worry about with granting code execution abilities to LLMs is that the sandboxing abilities we typically use for arbitrary code are pretty coarse-grained. Especially if you add metaprogramming to the mix; it's all well and good to have a framework for controlling access to the files on your filesystem, but if the agent can modify itself to use

File.open

directly it can bypass your framework. That said, I think you're right that being able to do metaprogramming will likely be important to make better use of them! Which makes it all the more important to figure out how to do that safely. I think the most promising approach there is using object-capabilities, which (to very much oversimplify) involves having a strict sandbox by default and selectively passing in capabilities for fine-grained authority. Your agent can write whatever code it wants, but if all that code only has access to the capabilities that are locally in-scope at that point, you're in a much better place I think. This is the most concise but detailed explanation I've seen of what that might look like at a language level. The trouble, of course, is that it's difficult to do that in most programming languages, and the languages I'm aware of that are working on it aren't done yet. 🙂

Hah you know what's really interesting about that...you can kind of look at the evolution of our practices and industry as this pendulum swinging between unsafe and safe (in the past I've heard informal vs formal) - where the informal/unsafe side rushes out ahead and gets some early wins and then once things get big enough the safe side comes in and makes it stable on each new platform....if you squint its kind of like smalltalk -> c++/java -> ruby/javascript -> typescript/rust

Interesting assertion regarding this pendulum swing. 🤔

@D. Schmudde I've mentioned that to a few people and they've said they never thought about it like that...is there something in particular that jumps out to you about it? I could probably expand on that idea in a future post...

Well there is going to be a lot of 'yeah-but-what-about-x's regarding the languages you put on that timeline. But I found it really provocative, @Scott. There is also this issue of patterns vs. proofs. Here I'm thinking the adoption of design patterns vs. compiler guarantees. Or perhaps Kay vs. Dijkstra (choosing those two on the spectrum because of Kay's famous quip). So I guess there has to be some sociological component with general trend lines that makes claims about the rise and fall of certain priorities as embodied or measured by languages and their design priorities. How else could you demonstrate the claims? Doing some static code analysis of repos in https://www.softwareheritage.org/? Yikes.

Haha I love the Kay vs Dijkstra framing 🙂 never put that together, but I heard someone say once that if you were taught OO with inheritance first, you were taught by a formalist, if polymorphism was first you were taught by an informalist...it would make sense that Kay and Dijkstra would be the patron saints of each side