@Konrad Hinsen's link https://feelingofcomputing.slack.com/archives/C5U3SEW6A/p1776099082547179 spawned this thought, among others, as I watched:

TL;DR — Old code is dying. The code isn't the problem. The business rules embedded in it are. LLMs can extract those rules into a factbase. A factbase can be rewritten into any target language. This is Design Recovery, and it is the right job for LLMs.

LLMs as Design Recovery Engines

Running coding agents backwards is a possibility. But not a trivial problem to solve. The "factbase" is what, exactly? For the code I deal with, it would have to include mathematical relations between numbers. The code being an approximation (floats etc) to the relations that one would like to see recovered. Not impossible, given that the number of numerical algorithms is limited and an LLM could have seen them all. But the structure of the factbase is part of the design that needs to be recovered. I don't know if an LLM can deal with that. And then there is the eternal question with LLMs: do you trust the output to be correct, or at least good enough? Much of the legacy code we are using has no tests, no specification, etc. It's just the code, plus input/output combinations (which are easy to produce).

I always make the LLMs work for their trust. They often can do reasonably well if you encourage them and force them to keep it real.

The problem with legacy code is that you don't know what is real. Usually there's nobody left who actually knows what the code does. The code is the only "source of truth". So you need to trust your LLM blindly.

And, even worse, code that is in production probably has last-minute bandaids to fix edge-cases and loopholes that don’t fit the “general architecture” of the rest of the code. At some point, the old code will need to be lifted into the present/future. I tend to think about problem solving thusly: (1) understand the problem, (2) solve it. Code is but a recipe of little steps that are combined together to produce a result. “Design” (architecture) is the extreme other end of this - it is the underlying thinking that someone chose to encode in little steps. The code gets written down, but, the design intent (DI) tends to not get written down and becomes obfuscated by details expressed in tiny steps. And, it’s not even a 1:1 mapping, the same DI could be expressed in multiple ways and combinations of steps. The factbase idea is to understand what exists in tiny steps, then iterate and infer bigger steps from patterns and idioms in the code (like manually disassembling machine code and trying to understand what it means). This can’t be done in one fell swoop, by only one entity. A “fact” is just a triple:

relation(subject, object)

. Say, something like

inscope(“x”,”f”)

,

type(“x”,int16)

, etc. A factbase of such facts doesn’t tell you much about the DI, but, someone might “see” a pattern in one place, then write a Prolog-like rule that says “if you find <a> followed by <b> then infer that this is a <c>” (it’s a lot like type checking, but deals with higher-level project-specific concepts). Repeat. After several iterations, more of the DI becomes clear(er). Discard or ignore the very low-level stuff (eg. delete <a>s and <b>s, leaving only <c>s), then see if the remaining high-level stuff can be converted into modern form. No guarantee that this would work 100%, but, it’s better than nothing. I would find it easier to trust an LLM (or another human) to do this in little steps than blindly trusting that they did the whole conversion in an opaque manner. Can the result be trusted? Not without lots of testing…

@guitarvydas Thanks for the details on your factbase! I had assumed it was something at a higher level, trying to deduce design intent.

To further belabour this point… This is, also, how I view “compiling” diagrams to code. Take a mess of graphical detail (.xml, .graphml, .svg, .json) prune out the 95% that isn’t semantically interesting. Apply various tools (jq, swipl, python, js, LLMs, etc.) to the remaining 5% and iterate until you can emit Python (or whatever) from what has been inferred. Easy, especially if you treat it in stages instead of trying to force it into one-fell-swoopism.