I wrote up some overall design direction for zest - https://github.com/jamii/zest/blob/main/docs/rationale.md. Currently bogged down in figuring out how to combine staged compilation and modules (https://github.com/jamii/zest/issues/1), but once I figure that out I can finally start fleshing out the runtime.

I like your wish list! And your so-far-documented design decisions look reasonable. At least in theory, it takes an implementation to find out how they work in practice. Wondering: have you compared your design in detail with existing systems? I think that would help in understanding what exactly you are missing, and to what degree such features can be added without creating problems elsewhere. For example, at first sight your with list and design decisions has a huge overlap with Common Lisp. What I cannot judge easily is how difficult it would be to morph Common Lisp towards full alignment with your goals. Given how well understood Common Lisp is (both good and bad aspects), a detailed comparison should be helpful. For for staged compilation and modules, Racket has probably the most sophisticated approach today, but it may well be more sophisticated that you would like for your system, given that Racket allows each module to be written in a different language.

Ah, I forgot to add the problems from https://www.scattered-thoughts.net/writing/there-are-no-strings-on-me/ to the 'legible' side of the wishlist. I'm explicitly trying to avoid the lisp/smalltalk model where loading code means executing a series of side-effects against a mutable environment.

FWIW: ASDF is not part of CL. It is a program written in CL that defines a "declarative" DSL with lots of hoary edge cases for determining dependences and loading the minimum number of files. It was invented back when machines ran slowly enough that Makefile seemed like a good idea for minimizing CPU cycles. The basic load function defined by the CL language is (LOAD ...) which you can run from the REPL. One can forego the use of ASDF and simply write a .lisp file with a bunch of (LOAD ...)s in it. It is my understanding that modern CL REPLs actually compile every command, then run the compiled command. In essence, if you type (LOAD "filename") at the REPL, the REPL compiles the command into a small binary program, then runs it, actually running the LOAD command built into the Lisp language. ASDF just generates a bunch of (LOAD ...) commands. The definitive reference for CL is CLHS, but it reads like a ton of bricks (it helps to have a degree in Law). If I had to re-learn Lisp, I would probably start with Practical Common Lisp

So still the same model I was complaining about in https://www.scattered-thoughts.net/writing/there-are-no-strings-on-me/.

Most live systems gain their interactivity from mutable environments and late binding. Code loading is an imperative process that can cause arbitrary side effects. The behaviour of the system can depend on what order code was loaded in, and when. There isn't even any guarantee that loading the same code in the same order will reproduce the same system.

Thanks @jamii for this important complement. The tensions between liveness/malleability and immutability are well-known, it will be interesting to see which compromise you will end up adopting. One issue I see with your idea of recompiling the world at every change, for consistency, is data obtained from obsolete code. For me this is a major reason for wanting a live system. The typical situation is that I have some code that I have used to load and process some data, burning a lot of CPU time. And then I discover a bug that makes me fix some of the code that has produced the data. I assume that in your scheme, the right way is to recompute the data after reloading the code, to ensure consistency. But if that takes an hour, the liveness is gone. It should be up to me, the user, to make the choice between consistency and fast feedback. What I expect from an ideal language/runtime is not to guarantee consistency at any cost, but to help me see and then understand critical situations. Current systems (Lisp, Smalltalk) fail mainly in not alerting me, but they are pretty good at letting me deal with the issues once I am aware of them.

Very interesting and principled set of design choices @jamii. When live reloading code, do you support the common hack of allowing new fields to be added to the end of structures? Since you want to pack structures I’m guessing not.

I assume that in your scheme, the right way is to recompute the data after reloading the code, to ensure consistency.

The point of not allowing nominal types is that data is independent of the code that produced it, so you can keep data around while reloading the code, or send it to a different process or save it to disk.

I like the name.

I wanted to just keep the fun parts of programming.